Book Image

Artificial Vision and Language Processing for Robotics

By : Álvaro Morena Alberola, Gonzalo Molina Gallego, Unai Garay Maestre
Book Image

Artificial Vision and Language Processing for Robotics

By: Álvaro Morena Alberola, Gonzalo Molina Gallego, Unai Garay Maestre

Overview of this book

Artificial Vision and Language Processing for Robotics begins by discussing the theory behind robots. You'll compare different methods used to work with robots and explore computer vision, its algorithms, and limits. You'll then learn how to control the robot with natural language processing commands. You'll study Word2Vec and GloVe embedding techniques, non-numeric data, recurrent neural network (RNNs), and their advanced models. You'll create a simple Word2Vec model with Keras, as well as build a convolutional neural network (CNN) and improve it with data augmentation and transfer learning. You'll study the ROS and build a conversational agent to manage your robot. You'll also integrate your agent with the ROS and convert an image to text and text to speech. You'll learn to build an object recognition system using a video. By the end of this book, you'll have the skills you need to build a functional application that can integrate with a ROS to extract useful information about your environment.
Table of Contents (12 chapters)
Artificial Vision and Language Processing for Robotics
Preface

Chapter 7: Build a Text-Based Dialogue System (Chatbot)


Activity 7: Create a Conversational Agent to Control a Robot

Solution

  1. Open up your Google Colab interface.

  2. Create a folder for the book and download the utils, responses, and training folder from Github and upload it in the folder.

  3. Import drive and mount it as follows:

    from google.colab import drive
    drive.mount('/content/drive')

    Note

    Every time you use a new collaborator, mount the drive to the desired folder.

  4. Once you have mounted your drive for the first time, you will need to enter the authorization code mentioned by clicking on the URL mentioned by Google and press the Enter key on your keyboard:

    Figure 7.28: Image displaying the Google Colab authorization step

  5. Now that you have mounted the drive, you need to set the path of the directory.

    cd /content/drive/My Drive/C13550/Lesson07/Activity01

    Note

    The path mentioned in step 5 may change as per your folder setup on Google Drive. The path will always begin with cd /content/drive/My Drive/

  6. Import the chatbot_intro file:

    from chatbot_intro import *
  7. Define the GloVe model:

    filename = '../utils/glove.6B.50d.txt.word2vec'
    model = KeyedVectors.load_word2vec_format(filename, binary=False)
  8. List the responses and training sentences files:

    intent_route = 'training/'
    response_route = 'responses/'
    intents = listdir(intent_route)
    responses = listdir(response_route)
    
    print("Documents: ", intents)

    Figure 7.29: A list of intent documents

  9. Create document vectors:

    doc_vectors = np.empty((0,50), dtype='f')
    for i in intents:
        with open(intent_route + i) as f:
            sentences = f.readlines()
        sentences = [x.strip() for x in sentences]
        sentences = pre_processing(sentences)
    doc_vectors= np.append(doc_vectors,doc_vector(sentences,model),axis=0)
    
    print("Shape of doc_vectors:",doc_vectors.shape)
    print(" Vector representation of backward.txt:\n",doc_vectors)

    7.30: Shape of doc_vectors

  10. Predict the intent:

    user_sentence = "Look to the right"
    
    user_sentence = pre_processing([user_sentence])
    user_vector = doc_vector(user_sentence,model).reshape(100,)
    intent = intents[select_intent(user_vector, doc_vectors)]
    intent

    7.31: Predicted intent

Congratulations! You finished the activity. You can add more intents if you want to and train the GloVe model to achieve better results. By creating a function with all the code, you programmed and developing a movement node in ROS, you can order your robot to make movements and turn around.