Book Image

Artificial Vision and Language Processing for Robotics

By : Álvaro Morena Alberola, Gonzalo Molina Gallego, Unai Garay Maestre
Book Image

Artificial Vision and Language Processing for Robotics

By: Álvaro Morena Alberola, Gonzalo Molina Gallego, Unai Garay Maestre

Overview of this book

Artificial Vision and Language Processing for Robotics begins by discussing the theory behind robots. You'll compare different methods used to work with robots and explore computer vision, its algorithms, and limits. You'll then learn how to control the robot with natural language processing commands. You'll study Word2Vec and GloVe embedding techniques, non-numeric data, recurrent neural network (RNNs), and their advanced models. You'll create a simple Word2Vec model with Keras, as well as build a convolutional neural network (CNN) and improve it with data augmentation and transfer learning. You'll study the ROS and build a conversational agent to manage your robot. You'll also integrate your agent with the ROS and convert an image to text and text to speech. You'll learn to build an object recognition system using a video. By the end of this book, you'll have the skills you need to build a functional application that can integrate with a ROS to extract useful information about your environment.
Table of Contents (12 chapters)
Artificial Vision and Language Processing for Robotics
Preface

Robot Positioning


By using one of the internal sensors mentioned in the preceding section, we can calculate the position of a robot after a certain amount of displacement. This kind of calculation is called odometry and can be performed with the help of the encoders and the information they provide. When discussing this technique, it's important to keep in mind the main advantage and disadvantage:

  • Advantage: It can be used to compute the robot's position without external sensors, which would result in a robot's design being much cheaper.

  • Disadvantage: The final position calculation is not completely accurate because it depends on the state of the ground and wheels.

Now, let's see how to perform this kind of calculation step by step. Supposing we have a robot that moves on two wheels, we would proceed as follows:

  1. First, we should compute the distance completed by the wheels, which is done by using the information extracted from the engine's encoders. In a two-wheeled robot, a simple schema could be like this:

    Figure 1.7: Schema of a two-wheeled robot's movement

    The distance traveled by the left wheel is the dotted line in Figure 1.6 tagged with DL, and DR represents the right wheel.

  2. To calculate the linear displacement of the center point of the wheel's axis, we will need the information calculated in the first step. Using the same simple schema, Dc would be the distance:

    Note

    If you were working with multi-axial wheels, you should study how the axes are distributed first and then compute the distance traveled by each axis.

    Figure 1.8: Schema of a two-wheeled robot's movement (2)

  3. To calculate the robot's rotation angle, we will need the final calculation obtained in the first step. The angle named α is the one we are referring to:

    Figure 1.9: Schema of a two-wheeled robot's movement (3)

    As shown in the diagram, α would be 90º in this case, which means that the robot has rotated a specific number of degrees.

  4. Once you've obtained all the information, it is possible to perform a set of calculations (which will be covered in the next section) to obtain the coordinates of the final position.

Exercise 1: Computing a Robot's Position

In this exercise, we are using the previous process to compute the position of a two-wheeled robot after it has moved for a certain amount of time. First, let's consider the following data:

  • Wheel diameter = 10 cm

  • Robot base length = 80 cm

  • Encoder counts per lap = 76

  • Left encoder counts per 5 seconds = 600

  • Right encoder counts per 5 seconds = 900

  • Initial position = (0, 0, 0)

  • Moving time = 5 seconds

Note

Encoder counts per lap is the measurement unit that we use to compute the amount of energy generated by an encoder after one lap on its axis. For example, in the information provided above we have the left encoder, which completes 600 counts in 5 seconds. We also know that an encoder needs 76 counts to complete a lap. So, we can deduce that, in 5 seconds, the encoder will complete 7 laps (600/76). This way, if we would know the energy generated by 1 lap, we know the energy generated in 5 seconds.

For the initial position, the first and second numbers refer to the X and Y coordinates, and the last number refers to the rotation angle of the robot. This data is a bit relative, as you have to imagine where the axes begin.

Now, let's follow these steps:

  1. Let's compute the completed distance of each wheel. We first compute the number of counts that each encoder performs during the time it moves. This can be easily computed by dividing the total movement by the given encoder time and multiplying it by the number of counts of each encoder:

    (Moving time / Encoder time) * Left encoder counts:

    (5 / 5) * 600 = 600 counts

    (Moving time / Encoder time) * Right encoder counts:

    (5 / 5) * 900 = 900 counts

    Once this has been calculated, we can use this data to obtain the total distance. As wheels are circular, we can compute each wheel's completed distance as follows:

    [2πr / Encoder counts per lap] * Total left encoder counts:

    (10π/76) * 600 = 248.02 cm

    [2πr / Encoder counts per lap] * Total right encoder counts:

    (10π/76) * 900 = 372.03 cm

  2. Now compute the linear displacement of the center point of the wheels' axis. This can be done with a simple calculation:

    (Left wheel distance + Right wheel distance) / 2:

    (248.02 + 372.03) / 2 = 310.03 cm

  3. Compute the robot's rotation angle. To do this, you can calculate the difference between the distance completed by each wheel and divide it by the base length:

    (Right wheel distance – Left wheel distance) / Base length:

    (372.03 - 248.02) / 80 = 1.55 radians

  4. Finally, we can compute the final position by calculating each component separately. These are the equations to use to obtain each component:

    Final x position = initial x position + (wheels' axis displacement * rotation angle cosine):

    0 + (310.03 * cos (1.55)) = 6.45

    Final y position = initial y position + (wheels' axis displacement * rotation angle cosine):

    0 + (310.03 * sin (1.55)) = 309.96

    Final robot rotation = initial robot rotation + robot rotation angle:

    0 + 1.55= 1.55

So, after this process, we can conclude that the robot has moved from (0, 0, 0) to (6.45, 309.96, 1.55).

How to Work with Robots

Like any other software development, the process of implementing applications and programs for robots can be done many different ways.

In the upcoming chapters, we will use frameworks and technologies that make it possible to abstract a specific problem and develop a solution that is easily adaptable to all kinds of robots and devices. In this book, we will be using Robot Operating System (ROS) for this purpose.

Another issue to consider before we start working with robots is the programming language to use. You surely know and have used some languages, but which one is the most appropriate? The real answer to this question is that there is no specific language; it always depends on the problem at hand. But during our book, and due to the kinds of activities that we will work on, we are going to use Python, which, as you may know, is an interpreted, high-level, general-purpose programming language that is used in AI and robotics.

By using Python, as with other languages, you can develop any functionality you want your robot to have. For example, you could give your robot the simple behavior of greeting when it detects a person. You could also program a more complex functionality, for example, to dance when it “hears” music.

Now we are going to go through some exercises and activities that will introduce you to Python for robotics, if you haven't used it before.

Exercise 2: Computing the Distance Traveled by a Wheel with Python

In this exercise, we are going to implement a simple Python function for computing the distance covered by a wheel using the same process that we performed in Exercise 1, Computing a Robot's Position. These are the steps to be followed:

  1. Import the required resources. In this case, we are going to use the number π:

    from math import pi
  2. Create the function with the parameters. To compute this distance, we will need the following:

    Wheel diameter in centimeters

    Encoder counts per lap

    Number of seconds used to measure encoders' counts

    Wheel encoder counts during the given number of seconds

    Total time of movement

    This is the function definition:

    def wheel_distance(diameter, encoder, encoder_time, wheel, movement_time):
  3. Begin with the implementation of the function. First, compute the distance measured by the encoder:

    time = movement_time / encoder_time
    wheel_encoder = wheel * time
  4. Transform the obtained distance from above to the one we expect, which would be the distance traveled by the wheel:

    wheel_distance = (wheel_encoder * diameter * pi) / encoder
  5. Return the final value:

    return wheel_distance
  6. You can finally check whether the function is correctly implemented by passing values to it and make the corresponding calculation manual:

    wheel_distance(10, 76, 5, 400, 5)

    This function call should return 165.34698176788385.

    Figure 1.10: Final distance covered by the wheel

The output in your notebook should look like this:

Exercise 3: Computing Final Position with Python

In this exercise, we use Python to compute the final position of a robot, given its initial position, its distance completed by the axis, and its rotation angle. You can do it by following this process:

  1. Import the sine and cosine functions:

    from math import cos, sin
  2. Define the function with the required parameters:

    The robot's initial position (coordinates)

    The completed distance by the robot's central axis

    The angle variation from its initial point:

    def final_position(initial_pos, wheel_axis, angle):

    Set a function by coding the formulas used in Exercise 1: Computing a Robot's Position.

    They can be coded like this:

    final_x = initial_pos[0] + (wheel_axis * cos(angle))
    final_y = initial_pos[1] + (wheel_axis * sin(angle))
    final_angle = initial_pos[2] + angle

    Note

    As you may guess by observing this implementation, the initial position has been implemented using a tuple, where the first element matches the “X”, the second with the “Y”, and the last with the initial angle.

    Return the final value by creating a new tuple with the results:

    return(final_x, final_y, final_angle)
  3. Again, you can test the function by calling it with all the arguments and computing the result by hand:

    final_position((0,0,0), 125, 1)

    The preceding code returns the following result:

    (67.53778823351747, 105.18387310098706, 1)

    Here, you can see the whole implementation and an example of a function call:

    Figure 1.11: Final position of the robot computed

Activity 1: Robot Positioning Using Odometry with Python

You are creating a system that detects the position of a robot after moving for a certain amount of time. Develop a Python function that gives you the final position of a robot after receiving the following data:

  • Wheels diameter in centimeters = 10 cm

  • Robot base length = 80 cm

  • Encoders counts per lap = 76

  • Number of seconds used to measure encoders' counts = 600

  • Left and right encoder counts during the given number of seconds = 900

  • Initial position = (0, 0, 0)

  • Movement duration in seconds = 5 seconds

Note

The functions implemented in the previous exercises can help you to complete the activity. There are a few steps that you can use to proceed ahead with this activity.

Following these steps will help you to complete the exercises:

  1. First, you need to compute the distance completed by each wheel.

  2. To move on, you need to calculate the distance completed by the axis.

  3. Now compute the robot's rotation angle.

  4. Then calculate the final position of the robot.

    Fig 1.12: Final position of a robot computed with the activity's Python function

The output would look like this:

Note

The solution for this activity can be found on page 248.