Tensors are the basic components in TensorFlow. A tensor is a multidimensional collection of data elements. It is generally identified by shape, type, and rank. Rank refers to the number of dimensions of a tensor, while shape refers to the size of each dimension. You may have seen several examples of tensors before, such as in a zero-dimensional collection (also known as a scalar), a one-dimensional collection (also known as a vector), and a two-dimensional collection (also known as a matrix).

A scalar value is a tensor of rank 0 and shape []. A vector, or a one-dimensional array, is a tensor of rank 1 and shape [number_of_columns] or [number_of_rows]. A matrix, or a two-dimensional array, is a tensor of rank 2 and shape [number_of_rows, number_of_columns]. A three-dimensional array is a tensor of rank 3. In the same way, an n-dimensional array is a tensor of rank n.

A tensor can store data of one type in all of its dimensions, and the data type of a tensor is the same as the data type of its elements.

The following are the most commonly used data types in TensorFlow:

Tensors can be created in the following ways:

Let's explore different ways of creating Tensors.

The constant valued tensors are created using the tf.constant() function, and has the following definition:

tf.constant(
  value,
  dtype=None,
  shape=None,
  name='const_name',
  verify_shape=False
  )

Let's create some constants with the following code:

const1=tf.constant(34,name='x1')
const2=tf.constant(59.0,name='y1')
const3=tf.constant(32.0,dtype=tf.float16,name='z1')

Let's take a look at the preceding code in detail:

Let's print the constants const1, const2, and const3:

print('const1 (x): ',const1)
print('const2 (y): ',const2)
print('const3 (z): ',const3)

When we print these constants, we get the following output:

const1 (x):  Tensor("x:0", shape=(), dtype=int32)
const2 (y):  Tensor("y:0", shape=(), dtype=float32)
const3 (z):  Tensor("z:0", shape=(), dtype=float16)

To print the values of these constants, we can execute them in a TensorFlow session with the tfs.run() command:

print('run([const1,const2,c3]) : ',tfs.run([const1,const2,const3]))

We will see the following output:

run([const1,const2,const3]) : [34, 59.0, 32.0]

The TensorFlow library contains several built-in operations that can be applied on tensors. An operation node can be defined by passing input values and saving the output in another tensor. To understand this better, let's define two operations, op1 and op2:

op1 = tf.add(const2, const3)
op2 = tf.multiply(const2, const3)

Let's print op1 and op2:

print('op1 : ', op1)
print('op2 : ', op2)

The output is as follows, and shows that op1 and op2 are defined as tensors:

op1 :  Tensor("Add:0", shape=(), dtype=float32)
op2 :  Tensor("Mul:0", shape=(), dtype=float32)

To print the output from executing these operations, the op1 and op2 tensors have to be executed in a TensorFlow session:

print('run(op1) : ', tfs.run(op1))
print('run(op2) : ', tfs.run(op2))

The output is as follows:

run(op1) :  91.0
run(op2) :  1888.0

Some of the built-in operations of TensorFlow include arithmetic operations, math functions, and complex number operations.

While constants store the value at the time of defining the tensor, placeholders allow you to create empty tensors so that the values can be provided at runtime. The TensorFlow library provides the tf.placeholder() function with the following signature to create placeholders:

tf.placeholder(
  dtype,
  shape=None,
  name=None
  )

As an example, let's create two placeholders and print them:

p1 = tf.placeholder(tf.float32)
p2 = tf.placeholder(tf.float32)
print('p1 : ', p1)
print('p2 : ', p2)

The following output shows that each placeholder has been created as a tensor:

p1 :  Tensor("Placeholder:0", dtype=float32)
p2 :  Tensor("Placeholder_1:0", dtype=float32)

Let's define an operation using these placeholders:

mult_op = p1 * p2

In TensorFlow, shorthand symbols can be used for various operations. In the preceding code, p1 * p2 is shorthand for tf.multiply(p1,p2):

print('run(mult_op,{p1:13.4, p2:61.7}) : ',tfs.run(mult_op,{p1:13.4, p2:61.7}))

The preceding command runs mult_op in the TensorFlow session and feeds the values dictionary (the second argument to the run() operation) with the values for p1 and p2.

The output is as follows: 

run(mult_op,{p1:13.4, p2:61.7}) :  826.77997

We can also specify the values dictionary by using the feed_dict parameter in the run() operation:

feed_dict={p1: 15.4, p2: 19.5}
print('run(mult_op,feed_dict = {p1:15.4, p2:19.5}) : ',
      tfs.run(mult_op, feed_dict=feed_dict))

The output is as follows:

run(mult_op,feed_dict = {p1:15.4, p2:19.5}) :  300.3

Let's look at one final example, which is of a vector being fed to the same operation:

feed_dict={p1: [2.0, 3.0, 4.0], p2: [3.0, 4.0, 5.0]}
print('run(mult_op,feed_dict={p1:[2.0,3.0,4.0], p2:[3.0,4.0,5.0]}):',
      tfs.run(mult_op, feed_dict=feed_dict))

The output is as follows:

run(mult_op,feed_dict={p1:[2.0,3.0,4.0],p2:[3.0,4.0,5.0]}):[  6.  12.  20.]

The elements of the two input vectors are multiplied in an element-wise fashion.

Tensors can be created from Python objects such as lists, NumPy arrays, and pandas DataFrames. To create tensors from Python objects, use the tf.convert_to_tensor() function with the following definition:

tf.convert_to_tensor(
  value,
  dtype=None,
  name=None,
  preferred_dtype=None
  )

Let's practice doing this by creating some tensors and printing their definitions and values:

tf_t=tf.convert_to_tensor(5.0,dtype=tf.float64)

print('tf_t : ',tf_t)
print('run(tf_t) : ',tfs.run(tf_t))

The output is as follows: 

tf_t : Tensor("Const_1:0", shape=(), dtype=float64)
run(tf_t) : 5.0

a1dim = np.array([1,2,3,4,5.99])
print("a1dim Shape : ",a1dim.shape)

tf_t=tf.convert_to_tensor(a1dim,dtype=tf.float64)

print('tf_t : ',tf_t)
print('tf_t[0] : ',tf_t[0])
print('tf_t[0] : ',tf_t[2])
print('run(tf_t) : \n',tfs.run(tf_t))

The output is as follows:

a1dim Shape :  (5,)
tf_t :  Tensor("Const_2:0", shape=(5,), dtype=float64)
tf_t[0] :  Tensor("strided_slice:0", shape=(), dtype=float64)
tf_t[0] :  Tensor("strided_slice_1:0", shape=(), dtype=float64)
run(tf_t) : 
 [ 1.    2.    3.    4.    5.99]

a2dim = np.array([(1,2,3,4,5.99),
                  (2,3,4,5,6.99),
                  (3,4,5,6,7.99)
                 ])
print("a2dim Shape : ",a2dim.shape)

tf_t=tf.convert_to_tensor(a2dim,dtype=tf.float64)

print('tf_t : ',tf_t)
print('tf_t[0][0] : ',tf_t[0][0])
print('tf_t[1][2] : ',tf_t[1][2])
print('run(tf_t) : \n',tfs.run(tf_t))

The output is as follows:

a2dim Shape :  (3, 5)
tf_t :  Tensor("Const_3:0", shape=(3, 5), dtype=float64)
tf_t[0][0] :  Tensor("strided_slice_3:0", shape=(), dtype=float64)
tf_t[1][2] :  Tensor("strided_slice_5:0", shape=(), dtype=float64)
run(tf_t) : 
 [[ 1.    2.    3.    4.    5.99]
  [ 2.    3.    4.    5.    6.99]
  [ 3.    4.    5.    6.    7.99]]

a3dim = np.array([[[1,2],[3,4]],
                  [[5,6],[7,8]]
                 ])
print("a3dim Shape : ",a3dim.shape)

tf_t=tf.convert_to_tensor(a3dim,dtype=tf.float64)

print('tf_t : ',tf_t)
print('tf_t[0][0][0] : ',tf_t[0][0][0])
print('tf_t[1][1][1] : ',tf_t[1][1][1])
print('run(tf_t) : \n',tfs.run(tf_t))

The output is as follows:

a3dim Shape :  (2, 2, 2)
tf_t :  Tensor("Const_4:0", shape=(2, 2, 2), dtype=float64)
tf_t[0][0][0] :  Tensor("strided_slice_8:0", shape=(), dtype=float64)
tf_t[1][1][1] :  Tensor("strided_slice_11:0", shape=(), dtype=float64)
run(tf_t) : 
 [[[ 1.  2.][ 3.  4.]]
  [[ 5.  6.][ 7.  8.]]]

In the previous sections, we learned how to define tensor objects of different types, such as constants, operations, and placeholders. The values of parameters need to be held in an updatable memory location while building and training models with TensorFlow. Such updatable memory locations for tensors are known as variables in TensorFlow.

To summarize this, TensorFlow variables are tensor objects in that their values can be modified during the execution of the program.

Although tf.Variable seems to be similar to tf.placeholder, they have certain differences. These are listed in the following table:

In TensorFlow, a variable can be created with the API function tf.Variable(). Let's look at an example of using placeholders and variables and create the following model in TensorFlow:

w = tf.Variable([.3], tf.float32)
b = tf.Variable([-.3], tf.float32)

x = tf.placeholder(tf.float32)
y = w * x + b

print("w:",w)
print("x:",x)
print("b:",b)
print("y:",y)

The output depicts the type of nodes as Variable, Placeholder, or operation node, as follows:

w: <tf.Variable 'Variable:0' shape=(1,) dtype=float32_ref>
x: Tensor("Placeholder_2:0", dtype=float32)
b: <tf.Variable 'Variable_1:0' shape=(1,) dtype=float32_ref>
y: Tensor("add:0", dtype=float32)

The preceding output indicates that x is a Placeholder tensor, y is an operation tensor, and that w and b are variables with a shape of (1,) and a data type of float32.

The variables in a TensorFlow session have to be initialized before they can be used. We can either initialize a single variable by running its initializer operation or we can initialize all or a group of variables.

For example, to initialize the w variable, we can use the following code:

tfs.run(w.initializer)

TensorFlow provides a convenient function that can initialize all of the variables:

tfs.run(tf.global_variables_initializer())

The global convenience function for initializing these variables can be executed in an alternative way. Instead of executing inside the run() function of a session object, the run function of the object returned by the initializer function itself can be executed:

tf.global_variables_initializer().run()

After the variables have been initialized, execute the model to get the output for the input values of x = [1,2,3,4]:

print('run(y,{x:[1,2,3,4]}) : ',tfs.run(y,{x:[1,2,3,4]}))

The output is as follows:

run(y,{x:[1,2,3,4]}) :  [ 0.          0.30000001  0.60000002  0.90000004]

TensorFlow provides various functions to generate tensors with pre-populated values. The generated values from these functions can be stored in a constant or variable tensor. Such generated values can also be provided to the tensor constructor at the time of initialization.

As an example, let's generate a 1-D tensor that's been pre-populated with 100 zeros:

a=tf.zeros((100,))
print(tfs.run(a))

Some of the TensorFlow library functions that populate these tensors with different values at the time of their definition are listed as follows:

If a variable is defined with a name that has already been used for another variable, then an exception is thrown by TensorFlow. Thetf.get_variable() function makes it convenient and safe to create a variable in place of using thetf.Variable() function. The tf.get_variable() function returns a variable that has been defined with a given name. If the variable with the given name does not exist, then it will create the variable with the specified initializer and shape.

Consider the following example:

w = tf.get_variable(name='w',shape=[1],dtype=tf.float32,initializer=[.3])
b = tf.get_variable(name='b',shape=[1],dtype=tf.float32,initializer=[-.3])

The initializer can either be a list of values or another tensor. An initializer can also be one of the built-in initializers. Some of these are as follows:

The tf.get_variable() function only returns the global variables when the code is run across multiple machines in distributed TensorFlow. The local variables can be retrieved by using the tf.get_local_variable() function.

Now that we have learned how to define tensors, constants, operations, placeholders, and variables, let's learn about the next level of abstraction in TensorFlow that combines these basic elements to form a basic unit of computation: the computation graph.