The basics of programming

Computer programming varies from language to language in terms of implementation, but there are remarkable similarities among these languages in how their internal logic works. These programming basics are applicable for all programming languages, with specific code implementations shown in Python:

x=0
y=1
xy = x+y
xy_str = str(xy)

str_var = "string"
int_var = 4
float_var = 5.7
list_var = [45,43,24]
tuple_var = (87,'a',34)
dict_var = {'key':'value'}
bool_var = True

for item in datalist:
    print(item)
x=0
while x < 1:
    x+=1

counter = 0
list_var = [34,54,23,54]
for item in list_var:
    print(item, counter)
    counter += 1 
l_var = [34,54,23,54]
for c,i in enumerate(l_var):
    print(i, c)

list_var = [1,'1',1.0]
for item in list_var:
  if type(item) == type(0):
    print('Integer')
  elif type(item) == type('a'):
    print('String')
  else:
    print('Float')

list_var = ['s','m','t']
m_var = list_var[0]
name_var = "logan"
l_var = name_var[0]

# This is a comment
x = 0 # also a comment

>>> str_var = 'red"
  File "<stdin>", line 1
    str_var = 'red"
                  ^
SyntaxError: EOL while scanning string literal

In the following sections, we take a look at some of these in more detail, as well as introducing you to functions and classes.

Variables

Variables are used to assign objects to labels or identifiers. They are used to keep track of pieces of data, to organize the flow of the data through the script, and to help programmers read the script.

variable = 1 # a variable assignment

We recommend you use descriptive variables that are neither too long nor too short. When variables are too short, they can become confusing to read. When they are too long, they can be confusing to write. Using underscores to separate words in variables is a common practice.

Variable formatting rules

Variables must start with a letter. They cannot start with a number or other symbol, otherwise a SyntaxError will occur. However, numbers and underscores can be used in them:

>>> 2var = 34
  File "<stdin>", line 1
    2var = 34
     ^
SyntaxError: invalid syntax
>>> two_var = 34
>>> two_var
34

Assigned to vs is equal to (value comparison)

In Python, variables are assigned to an object using the equals sign (=). To check if a value is equal to another value (in other words, to compare them), use a double equals sign (==):

variable = 1 # a variable assignment
variable == 1 # a comparison

Data types

The data type of a variable determines its behavior. For instance, the character 5 could be an integer type (5), a float (5.0), or a string ("5"). Each version of 5 will have different available tools, such as the replace() method for strings, which can replace characters in the string with other characters.

The following table presents key data types in Python, along with the corresponding data type object in Python:

Checking the data type

To check the data type of a Python variable, use the type() function:

>>> x = 0
>>> type(x)
<class 'int'>

Strings

All text data is represented as the String data type in Python. These are known as strings. Common data stored as strings includes names, addresses, or even whole blog posts.

Strings can also be templated in code to allow for “fill-in-the-blank” strings that are not set until the script is run. Strings are technically immutable but can be manipulated using built-in Python string tools and the separate String module.

Here are some of the key concepts relating to strings:

Quotation marks

Strings must be surrounded by quotation marks. In Python, these can be either single or double quotes, but they must be consistent. If a single quote is used to start the string, a single quote must be used to stop it, or you will get an error:

>>> string_var = 'the red fox"
  File "<stdin>", line 1
    string_var = 'the red fox"
                             ^
SyntaxError: EOL while scanning string literal

A correct way:

>>> string_var = 'the red fox'
>>> string_var
'the red fox'

Multiple line strings

Multiple line strings are created by a pair of three single quotes or double quotes at the beginning of the string, and three at the end.

In the following example, the variable string_var is a multiple-line string (\n is a Python character representing a new line):

>>> string_var = """the red fox chased the
... dog across the yard"""
>>> string_var
'the red fox chased the\ndog across the yard'

String addition (and more)

Strings can be “added” together to create a new string. This process allows you to build strings from smaller strings, which can be useful for populating new fields composed of other fields in a data file, and other tasks.

In this example, the string "forest" is assigned to string_var. Another string is then added to string_var to create a longer string:

>>> string_var = "forest"
>>> string_var += " path" # same as string_var = string_var+ " path"
>>> string_var
'forest path'

String formatting

Strings in code often make use of “placeholders” for data that will be filled in later. This is known as string formatting, and there are multiple ways to perform string formatting using Python.

Here are the key concepts:

String format function

This method of formatting is the preferred form for Python 3. It allows you to pass the variables to the format() function, which is built into all strings, and to have them fill up placeholders within the string. Any data type can be passed to the format() function.

In the following example, the string template is filled with details contained in other variables using the format() string function. The placeholders are filled in the order that the variables are listed, so they must be in the correct order.

The curly brackets are the placeholders, and the format() function will accept arguments and fill in the string:

>>> year = 1980
>>> day = "Monday"
>>> month = "Feb"
>>> template = "It was a cold {} in {} {}"
>>> template.format(day, month, year)
'It was a cold Monday in Feb 1980'

In the next example, the placeholders are named, and are passed to keyword arguments in the format() function. The arguments are named and do not need to be in order in the format() function:

>>> template = 'It was a cold {day} in {month} {year}'
>>> template.format(month=month, year=year, day=day)
'It was a cold Monday in Feb 1980'

In this last example, the placeholders are numbered, which makes it much easier to repeat a string:

>>> template = "{0},{0} oh no,{1} gotta go"
>>> template.format("Louie", "Me")
'Louie,Louie oh no,Me gotta go'

String literals

There is a new (as of Python 3.6) method of formatting strings, known as formatted string literals. By adding an f before strings, placeholder variables can become populated by variables without using the format() function.

In this example, the variables are formatted directly into the string literal, which has an f before the string to indicate that it is a string literal:

>>> year = 1980
>>> day = "Monday"
>>> month = "Feb"
>>> str_lit = f"It was a cold {day} in {month} {year}"
>>> str_lit
'It was a cold Monday in Feb 1980'

String placeholder operators

An older but still useful method for inserting data into strings is the data type string operators. These use placeholders that will format inserted strings in specific ways. However, they are data-specific, meaning that a number inserted into the string must use a number placeholder, and a string being inserted must use a string placeholder, or an error will result.

The placeholders are %s for strings and %d or %f for numbers. They each have optional features specific to the data type. For instance, the %f number placeholder can be manipulated to hold only a specific number of decimal points:

>>> month = '%0.2f' % 3.1415926535
>>> month
3.14

To use them, you place the placeholder in the string template, and then follow the string with a percent sign (%) and the values to be passed into the string template in a tuple after the percent sign:

>>> year = 1980
>>> month = "February,"
>>> str_result = "It was a cold  %s %d" % month, year
>>> str_result 
'It was a cold February, 1980'

String manipulation

String manipulation is common and lots of tools are built into the String data type. These allow you to replace characters in a string or find their index location in the string.

The find() and index() methods are similar, but find() can be used in conditional statements. If the character is not found in the string, find() will return -1, while index() will return an error.

The join() method is used to join together a list of string data. The split() method is the opposite: it splits a string into a list based on a supplied character or the default empty space.

Here is a non-exhaustive list of methods and examples of their use:

string_list = ['101 N Main St','Eureka','Illinois 60133']
address = ', '.join(string_list)

address = '101 N Main St'.replace("St","Street")

str_var = 'rare'
str_index = str_var.find('a') # index 1
str_index = str_var.find('r') # index 0
str_index = str_var.rfind('r') # index 2
str_index = str_var.rfind('d') # index -1

name = "Laura"
name_upper = name.upper()
name_lower = name.lower()
name_title = name_lower.title()

str_var = 'rare'
str_index = str_var.index('a') # index 1
str_index = str_var.index('r') # index 0
str_index = str_var.rindex('r') # index 2
str_var.index('t') # this will cause an error 

latitude,longitude = "45.123,-95.321".split(",")
address_split = '101 N Main St'.split()

String indexing

String indexing is similar to list indexing, as we will see later on. Individual characters, or groups of characters, can be selected from a string by passing the index of the character needed to the string in square brackets, where 0 is the index of the first character.

In the following example, the d from readiness is accessed by passing the index [3] to square brackets next to the string:

>>> str_var = "readiness"
>>> d_var = str_var[3]
>>> d_var 
'd'

Groups of characters are selected by passing a start and end index, where the end index is the index of the first character you do not want to include:

>>> str_var = "readiness"
>>> din_var = str_var[3:6] # index 6 is e
>>> din_var
'din'
>>> dine_var = str_var[3:7] # index 7 is s
>>> dine_var
'dine'

Integers

The Integer data type represents whole numbers. It can be used to perform addition, subtraction, multiplication, and division (with one caveat as noted below):

>>> int_var = 50
>>> int_var * 5
250
>>> int_var / 5
10.0
>>> int_var ** 2
2500

Floating numbers

Floating-point numbers in Python are used to represent real numbers as 64-bit double-precision values. Sometimes, using binary systems to represent decimal-based numbers can be a bit odd, but in general, these will work as expected:

>>> x = 5.0
>>> x * 5
25.0
>>> x ** 5
3125.0
>>> x/2.3
2.173913043478261

One unique result from floating-point division is the case of 1/3. Because it is a binary representation, the assumption that (1/3)* 3= 1 is True, even though the value 0.3333333333333333 (produced by the division operation) would never add up to 1 when added together three times in a base 10 system. Here are some examples of binary math in action:

>>> 1/3
0.3333333333333333
>>> (1/3) * 3
1.0
>>> (1/3) + (1/3)
0.6666666666666666
>>> (1/3) + (1/3) + (1/3)
1.0
>>> (1/3) + (1/3) + 0.3333333333333333
1.0
>>> (1/3) + (1/3) + 0.3333
0.9999666666666667
>>> (1/3) + (1/3) + 0.3333333333
0.9999999999666667
>>> (1/3) + (1/3) + 0.333333333333333
0.9999999999999996
>>> (1/3) + (1/3) + 0.33333333333333333 1.0

Conversion between strings, integers, and floats

Conversion between data types is possible in Python using built-in functions that are part of the standard library. As we saw earlier, the type() function is useful to find the data type of an object. Once identified, the data object can be converted from Integer (int() function) to String (str() function) to Float (float() function), as long as the character would be valid in that data type.

In these examples, a character is converted from String to Integer to Float to String using the int(), str(), and float() functions:

>>> str_var = "5"
>>> int_var = int(str_var)
>>> int_var
5
>>> float_var = float(int_var)
>>> float_var
5.0
>>> str_var = str(float_var)
>>> type(str_var)
'<class 'str'>'

Data structures or containers

Data structures, also called data containers and data collections, are special data types that can hold, in a retrievable order, any data item of any data type (including other data containers). Data containers are used to organize data items by index in tuples or lists, or by key:value pairs in dictionaries.

To get data out of data containers, square brackets are used to pass either indexes (lists and tuples) or keys (dictionaries). If there is more than one level of data container (in other words, one container contains another), first the data container inside is referenced using an index or key inside a first square bracket, and then the data inside the container is accessed using a second.

The following table summarizes the different types of data containers and how data is retrieved from each of them:

tuple_var = ("blue", 32,[5,7,2],'plod',{'name':'magnus'})
plod_var = tuple_var[-2]
magnus_var = tuple_var[-1]['name']

list_var = ['fast','times',89,4.5,(3,8),{'we':'believe'}]
times_var = list_var[1]
dict_var = list_var[-1]
believe_var = list_var[-1]['we']

list_var = [1,1,4,6,7,6]
set_var = set(list_var) # removes duplicates
{1, 4, 6, 7}   # result

dict_var = {"key": "value"}
dict_info = {"address": "123 Main Street", "name": "John"}
name = dict_info["name"] # gets the name value from the key
address = dict_info["address"] # gets the address value

Tuples

Tuples are ordered lists that can hold any data type, even in the same tuple. They are immutable, meaning they cannot be altered, and data cannot be added to or removed from the tuple once it has been created. They have length and the built-in len() function can be used to get the length of the tuple.

In Python, they are declared by using round brackets, (), or the tuple() function. Data is accessed using zero-based indexing by passing the index to square brackets next to the tuple.

In the following example, a tuple is assigned to the variable name tuple_var(), and data is accessed using indexing:

>>> tuple_var = ("red", 45, "left")
>>> type(tuple_var)
<class 'tuple'>
>>> ("red",45,"left")[0]
'red'
>>> tuple_var[0]
'red'

Lists

Lists (often called arrays in other programming languages) are data containers that can hold any other type of data type, even in the same list, just like tuples. Unlike tuples, though, lists can be altered after they are created. In Python, they are declared by using square brackets, [], or the list() function. Data is accessed using zero-based indexing by passing the index to square brackets next to the list.

In this example, a list is assigned to the variable name list_var, and data is accessed using indexing:

>>> list_var = ["blue",42,"right"]
>>> type(list_var)
<class 'list'>
>>> ["blue",42,"right"][0]
'blue'
>>> list_var[0]
'blue'

Conversion between lists and tuples

Lists can be copied into a new tuple object using the tuple() function. Conversely, tuples can be copied into a list data type using the list() function. This does not convert the original data item but instead creates a copy of the data item in the new data type.

In the following example, the list is copied into a tuple data type, and then the tuple is copied into a list data type. Note that the brackets change with each new data type created:

>>> tuple_copy = tuple(list_var)
>>> tuple_copy
('blue', 42, 'right')
>>> list_copy = list(tuple_copy)
>>> list_copy
['blue', 42, 'right']

List operations for lists only

Using the append() method, a list can be appended to, which means one data item is added to the list. Using the extend() method, a list can also be extended, which is where all data items in a second list are all added to the first list:

>>> list_orig = [34, 'blanket', 'dog']
>>> list_orig.append(56)
>>> list_orig
[34,'blanket','dog',56]
>>> list_first = [34, 'blanket', 'dog']
>>> list_second = ['diamond', '321', 657]
>>> list_orig.extend(list_second)
>>> list_orig
[34,'blanket','dog','diamond','321'.657]

The items in a list can be reversed or sorted, using the reverse() method or the sort() method respectively:

>>> list_var = [34,'blanket','dog']
>>> list_var.reverse()
>>> list_var
['dog','blanket',34]

In Python 3, sorting is only allowed on lists that do not have mixed data types:

>>> list_var = [34,5,123]
>>> list_var.sort()
>>> list_var
[5, 34, 123]

List operations for both tuples and lists

Lists and tuples can be iterated over using for loops, which we will look at shortly. They can both be sliced as well, creating a subset of the list or tuple that will be operated on for the for loop or another operation. Built-in functions allow for the calculation of the maximum (using the max() function) or minimum (the min() function) value of a list/tuple, or even the sum of a list or tuple, given the data type of the items in the list is correct.

Slicing

Slicing a list or tuple will create a new list or tuple. The slice is created by passing indexes to the list or tuple in square brackets, separated by a colon. The first index is the start index, and it can be ignored if it is index 0 (the beginning of the original list). The second index is the index of the first value that you do not want to include (it can be blank if you want the rest of the original list).

In this first example, we see a tuple with three data items sliced to only include the first two items. The string "left" is at index 2 in the tuple, meaning that the last index in the slice will be 2. The slice is assigned to variable name tuple_slice:

>>> tuple_var = ("red", 45, "left")
>>> tuple_slice = tuple_var[:2]
>>> tuple_slice
('red', 45)

In this next example, we see a list with four data items sliced to only include the last two items. The first index is the index of the first data item we want (the string "right"). The last index is blank:

>>> list_var = ["blue", 42, "right", "ankle"]
>>> list_slice = list_var[2:]
>>> list_slice
['right', 'ankle']

Sets

Sets represent a collection of distinct objects. In Python, sets are unordered, no duplicates are allowed, and all data items inside a set must be immutable.

Set operations

Sets are especially useful for getting all distinct members of a list:

>>> orig_list = ["blue", "pink", "yellow", "red", "blue", "yellow"]
>>> set_var = set(orig_list)
>>> set_var
{'pink', 'yellow', 'blue', 'red'}

Sets cannot be accessed using indexing, because they are unordered and therefore are not subscriptable:

>>> set_var[0]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: 'set' object is not subscriptable

However, they can be iterated over using looping:

>>> for item in set_var:
...     print(item)
... 
pink
yellow
blue
red

Dictionaries

Dictionaries are key:value stores, meaning they are data containers that use unordered key and value pairs to organize data. Keys are used as reference points for organization and retrieval. When a key is supplied to a dictionary in square brackets, the value is returned:

>>> dict_var = {"key":"value"}
>>> dict_var['key']
'value'
>>> dict_var = {"address":"123 Main St", "color":"blue"}
>>> dict_var["address"]
'123 Main St'
>>> dict_var["color"]
'blue'

Keys and values

Keys can be any immutable data type, meaning lists cannot be used as keys, but strings, integers, floats, and tuples can. Values can be any type of data, including other dictionaries.

All keys in a dictionary can be accessed as a list using the dictionary keys() function. In Python 3.x, the function is a generator, which means it must be called over and over to get each key. This generator can also be passed to the list() function to convert it into a list.

All values in a dictionary can be accessed as a list using the dictionary values() function. In Python 3.x, the function is a generator.

Iteration

The core of computer programming is iteration: recursively performing the same action, analysis, function call, or whatever your script is built to process. Computers excel at this type of task: they can quickly iterate through a dataset to perform whatever action you deem necessary, on each data item in the set.

Iteration is run on iterators. An iterator is a Python object that contains other objects, each of which can be processed in a loop. Iterators can be lists or tuples or even strings, but not integers.

For loops

A for loop is an iteration implementation that, when presented with a data list, will perform an operation on each member of the list.

In the following example, a list of integers is assigned to the variable name data_list. The list is then used to construct a for loop using the format for {var} in {iterable}, where {var} is a variable name that is assigned to each object in the list, one at a time as the loop progresses. One convention is to use item, but it can be any valid variable name:

data_list = [45,56,34,12,2]
for item in data_list:
    print (item * 2)

This is the output:

90
112
68
24
4

While loops

A while loop is an iteration implementation that will loop until a specific threshold is met. While loops can be dangerous as they can cause an infinite loop in a script if the threshold is never met.

In the following example, the while loop will run, doing nothing but adding 1 to x until it reaches 100, upon which the threshold is met and the while loop will end:

x = 0
while x < 100:
    x = x + 1   # same as x += 1

Counters and enumerators

Iteration in for loops or while loops often requires the use of counters (also called enumerators) to track loops in an iteration.

for loops have the option to use the enumerate() function by passing the iterator to the function and using a count variable (which can be any valid variable name, but count is logical) in front of the item variable. The count variable will keep track of the loops, starting at index zero:

>>> data_list = ['a','b','c','d','e']
>>> for count,item in enumerate(data_list):
...     print(count, item)
... 
0 a
1 b
2 c
3 d
4 e

In Python, the shortcut x += y is used to increase the value of x while keeping the same variable name, and is the same as x = x + y:

>>> x = 0
>>> while x <100:
...    x = x + 1
>>> x
100
>>> x = 0
>>> while x <100:
...    x += 1
>>> x
100

Conditionals

if statements, elif statements (short for else if), and else statements are used to create conditions that will be used to evaluate data objects. If statements can be used by themselves (elif and else are optional) and are used by declaring the keyword if and then the condition the data must meet.

In the following example, the data type of objects in a list is compared (notice the two equals signs, meaning it is a comparison) to the data type for integers, shown here as type(0), or for strings, shown as type('a'). If an object in the list meets one of the conditions, a specific print() statement is triggered:

list_var = [1,'1',1.0]
for item in list_var:
    if type(item) == type(0):
        print('Integer')
    elif type(item) == type('a'):
        print('String')
    else:
        print('Float') 

If versus else

if statements are usually specific to one condition, whereas else statements are used as catch-alls to ensure that any data that goes through the if statement will have some way of being dealt with, even if it doesn’t meet the condition of the if statement. elif statements, which are dependent on the if statement existing and are also condition-specific, are not catch-all statements.

Zero-based indexing

As we have seen, iteration occurs over lists or tuples that contain data. Within the list, these data are differentiated by list order or position. Items in a list are retrieved by item index, the (current) position of the data in the list.

In Python, like most computer programming languages, the first item in a list is at index 0, not index 1.

This is a bit confusing to beginners but is a programming standard. It is slightly more computationally efficient to retrieve an item in a list that starts with 0 than a list that starts with 1, and this became the standard in C and its precursors, which meant that Python (written in C) uses zero-based indexing.

Data extraction using index position

This is the basic format of data retrieval from a list. This list of strings has an order, and the string "Bill" is the second item, meaning it is at index 1. To assign this string to a variable, we pass the index into square brackets:

names = ["Silas", "Bill", "Dara"]
name_bill = names[1]

Data extraction using reverse index position

This is the second format of data retrieval from a list. List order can be used in reverse, meaning that the indexing starts from the last member of the list and counts backward. Negative numbers are used, starting at -1, which is the index of the last member of the list, -2 is the second-to-last member of the list, and so on.

This means that, in the following example, the "Bill" and "Silas" strings are at indexes -2 and -3 respectively when using reverse index position, and so -2 (or -3) must be passed to the list in square brackets:

names = ["Silas", "Bill", "Dara"]
name_bill = names[-2]
name_silas = names[-3]

Functions

Functions are subroutines defined by code. When called, or run, functions will do something (or nothing, if written that way). Functions often accept parameters, and these can be required or optional.

Functions make it easy to perform the same action over and over without writing the same code over and over. This makes code cleaner, shorter, and smarter. They are a good idea and should be used often.

Components of a function

Here are the main parts that make up a function in Python:

• def keyword: Functions are defined using the def keyword, which is short for “define function.” The keyword is written, followed by the name of the function and round brackets, (), into which expected parameters can be defined.
• Parameters: Parameters or arguments are values expected by functions and supplied by the code at runtime. Some parameters are optional.
• Return statement: Functions allow for data to be returned from the subroutine to the main loop using return statements. These allow the user to calculate a value or perform some action in the function and then return back a value to the main loop.
• Docstrings: Functions allow for a string after the definition line that is used to declare the purpose of the function:

  def accept_param(value=12):
      'this function accepts a parameter' # docstring 
      return value
  

Note that optional parameters with default values must always be defined after the required parameters within functions.

Namespaces

In Python, there is a concept called namespaces. These are refined into two types of namespaces: global and local.

All variables defined in the main part of a script (outside of any functions) are considered to be in the global namespace. Within the function, variables have a different namespace, meaning that variables inside a function are in a local namespace and are not the same as variables in the main script, which are in the global namespace. If a variable name inside a function is the same as one outside of the function, changing values inside the function (in the local namespace) will not affect the variable outside the function (in the global namespace).

Function examples

In this first example, a function is defined and written to return "hello world" every time it is called. There are no parameters, but the return keyword is used:

def new_function():
    return "hello world"
>>> new_function()
'hello world'

In this next example, an expected parameter is defined in the brackets. When called, this value is supplied, and the function then returns the value from the local namespace back to the global namespace in the main loop:

def accept_param(value):
    return value 
>>> accept_param('parameter')
'parameter'

In this final example, an expected parameter has a default value assigned, meaning it only has to be supplied if the function uses a non-default parameter:

def accept_param(value=12):
    return value
>>> accept_param()
12
>>> accept_param(13)
13

Classes

Classes are special blocks of code that organize multiple variables and functions into an object with its own methods and functions. Classes make it easy to create code tools that can reference the same internal data lists and functions. The internal functions and variables are able to communicate across the class so that variables defined in one part of the class are available in another.

Classes use the idea of self to allow for the different parts of the class to communicate. By introducing self as a parameter into each function inside a class, the data can be called.

Here is an example of a class:

class ExampleClass():
    def __init__(self, name):
        'accepts a string'
        self.name = name
    def get_name(self):
        'return the name'
        return self.name

Classes are called or instantiated to create a class object. This means the class definition is kind of like a factory for that class, and when you want one of those class objects, you call the class type and pass the correct parameters if required:

>>> example_object = ExampleClass('fred')
>>> example_object.get_name()
'fred'