Book Image

Speed Up Your Python with Rust

By : Maxwell Flitton
5 (2)
Book Image

Speed Up Your Python with Rust

5 (2)
By: Maxwell Flitton

Overview of this book

Python has made software development easier, but it falls short in several areas including memory management that lead to poor performance and security. Rust, on the other hand, provides memory safety without using a garbage collector, which means that with its low memory footprint, you can build high-performant and secure apps relatively easily. However, rewriting everything in Rust can be expensive and risky as there might not be package support in Rust for the problem being solved. This is where Python bindings and pip come in. This book will help you, as a Python developer, to start using Rust in your Python projects without having to manage a separate Rust server or application. Seeing as you'll already understand concepts like functions and loops, this book covers the quirks of Rust such as memory management to code Rust in a productive and structured manner. You'll explore the PyO3 crate to fuse Rust code with Python, learn how to package your fused Rust code in a pip package, and then deploy a Python Flask application in Docker that uses a private Rust pip module. Finally, you'll get to grips with advanced Rust binding topics such as inspecting Python objects and modules in Rust. By the end of this Rust book, you'll be able to develop safe and high-performant applications with better concurrency support.
Table of Contents (16 chapters)
1
Section 1: Getting to Understand Rust
5
Section 2: Fusing Rust with Python
11
Section 3: Infusing Rust into a Web Application

Understanding variable ownership

As we pointed out in the introduction discussing why we should use Rust, Rust doesn't have a garbage collector; however, it is still memory-safe. We do this to keep the resources low and the speed high. However, how do we achieve memory safety without a garbage collector? Rust achieves this by enforcing some strict rules around variable ownership.

Like typing, these rules are enforced when the code is being compiled. Any violation of these rules will stop the compilation process. This can lead to a lot of initial frustration for Python developers, as Python developers like to use their variables as and when they want. If they pass a variable into a function, they also expect that variable to still be able to be mutated outside the function if they want. This can lead to issues when implementing concurrent executions. Python also allows this by running expensive processes under the hood to enable the multiple references with cleanup mechanisms when the variable is no longer referenced.

As a result, this mismatch in coding style gives Rust the false label of having a steep learning curve. If we learn the rules, we only must rethink our code a little, as the helpful compiler enables us to adhere to them easily. You'll also be surprised how this approach is not as restrictive as it sounds. Rust's compile-time checking is done to protect against the following memory errors:

  • Use after frees: This is where memory is accessed once it has been freed, which can cause crashes. It can also allow hackers to execute code via this memory address.
  • Dangling pointers: This is where a reference points to a memory address that no longer houses the data that the pointer was referencing. Essentially, this pointer now points to null or random data.
  • Double frees: This is where allocated memory is freed, and then freed again. This can cause the program to crash and increases the risk of sensitive data being revealed. This also enables a hacker to execute arbitrary code.
  • Segmentation faults: This is where the program tries to access the memory it's not allowed to access.
  • Buffer overrun: An example of this is reading off the end of an array. This can cause the program to crash.

Rust manages to protect against these errors by enforcing the following rules:

  • Values are owned by the variables assigned to them.
  • As soon as the variable goes out of scope, it is deallocated from the memory it is occupying.
  • Values can be used by other variables, if we adhere to the conventions around copying, moving, immutable borrowing, and mutable borrowing.

To really feel comfortable navigating these rules in code, we will explore copying, moving, immutable borrowing, and mutable borrowing in more detail.

Copy

This is where the value is copied. Once it has been copied, the new variable owns the value, and the existing variable also owns its own value:

Figure 1.3 – Variable Copy path

Figure 1.3 – Variable Copy path

As we can see with the pathway diagram in Figure 1.3, we can continue to use both variables. If the variable has a Copy trait, the variable will automatically copy the value. This can be achieved by the following code:

let one: i8 = 10;
let two: i8 = one + 5;
println!("{}", one);
println!("{}", two);

The fact that we can print out both the one and two variables means we know that one has been copied and the value of this copy has been utilized by two. Copy is the simplest reference operation; however, if the variable being copied does not have a Copy trait, then the variable must be moved. To understand this, we will now explore moving as a concept.

Move

This is where the value is moved from one variable to another. However, unlike Copy, the original variable no longer owns the value:

Figure 1.4 – Variable Move path

Figure 1.4 – Variable Move path

Looking at the path diagram in Figure 1.4, we can see that one can no longer be used as it's been moved to two. We mentioned in the Copy section that if the variable does not have the Copy trait, then the variable is moved. In the following code, we show this by doing what we did in the Copy section but using String as this does not have a Copy trait:

let one: String = String::from("one");
let two: String = one + " two";
println!("{}", two);
println!("{}", one);

Running this gives the following error:

let one: String = String::from("one");
    --- move occurs because 'one' has type 
    'String', which does not implement the 
    'Copy' trait
let two: String = one + " two";
        ------------ 'one' moved due to usage in operator
println!("{}", two);
println!("{}", one);
        ^^^ value borrowed here after move

This is really where the compiler shines. It tells us that the string does not implement the Copy trait. It then shows us where the move occurs. It is no surprise that many developers praise the Rust compiler. We can get round this by using the to_owned function with the following code:

let two: String = one.to_owned() + " two";

It is understandable to wonder why Strings do not have the Copy trait. This is because the string is a pointer to a string slice. Copying actually means copying bits. Considering this, if we were to copy strings, we would have multiple unconstrained pointers to the same string literal data, which would be dangerous. Scope also plays a role when it comes to moving variables. In order to see how scope forces movement, we need to explore immutable borrows in the next section.

Immutable borrow

This is where one variable can reference the value of another variable. If the variable that is borrowing the value falls out of scope, the value is not deallocated from memory as the variable borrowing the value does not have ownership:

Figure 1.5 – Immutable borrow path

Figure 1.5 – Immutable borrow path

We can see with the path diagram in Figure 1.5 that two borrows the value from one. When this is happening, one is kind of locked. We can still copy and borrow one; however, we cannot do a mutable borrow or move while two is still borrowing the value. This is because if we have mutable and immutable borrows of the same variable, the data of that variable could change through the mutable borrow causing an inconsistency. Considering this, we can see that we can have multiple immutable borrows at one time while only having one mutable borrow at any one time. Once two is finished, we can do anything we want to one again. To demonstrate this, we can go back to creating our own print function with the following code:

fn print(input_string: String) -> () {
    println!("{}", input_string);
}

With this, we create a string and pass it through our print function. We then try and print the string again, as seen in the following code:

let one: String = String::from("one");
print(one);
println!("{}", one);

If we try and run this, we will get an error stating that one was moved into our print function and therefore cannot be used in println!. We can solve this by merely accepting a borrow of a string using & in our function, as denoted in the following code:

fn print(input_string: &String) -> () {
    println!("{}", input_string);
}

Now we can pass a borrowed reference into our print function. After this, we can still access the | variable, as seen in the following code:

let one: String = String::from("one");
print(&one);
let two: String = one + " two";
println!("{}", two);

Borrows are safe and useful. As our programs grow, immutable borrows are safe ways to pass variables through to other functions in other files. We are nearly at the end of our journey toward understanding the rules. The only concept left that we must explore is mutable borrows.

Mutable borrow

This is where another variable can reference and write the value of another variable. If the variable that is borrowing the value falls out of scope, the value is not deallocated from memory as the variable borrowing the value does not have ownership. Essentially, a mutable borrow has the same path as an immutable borrow. The only difference is that while the value is being borrowed, the original variable cannot be used at all. It will be completely locked down as the value might be altered when being borrowed. The mutable borrow can be moved into another scope like a function, but cannot be copied as we cannot have multiple mutable references, as stated in the previous section.

Considering all that we have covered on borrowing, we can see a certain theme. We can see that scopes play a big role in implementing the rules that we have covered. If the concept of scopes is unclear, passing a variable into a function is changing scope as a function is its own scope. To fully appreciate this, we need to move on to exploring scopes and lifetimes.