For handling requests through the use of multiple application instances, we use the concept of horizontal scaling, where we launch more than one instance of the same application behind a load balancer. The load balancer is then responsible for distributing the incoming requests across this pool of application instances.

The process pools can be implemented through the use of ProcessPoolExecutor from the concurrent.futures library in Python. An example of how to use ProcessPoolExecutor to distribute the requests over a pool can be seen in the Using thread pools for handling incoming connections section of this chapter.

It is completely possible to have a program that combines the use of multiprocessing and multithreading. The following snippet of code shows this implementation:

import threading
import multiprocessing

def say_hello():
    print("Hello")

def start_threads():
    thread_pool = []
    for _ in range(5):
        thread = threading.Thread(target=say_hello)
        thread_pool.append(thread)
    for thread in thread_pool:
        thread.start()
    for thread in thread_pool:
        thread.join()

def start_process():
    process_pool = []
    for _ in range(3):
        process = multiprocessing.Process(target=start_threads)
        process_pool.append(process)
    for process in process_pool:
        process.start()
    for process in process_pool:
        process.join()

if __name__ == '__main__':
    start_process()

The preceding way of achieving this is valid and can be easily implemented without any issues, though you might find that it has a limited number of use cases, and its use will be limited by the implementation of the GIL.

A simple example of implementing a socket server is shown in the Implementing a simple socket server with AsyncIO section of this chapter. Another way is to implement a fully functional web server through the use of AsyncIO is by using the aiohttp framework, which provides an AIO-based HTTP server.