Home automation structure

After some background information about different architectures used for home automation and also aspects to motivate you to build your home automation system, it is now time to detail each component. The architecture that I will focus on in this book is the one that I have implemented in my home, so it is proven that it works. This architecture has the same idea of the client-server presented previously in Figure 1.6, but here, I have added the proper names and identifications of each part of the system. This architecture is presented in Figure 1.7:

Figure 1.7: Home automation system architecture explored in this book

This architecture assumes you have a Wi-Fi 802.11g or above router in your home, but this router does not necessarily need to be connected to the internet unless you want to access your home automation system remotely.

In the following subsections, each part of the system presented in Figure 1.7 is discussed in detail.

Home automation server

The home automation server is the central connection or the heart of a home automation system. It centralizes all the communication received from the sensors and transmits them to the actuators. It contains all the logic of the automation created by the user and also allows the configuration of new devices to be added to the system. The server also manages other connections to the system, such as applications that can be connected to it using a cell phone running Google Android or Apple iPhone Operating System (iOS).

The home automation server is composed of two distinct parts: hardware and software. Depending on the level the user wants to go in the system, the hardware can vary from a desktop or notebook to their own design. It will all depend on the cost and time invested in creating or building the server. Due to the availability of open source hardware, I recommend that the system is implemented using an off-the-shelf device. I strongly recommend that a board such as a Raspberry Pi 4 is used, but if you already have an old computer or any other hardware that can run Linux, it could be used. Another commonly used hardware option is the Intel NUC platform.

The home automation server should be able to run Linux or some other popular operating system such as Windows or macOS. Another requirement is that the server should support Wi-Fi 802.11n wireless protocol or above. Currently, I’m using a Raspberry Pi 4 Model B 2019 with 4 GB of RAM in my home automation system. This hardware will be the focus of the home automation server used in this book.

Speaking about the software component of the home automation server, you can try to build your own software to manage the role of the automation server. The only advantage of this method is the learning process that you will go through by creating it. I must confess that I learned a lot when I implemented the X10 task controller. The original software from the microchip application was written in Assembly and I migrated it to the C programming language. Later, I migrated the C application to the Arduino-like format used in the Particle Wi-Fi board. This learning process was useful for me in other aspects of my job and career, but nothing compares to the variety of home automation software available today.

As the name of the book somehow anticipates, Home Assistant (https://www.home-assistant.io/) is the software used in the home automation server adopted in this book. The reasons I chose Home Assistant to use at my home were because of the documentation, support, integrations, and add-ons it gives to the users. Home Assistant is one of the most used open source software for home automation projects and supports Linux, Windows, and macOS operating systems. It supports lots of add-ons that allow the system to be expanded, as will be explained in Chapter 7. Another open source software that can be used in home automation systems is openHAB (https://github.com/openhab).

One paradigm followed by this book is that you are free to choose your sensors and actuators to be integrated into your system, which is one of the reasons for using Home Assistant. In this case, the home automation server software should allow easy integration of sensors off the shelf and even sensors you decide to create. There are some home automation software and systems that interlock the hardware and software in their architecture, not allowing other types of devices to be used in your home automation system different from the architecture they offer. In my opinion, this kind of approach prevents the user from being free and flexible in the choice of sensors and actuators. The most popular systems of this nature are Insteon, Lutron, and SmartThings, owned by Samsung Electronics. In this regard, Home Assistant supports different ways of integrating the different sensors and actuators so it will not hold you to some particular platform or architecture. This free choice will certainly contribute to lowering the cost of your system since you will have more options of devices to select.

In general, I recommend that you choose compact hardware so it can easily be installed in any place in your home, preferably somewhere hidden, and then eventually, you have to be physically connected to it using a computer video screen. Once the home automation server is installed and initially configured, you will not need to access it physically anymore unless it presents some issue or if you want to install some other software on it that is not able to be installed remotely. If the idea is to use the hardware only as a home automation server, all configuration, automation, and any other new setup can be done remotely via a web browser. It is worth mentioning that the location for the installation should not be too far from the main source of the Wi-Fi router used in your home. If you need more reliability in your home automation server connectivity, it is recommended that you use a wired connection such as Ethernet (IEEE 802.3) connected to your Wi-Fi router.

In my home, the home automation server is installed at the back of my computer video screen using 3M VHB Scotch SJ3550 Dual Lock adhesive tape. Figure 1.8 shows this installation:

Figure 1.8: My home automation server

Figure 1.8 shows the home automation server using Raspberry Pi on the left and also an ESP32 Wi-Fi/Bluetooth module (upside down) on the right used in the five-zone temperature logger explained in Chapter 10. In the next chapter, we will introduce Home Assistant as the software for the home automation server and will guide you to install it on a Raspberry Pi 4 as the hardware used for this server and used in this book.

Sensors

The sensors and actuators are what bring life to your home automation system and allow you to create automation, configuration, and integration using Home Assistant.

Sensors are electronic devices that contain certain electronic components able to measure or detect some measurement or status of what or where they are installed. They always provide an output signal that is used as an input signal to the home automation server to inform it about what the sensor is detecting.

The sensors can be generically classified into two types: analog and digital. Analog sensors are the ones that can provide discrete values of output signals. In other words, these sensors can have multiple values to express the information being measured. A temperature sensor, for example, is an analog sensor that can express in its output different measurements about each temperature value captured or sensed by it.

A digital sensor provides binary information in its output, meaning two states which can vary depending on the sensor. The common states associated with these sensors are turned on, turned off, open, and closed. These sensors can also inform about two values in reference to a threshold. In this case, one state will be above the threshold, and the other state will be below the threshold. One good example of a digital sensor used in home automation systems is a door sensor. It is basically composed of an electronic component called a reed switch, which works as a magnetic switch to provide two states: closed, to inform the door is closed, and opened, to inform the door is opened.

Table 1.1 lists some examples of sensors commonly used in home automation systems, including type and measurement sense:

Table 1.1: Examples of sensors used in home automation systems

A general block diagram of a sensor for home automation is demonstrated in Figure 1.9:

Figure 1.9: Home automation sensor block diagram

It is important to understand each one of the blocks discussed, to help you put together the parts and components to build your own sensor, which is discussed in Chapter 3 of this book.

The first component part of a sensor, as seen on the left of Figure 1.9, is usually an electronic component able to capture some measurements related to the environment where it is installed. We called it the sensor element. Within the electronic components used, some of the following examples can be found:

• Reed switches/magnetic switches: These are electromechanical components composed of two ferromagnetic metallic plates hermetically sealed by a glass enclosure. These plates move up to contact one to the other when a magnetic field is applied. They provide two statuses, open and closed, and are commonly used in door sensors.
• Passive Infrared (PIR) sensors: These are electronic components that are able to detect the changes in infrared radiation when something crosses their field of view. The sensor converts these variations in voltage changes to be formatted as motion changes. Despite the voltage variation being an analog value, the application (motion sensors) where these sensors are used only considers the digital variation of it. They are used in motion sensors.
• Light Dependent Resistor (LDR) sensors: These are electronic components that vary their resistance when light is applied. They provide an analog value according to the light intensity variation. They are used in light sensors.
• Thermal Resistors (thermistors) and temperature measurement Integrated Circuits (ICs): These temperature sensors are electronic components built with different technologies to provide an analog value associated with the temperature to which the sensor is submitted. In the past, resistors were made with the purpose of changing their values with the temperature, or thermistors were used, but today different electronic components manufacturers are using ICs as sensors, which provide much more reliability and measurement range. As the name suggests, they are used in temperature sensors.

The next block part of the sensor, as seen in the middle block of Figure 1.9, is the signal conditioner. These circuits are usually required to modify the signal generated by the sensor. This signal from the sensor could require some voltage level conversion, or in a lot of situations, the signal coming from the sensor has small voltage or current variations, so it needs to be amplified. Also, this same signal could be generated with a lot of noise and needs to be filtered. Signal conditioner circuits can perform the filtering process to make sure the signal will be cleaned and well-interpreted by the upstream circuit.

In summary, signal conditioner circuits translate the information from the electronic sensor components into something understandable by the next part of the sensor, the wireless microcontroller module. The signal could be conditioned to be read by the microcontroller as a digital binary or analog input. In some cases, as we will see in Chapter 3, the signal conditioner could encode the signal to generate some digital communication standard to interface to the microcontroller. These standards can be in different formats, but the most popular are serial communication interfaces such as Inter-Integrated Circuit (I2C) and Serial Peripheral Interface (SPI).

The last block of a home automation sensor (the right block in Figure 1.9) is the wireless microcontroller module. This module receives the information translated or converted from the sensor by the signal conditioner circuit and sends it via wireless to be managed by the home automation server.

If you are curious enough to open one of the home automation sensors, you may find a shield in the format of a metallic box where this wireless microcontroller electronic component is located. In some cases, to reduce the circuit footprint and cost, these microcontrollers are found without the metallic shield box. Actually, in most cases, these electronic components are called System on Chip (SoC). They are large-scale integrated circuits that combine a microcontroller and wireless radio in one single chip. Due to this large-scale integration, few external electronic components are required, keeping the size of the entire circuit small, which is highly desirable when it comes to a home automation sensor.

These SoCs are commonly used by many hobbyists to implement various kinds of Internet of Things (IoT) projects. The most popular of these components are from a Chinese component manufacturer called Espressif Systems. Espressif manufactures the ESP8265, ESP8266, and ESP32 SoCs. The main reasons for the mass utilization of these SoCs are, in my opinion, the easy availability of them in the market, affordable cost, and an active community of developers. You can easily find wireless communication modules based on the ESP8266 and ESP32 from the biggest world-famous dealers costing around $10 or less. You can also develop and deploy software programs using Arduino sketches via the Arduino Integrated Development Environment (IDE). This means if you know how to develop a program for Arduino using the different software libraries available, you should be able to develop applications using ESP8266 and ESP32.

In the next chapters, we will build some projects for sensors, and we will use the ESP8266 and ESP32 Wi-Fi-based modules to do it.

Actuators

The actuators receive the information from the home automation server and provide a type of output, such as turning on or off a switch connected to an appliance, for example. It can also receive information to send a command to simply turn a lamp on or off or change its color or brightness.

Likewise as the sensors, the signal sent to an actuator output can be digital or analog. Turning a switch on or off is a good example of digital binary information sent to an actuator. Other examples of actuators can be given according to Table 1.2:

Table 1.2: Examples of actuators

The block diagram of an actuator follows the opposite sequence found in a sensor and is configured according to Figure 1.10:

Figure 1.10: Home automation actuator block diagram

The wireless microcontroller module referenced in Figure 1.10 in the left block is usually the same found in the sensors and, again, mostly the ones manufactured by Espressif Systems, ESP8266, and ESP32. It receives information from the home automation server and translates it into an output command that will ultimately control the actuator element. This command could be a voltage level representing a binary signal, an analog signal, or even a serial digital communication protocol such as I2C or SPI.

The wireless microcontroller module becomes very versatile since it provides different input/output pins, serial interfaces, and other features and peripherals, allowing multiple combinations of electronic components and circuits. These can lead to sensors and actuators or more than one sensor element being used in the same enclosure. You can build a sensor that can be used for temperature and motion detection at the same time using just one wireless microcontroller, as can be seen in Chapter 3.

The signal conditioner circuit (the middle block in Figure 1.10) has the same idea used in the sensors, but in this case, it is conditioning the output signal sent by the wireless microcontroller module. It could be a voltage or current level converter, for example. This circuit is required in most cases due to a wireless microcontroller module limitation preventing the end actuator element from working properly.

The actuator element presented in Figure 1.10 on the right is what promotes the final effect when the home automation actuator is triggered. It is an electronic component or circuit. The simplest example of an actuator element is represented by an electronic component called an electromechanical relay. Combined with a conditional circuit, it allows direct control from a wireless microcontroller module. The relay converts an electronic binary signal into a mechanical switch command. So, the relay is an electronically controlled switch. This switch attached to an outlet can connect or disconnect the voltage from it, allowing it to turn an appliance on or off, for instance. Figure 1.11 shows a typical circuit used in a relay-based actuator and also a photo of a commercial actuator where each block of it is represented:

Figure 1.11: Circuits found in a home automation actuator

Communication media

Communication media is the environment or infrastructure where the home automation server talks to sensors, actuators, and other applications or devices that will connect to your system. It is important to understand how the communication media is structured and works so you can choose the proper media to implement your system and configure it by knowing which parameters to set up.

Wired systems

Commercial home automation systems can use wired media to communicate. These systems came in different formats, such as structured cabling using Unshielded Twisted Pair (UTP), coaxial, and fiber optic cables. Usually, they attend to some specific types of systems such as phones, cable TV, multimedia, and computers. As a disadvantage, this media type somehow limited the expansion of the systems and was not designed to automate a home, just to allow some type of devices to share or communicate using the same media. It is also not flexible since the communication node is available where a cable is present. Among the advantages of wired systems is the reliability and communication speed that some technologies can offer such as fiber optic cables.

As highlighted previously, one good example of wired systems targeting a home automation system was the X10 technology. The X10 uses the power line available in each home to encode binary information, and devices attached to the power line can decode this information and process it accordingly.

Wireless systems

Wireless systems are much more commonly used for home automation systems. The most common wireless protocols used are Wi-Fi and Zigbee. Wi-Fi (IEEE 802.11) is well-known, supported, and available. It works under the 2.4 and 5 GHz operating frequencies. When implemented in the sensors and actuators, it allows direct communication to the Raspberry Pi automation server, which also implements this protocol. The issue with the Wi-Fi wireless protocol is that it was not developed exactly for home automation and IoT systems. The power consumption in the Wi-Fi protocol is high, which in some situations prevents it from being extensively used in battery-powered sensors, for example.

The Zigbee wireless protocol (IEEE 802.15.4) is more applicable for home automation systems in some circumstances. It operates under the 2.4 GHz Industrial, Scientific, and Medical (ISM) band. It was created with a low overhead in the communication layer (low bandwidth), allowing efficiency when it comes to small-scale projects. It also has low-power radios compared to Wi-Fi, so it is more suitable for battery-powered applications. Due to the simplified communication stack, the Zigbee protocol can wake up from low power modes much quicker compared to the Wi-Fi protocol, so some sensors, such as door sensors, can react faster to events. The drawback of using the Zigbee protocol is that it is not as available as Wi-Fi, and when using commercial sensors, usually a specific router is required to add another layer of communication or device to the communication infrastructure, which adds costs and more configuration to be done in your home automation system. To date, I have not felt the need to add any infrastructure related to the Zigbee protocol in my home automation system. You have to evaluate your applications, and if you need to use lots of battery-powered sensors, it is worth considering Zigbee wireless communication in your system.

Another wireless protocol used for home automation systems is Z-Wave. It is a proprietary technology owned by the Silicon Labs company. It uses a mesh network arrangement, allowing devices to communicate among themselves. The mesh network can expand the range of the network and become stronger since it will continue to work if one node or the main node fails. Another advantage of Z-Wave wireless is that it operates under the sub-1 GHz frequency range band, which is less congested than 2.4 GHz, so it allows it to be low power, long range, and with less Radio Frequency (RF) interference.

Finally, I would like to mention the Bluetooth Low Energy (BLE) wireless communication protocol. This Bluetooth variation has many advantages related to home automation systems. Some of them are power consumption, security features, interoperability, ease of use, and low cost. These features, combined with Wi-Fi, can create other powerful communication protocols such as the recently launched Matter, which we will cover in Chapter 11.

With regards to wireless systems, it is worth commenting that the communication range is one aspect that should be considered. The range of these wireless devices can communicate is up to 50–100 meters inside a home, so it is good to evaluate where your wireless router will be installed so the Wi-Fi signal can cover your entire house. Depending on the configuration, you may need to use repeaters spread throughout your house. Another aspect to consider is that the wireless signal can be attenuated differently depending on the material your house is built. It means that a wall made of bricks and cement can attenuate more the signal compared to a wall made of wood. Make sure your entire house has enough wireless signal coverage; otherwise, your system will not work properly.

Important note

The wireless communication range of sensors and actuators is limited by the small size of the internal circuits, including the antenna, which is usually made as part of the Printed Circuit Boards (PCBs).

The communication media used in this entire book is the wireless Wi-Fi network. As mentioned, it is not required that this Wi-Fi network be connected to the internet unless you need to access your home automation system remotely.

Protocols and brokers (MQTT)

Protocols and brokers define the entire home automation structure and what language the devices in the system use to talk to each other. A protocol implements a set of commands, configurations, and communication schemes that allow one device to exchange information better with another. It can implement some sophisticated mechanisms to improve reliability, such as error correction, package retransmission, and so on.

In the sequence, I will provide details about a wired (X10 protocol) and wireless communication protocol (MQTT). MQTT will be very important to understand because it will be used in most of the future chapters of this book.

The X10 protocol

The X10 wired protocol works as follows. In the USA, the AC typically has a 60 Hz frequency and is represented by a sine wave crossing zero with a peak of 120 V. Each time that the current crosses the zero, a binary representation is encoded to the sine wave to represent ones and zeros. The binary 1 is represented by a 120 KHz burst with a 1-millisecond duration, and a binary 0 is represented by an absence of the 120 KHz burst after the crossing. In the three AC phase distribution systems, these bursts should be transmitted in each phase spaced 2.778 ms from each other.

One basic message encoded in the X10 format has 11 cycles of a power line. The first two cycles represent a Start Code (b1110), the following four cycles are the House Code (1…16), and the last five are a Number Code (1…16) or a Function Code (on, off, and so on). This complete block, including the Start Code, House Code, and Key Code, should always be transmitted twice spaced by three power line cycles between them. In order to improve reliability, the X10 protocol also states that the House Code and Key Code should be decoded as a complement form on alternate half cycles of the power line. This means, for example, if no signal is transmitted in the first half cycle (binary 0) then a 1 millisecond 120 KHz burst of the signal should be transmitted in the next half cycle (binary 1). Figure 1.12 explains how it works and also includes a table with the House Codes and Key Codes:

Figure 1.12: Information about the X10 protocol

Figure 1.12 at the very top on the left presents how the 120 KHz is encoded in the three-phase power line system. Actually, this is just a representation since the 120 KHz burst is superimposed on the 60 Hz sinusoidal signal. On the left, in the middle of Figure 1.12, the protocol fields for the House Code/Key Code and House Code/Function Code are presented. An example of how the house address A2 is coded is presented in the bottom left of Figure 1.12. The table on the right of Figure 1.12 presents the numbers used by House Code, Number Code, and Function Code representations.

As you may note, per the protocol specifications, X10 was developed to use in home automation systems, which can address 256 devices minimum with some possibility of extension using an Extended Code. It is relatively easy to implement the software code that implements the protocol. I implemented an X10 task controller using a PIC16F877 microcontroller, which was able to store 16 different events associated with the day and time. It was based on the Application Note (AN) 236 from Microchip (https://www.microchip.com/en-us/application-notes/an236). I used an X10 modem (PSC05) to interface the PIC microcontroller and encode the X10 signals to be transmitted to the power line.

The MQTT protocol

Before going into detail about the MQTT protocol, it is worth mentioning that wireless communication media has different layers. Each layer is responsible for a different level of abstraction of the communication, from how the data received or transmitted will interface to the medium to how the frames or datagrams are organized. It is not the goal of this book to detail these layers since you will not need to implement anything related to the level of the common wireless protocol. The only thing you will need to know for now is how to configure or connect to the Wi-Fi Service Set Identifier (SSID) and password of the router used in your home.

The MQTT message protocol is designed to run above the Wi-Fi or Zigbee communication layers. It was developed to run on IoT devices, meaning lightweight code that perfectly fits small microcontrollers. Besides that, it requires minimal network bandwidth and implements Quality of Service (QoS), improving the reliability of the message delivery.

MQTT is very suitable for home automation systems because it follows the client-server approach discussed previously. It implements an MQTT broker located in the home automation server, and the sensors are the MQTT clients. These MQTT clients are configured as publishers and publish or send information to MQTT brokers. This information is published by the MQTT clients under a specific topic. This topic is commonly the main information that the MQTT client wants to inform all interested parties. The MQTT broker holds the information from the MQTT clients and makes them available. If another MQTT client wants access to the information published by some other client, it subscribes to the topic of interest by sending a request to the MQTT broker. Once the MQTT client is subscribed to the topic of interest, the MQTT broker starts to publish the information under the topic subscribed to the MQTT client that requests the information. Figure 1.13 illustrates how the MQTT communication message protocol works:

Figure 1.13: MQTT protocol example for garage motion detection

Figure 1.13 shows a simplified example of a garage light motion sensor I implemented at my home. The motion sensor is installed at the top of my garage, and when a motion is detected, it sends a message via MQTT to the broker, which depending on the hour of the day, sends a message to an outlet actuator installed in my garage lights and turns on the lights. It keeps the lights on for 90 seconds, and if no other motion is detected after this time, the MQTT broker sends a message to the outlet actuator to turn it off. We will build this motion sensor together in Chapter 3 and we will see the automation I described using the garage lights actuator in Chapter 5 and Chapter 7 implemented in different ways.

By using the Arduino pubsubclient library (https://github.com/knolleary/pubsubclient), it is possible to implement a software code for an MQTT client. We will use another approach in this book by using software installed in the sensors so you will not need to create your own software to implement an MQTT client. Regarding the MQTT broker, Home Assistant has an integration that allows it to be configured and implemented.

For now, the only important information about MQTT you should know besides the basic information provided previously is some parameters that need to be configured on the MQTT client or server:

• MQTT Host IP: This is the IP address used to set up the MQTT broker.
• MQTT Port: This is the port used to connect the MQTT service. A typical value is 1883.
• Topic: This is the name used to identify the topic.
• Full Topic: This is the full name used with the topic to identify certain classes of devices.

More information about the MQTT protocol can be found on the project's web page (https://mqtt.org).

User interface and other clients

The user interface is a component part of the home automation server, which allows the end user to interact with and configure the home automation system. In the case of this book, the user interface is implemented by Home Assistant.

The other clients are external devices that are not necessarily a direct part of the home automation system but can be used to access the automation server and use the Home Assistant user interface to control or configure different devices in your home. Some examples of these clients are other computers or your smartphone connected to your home wireless network.

Note

Your smartphone can be used to access the system remotely and as a sensor to indicate you are at home.

In this section, we were able to understand what the components of a home automation system are. We also explained how these components work and how they can be connected and used as intended.

In the next section, I will provide you with an idea of how my home automation system is distributed in terms of sensors and actuators. You will learn about how many devices I have, where they are located, associated costs, and so on. I hope that this motivates you to create your own.