Adopting a collaborative development process 
using GitHub

While you may be working on your own, it is good practice to adopt a development process that allows others to collaborate and one that ensures that the code is always ready to be deployed to production. We will achieve both aims by using a remote repository and Continuous Integration (CI).

A remote repository acts as a backup for all your code and makes it much easier to set up CI (testing, linting, and so on). We’ll use GitHub as I find it to have all the features needed, although other platforms, such as GitLab, are also valid and commonly used in the industry.

Rather than creating the repository through GitHub’s UI, we’ll use Terraform as set up earlier. To do so, we’ll first need a personal access token from GitHub, as explained at https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/creating-a-personal-access-token. The token will need the repo, workflow, and delete_repo scopes. This token is a secret and hence best placed in infrastructure/secrets.auto.tfvars and encrypted as described earlier in the Managing secrets section. The code should be placed into infrastructure/secrets.auto.tfvars as follows (replace abc1234 with your token):

github_token = "abc1234"

Terraform itself does not know how to interact with GitHub, which means that we need to install the GitHub provider to do so. This is done by adding the following code to infrastructure/main.tf:

terraform {
  required_providers {
    github = {
      source  = "integrations/github"
      version = "~> 4.0"
    }
  }
  required_version = ">=1.0"
}

With the provider present, we can describe the repository we would like to exist by adding the following code to infrastructure/github.tf:

variable "github_token" {
  sensitive = true
}
 
provider "github" {
  token = var.github_token
}
 
resource "github_repository" "tozo" {
  name       = "tozo"
  visibility = "private"
}

Finally, to actually create the repository, we need to initialize and apply Terraform as follows:

terraform init 
terraform apply

We should now set up git so that it knows about the remote repository. To do this, we’ll need the correct path, which will depend on your GitHub account name and the name of your project. As my GitHub account name is pgjones and this project is called tozo, the path is pgjones/tozo, making the following command:

git remote add origin [email protected]:pgjones/tozo.git

To have our local branch track the remote origin main branch, run the following command:

git push --set-upstream origin main 

To push our local changes on our main branch to the remote feature branch, run the following command:

git push origin main:feature

To pull the remote main branch to update our local branch, run the following command:

git pull origin main

Most in this industry operate a development workflow based on merge (pull) requests, which we’ll also adopt. This workflow consists of the following steps:

1. Develop a feature locally consisting of as few commits as makes sense (small changes).
2. Push the feature to a remote feature branch.
3. Open a merge request based on that branch.
4. Review the merge request, merging it to the main branch only if CI passes.
5. Pull the latest main branch and repeat.

With the repository created, we can now look at adding CI.

Adding continuous integration

GitHub provides a CI system called Actions that has a free tier, which we’ll use. To start, we need to create the following folder structure:

tozo
└── .github
    └── workflows

Now we can configure a workflow that runs jobs on every change to the main branch and every merge request by adding the following code to .github/workflows/ci.yml:

name: CI
 
on:
  push:
    branches: [ main ]
  pull_request:
    branches: [ main ]
  workflow_dispatch:
 
jobs:

This allows us to add jobs for the infrastructure, backend, and frontend.

Adding CI for the infrastructure code

We previously set up the commands to format and lint the infrastructure code as follows:

terraform fmt --check=true
terraform validate

To have these run as part of CI, we need to add the following job to the .github/workflows/ci.yml file within the jobs section:

  infrastructure:
    runs-on: ubuntu-latest
 
    steps:
      - name: Install Terraform
        run: |
          sudo apt-get update && sudo apt-get install -y gnupg             software-properties-common curl
          curl -fsSL https://apt.releases.hashicorp.com/gpg |             sudo apt-key add -
          sudo apt-add-repository "deb [arch=amd64] https://            apt.releases.hashicorp.com $(lsb_release -cs) main"
          sudo apt-get update && sudo apt-get install terraform
      - uses: actions/checkout@v3
 
      - name: Initialise Terraform
        run: terraform init
 
      - name: Check the formatting
        run: terraform fmt --check=true --recursive
 
      - name: Validate the code
        run: terraform validate

We can now add a job for the backend code.

Adding CI for the backend code

We previously set up the commands to format, lint, and test the backend code as follows:

pdm run format
pdm run lint
pdm run test

To have these run as part of CI, we will need to have a database service running as well, as the tests run against the database. Fortunately, GitHub supports PostgreSQL database services by running a PostgreSQL database alongside the CI job. We can make use of this database service and run the commands by adding the following job to the jobs section in .github/workflows/ci.yml:

  backend:
    runs-on: ubuntu-latest
 
    container: python:3.10.1-slim-bullseye
 
    services:
      postgres:
        image: postgres
        env:
          POSTGRES_DB: tozo_test
          POSTGRES_USER: tozo
          POSTGRES_PASSWORD: tozo
          POSTGRES_HOST_AUTH_METHOD: "trust"
        options: >-
          --health-cmd pg_isready
          --health-interval 10s
          --health-timeout 5s
          --health-retries 5
    defaults:
      run:
        working-directory: backend
 
    env:
      TOZO_QUART_DB_DATABASE_URL: "postgresql://tozo:tozo@        postgres:5432/tozo_test"
 
    steps:
      - uses: actions/checkout@v3
 
      - name: Install system dependencies
        run: apt-get update && apt-get install -y postgresql           postgresql-contrib
 
      - name: Initialise dependencies
        run: |
          pip install pdm
          pdm install
 
      - name: Linting
        run: pdm run lint
 
      - name: Testing
        run: pdm run test

We can now add a job for the frontend code.

Adding CI for the frontend code

We previously set up the commands to format, lint, test, and build the frontend code as follows:

npm run format
npm run lint
npm run test
npm run build

We can make use of the service and run the commands by adding the following job to the jobs section of .github/workflows/ci.yml:

  frontend:
    runs-on: ubuntu-latest
 
    defaults:
      run:
        working-directory: frontend
 
    steps:
      - name: Use Node.js
        uses: actions/setup-node@v2
        with:
          node-version: '18'
 
      - uses: actions/checkout@v3
 
      - name: Initialise dependencies
        run: npm ci --cache .npm --prefer-offline
      - name: Check formatting
        run: npm run format
 
      - name: Linting
        run: npm run lint
 
      - name: Testing
        run: npm run test
 
      - name: Build
        run: npm run build 

We now have everything we need in place to start developing our app. The folder structure at this stage is as follows:

tozo
├── .github
│   └── workflows
├── backend
│   ├── src
│   │   └── backend
│   └── tests
├── frontend
│   ├── public
│   └── src
└── infrastructure

We now have all of our checks running on every change to the main branch and for every pull request. This should ensure that our code remains at a high quality and alert us to any issues that may otherwise be missed.