In this book, we will look at the concepts of continuous integration (CI) and continuous delivery (CD) and apply these concepts using tools such as Jenkins, Travis CI, and CircleCI. There will be many hands-on scripts that we will write, and we will explore real-world CI/CD automation scripts and scenarios. This chapter will help illustrate the concept of automation by explaining the current practices of a fictitious company called Billy Bob's Machine Parts. Billy Bob's Machine Parts has a lot of manual processes in place and there is some tension between quality assurance (QA) and the developer team because the software releases are only done by the lead developer and all QA testing is done manually.

The following topics will be covered in this chapter:

• Manual processes – a hypothetical scenario
• Employee frustration
• Introducing automation
• Developer productivity
• Breaking down communication barriers
• Creating an environment of collaboration

This chapter will describe a mock manual process and the flaws inherent in manual testing and manual processes, and will explain how using CI/CD can drastically improve developer productivity. In this scenario, each member has set up a manual set of processes that are quite time consuming to finish. Additionally, the steps have to be done again if QA encounters issues in the latest release version.

We will look at different scenarios across multiple departments in our fictitious company. Some of the scenarios will look at pain points for the developer team, QA team, customer success team, and sales team. We will build up possible scenarios that could occur across these teams, identifying areas that are good candidates for automation and how communication across these teams exposes areas that can be greatly improved with automation.

The following diagram shows a few business scenarios:

Betty Sue is part of the QA department at Billy Bob's Machine Parts company. At Billy Bob's Machine Parts company, there is a mid-sized developer team. Eric, the lead developer, starts the manual release process on Thursday morning at the end of a quarter. It takes two days for Eric to finish the release cycle. Unfortunately, he is the only one who can do the release in the developer team. Eric runs all the tests on his local workstation and integrates emergency patches if necessary. Once Eric is done, he emails a ZIP file to Betty Sue in the QA department.

Betty Sue has a couple of QA engineers and starts the manual testing cycle for the new release on Monday morning. Betty notifies Eric that she has already found several issues in the latest release. Betty gets an Excel spreadsheet ready that documents the issues that the latest release has introduced. At the end of the week, Betty has broken down the list of issues with the latest release into critical-, high-, medium-, and low-priority bugs.

During a release cycle, both Eric and Betty redo each step when issues are addressed. Eric has to repackage all the software components and rerun all of the tests on his local workstation. Betty has to redo the testing cycle because she has to check for regressions and ensure that the latest fixes do not break existing functionality in software components.

Michael, a junior developer on the team, is also doing manual processes. Michael gets a list of issues from Eric and starts working on the more high-priority bugs from the list. Michael tries to address and fix each bug but does not write any regression tests, ensuring that the newer code has not broken existing functionality. When Michael is done, he tells Eric that everything is good on his end. Unfortunately, Eric sees test failures when he runs all the tests on his local workstation. Eric informs Michael that he needs to be more careful while working on fixing issues on the bug list.

Dillon, a member of the QA department, starts testing parts of the new release and tells Betty that there are several issues with the release. He has created a checklist of issues and sends it to Betty. Unfortunately, some of the work that Dillon has done has been duplicated by Betty as they have similar items highlighted in two different checklists. Betty informs Dillon that QA needs to make sure that duplicate work is not being done. Dillon goes back and highlights parts of the release that he will be testing.

Jennifer leads the customer success team and gets notified when newer releases are ready to be opened up to customers by the QA department. Jennifer starts preparing videos of the latest release features and consequently asks QA questions about changes in the new release.

Bobby is an experienced member of the customer success team and starts creating videos on the latest features. When the release videos are published to the company blog, QA realizes that some of the videos incorrectly state features that are still in the beta release schedule. Jennifer now quickly scrambles the customer success team and asks QA to clearly mark some features as beta before they are sent to the customer success team.

The sales team has been emailing notes that sales engineers have taken during meetings with prospective clients. Sandy has hand-typed detailed notes about each prospective client and uses an Excel spreadsheet to categorize important sales information. Unfortunately, the sales team emails new changes to Excel spreadsheets to the sales department. Sometimes, confusion ensues because a sales engineer will open an older Excel document and mistakenly give outdated information to other sales engineers.

The UI/UX team tends to work with a lot of mockups and wireframes. Often, during the prototyping phase, UI/UX engineers will embed notes into mockups with transition notes detailing validation states and page interactions. Victor sees a note in one of the wireframes and realizes that there is important page logic embedded in the wireframes. Victor asks the UI/UX team if the notes can be shared with the developer team. The UI/UX team also work with art boards and create ZIP files for each feature work. For instance, Sandy was assigned work on feature x and has been taking detailed notes on the UI interaction for the new page. A lot of work by the UI/UX team tends to be highly visual, and colors mean distinct things. The visual aspect of the work tends to imply certain actions that should occur during phases of UI flow. Developers tend to work with more concrete items so it is not always apparent what natural flows should occur. For instance, if you delete an item, do you bring up a modal, which is a little window for confirmation, or do you just immediately delete an item? When submitting a form, does the UI show indications of an error in a certain color and a warning in another color? What positions should the validation be placed in? Sometimes, UI interaction flows are not described in detail and the developers must go back and forth with UI/UX. Documenting why decisions were made in a decision file is important.

Betty Sue has emailed Victor a list of issues that are categorized by priority. Higher priority issues must be dealt with first, while lower priority issues are dealt with later. Victor gets the list of issues with the newest release and notifies the developer team that they must stop the new feature work they are doing immediately and start fixing the issues with the new release. David, a senior developer on the team, is frustrated because he was in a nice rhythm and now is scrambling to reorient himself with work he did a month ago.

Michael, a junior developer on the team and still new to the codebase, is worried about a higher priority issue that is on the list. Michael scrambles to resolve the higher priority issue but does not think to write any regression test cases. Michael quickly writes a patch for his high-priority ticket and sends his patch to Victor. Victor quickly finds regressions and broken test cases with Michael's patch. Michael did not know that he should write regression test cases to ensure that there is no regressions.

The process of releasing a new patch is not properly documented and newer developers such as Michael frequently create regressions that break existing work. Victor teaches Michael the concept of regression testing and Michael quickly writes a software patch with regression test cases.

Once Victor gets all the new software patches ready, he gets started on a hot-fix release and reruns all the tests on his local machine. Betty gets a new ZIP file with the latest release and starts the manual test processes again. The QA department is manually testing parts of the product, so it is a time-consuming task to test all the parts of the product. Betty has found some more issues with the latest release and sends Victor a smaller list to start working on later in the week.

David is abruptly stopped by Victor and is told to drop his new feature work because the latest changes have defects. David spends the next two hours trying to reorient himself with the issues in the new release. Once confident that he has tracked the issue, he spends the afternoon doing a fix. David notifies Victor that the latest changes are ready to be tested. Victor starts running tests on his workstation and immediately sees that some integration tests are now failing due to the latest changes, and notifies David that these issues must be addressed. David is now frustrated and works late into the evening getting another fix out. The next morning, Victor runs all the tests and everything is passing, so he sends Betty a new ZIP for the latest hot fix. Betty starts the manual testing process the next day and unfortunately finds a couple of small issues again and lets Victor know in the afternoon that there are still some issues with the latest release.

Victor, admittedly frustrated at this moment, huddles all the developers into a room and says no one is to leave until all the issues are addressed. After a long evening stuck in the office, all the latest issues are addressed and Victor lets everyone go home. The next morning, Victor packages up the latest release and emails Betty a new ZIP file. Betty, a little worried after the last test cycle, is pleased that all the bugs have been addressed and gives the QA stamp of approval and lets Victor know that the latest release is ready to go. Both the developer team and the QA team celebrate the end of the work week with a company-sponsored lunch and go home for the weekend.

While QA was testing the hot fix, some of the QA team members had overlapping work. Dillon was frustrated because some his work overlapped work that Betty had done. There is no automation in place in the QA department, so all the work is done manually for each release regardless of whether it is a patch or regular release, and QA must retest all the parts of the UI. Nate, a newer member in the QA team, asks Dillon whether there is a better way to work than doing manual testing but is told that these practices are already in place in the QA department.

Tony, a member of the customer success team, is frustrated with the new release because he spent a lot of time creating new videos for customer X and was told that some of his videos cannot be published and will need to be put in a reserve library. The QA department made a decision at the last moment to halt feature Y but did not communicate this information to the other departments.

Victor, one of the lead sales engineers, is doing a company demonstration and shows an export PDF feature to a prospective client. During the demo, Victor clicks on the export PDF feature and gets a glaring error message. Victor quickly moves to another aspect of the product, saying that this is a temporary glitch and that he will showcase this in another demonstration. Victor finds out that one of the developers made what was supposed to be a simple change in one of the backend services and broke the export PDF feature in the production environment. Victor finds out that the prospective client has decided to go with another software solution and is now visibly distraught because he was counting on this new client to get his end of year bonus.

Samantha, a member of the UI/UX team, is told that one of her mockups is missing a validation flow. Samantha was seeking clarification during the prototyping phase for feature Z and was told that the page did not need any validation, but David has argued that a validation flow is needed. Samantha is visibly distraught and decides to take the day off, and now David is behind schedule for feature Z work:

Here is an illustration called Johnny The Automation Bot, which is used to depict the different departments in a company. Each limb on Johnny represents a different department in a company:

Johnny The Automation Bot is an illustration of the areas that can benefit greatly from automation processes. Automation can be thought of as a procedure or system whereby a machine completes a job that a human would ordinarily do. Automation requires knowing what manual processes are being done, communicating with other departments, and finding what processes are being done manually. CI and CD, as we will see later in the book, are processes that greatly enhance company productivity and processes because they remove developer assumptions and specific environment settings.

The QA department is manually testing the product, and manual testing is a time-consuming and error-prone activity. Johnny suggests that the QA department starts writing acceptance tests using Selenium WebDriver. Selenium is a browser automation tool and the QA department can use a language such as Python to write acceptance tests. Johnny says that the advantage of writing automated tests with Selenium is that they can be written once and reused again and again. This will have the added benefit that these tests can be hooked into a CI/CD pipeline, as we will see later in the book.

Betty from the QA department finds out that the customer success team is making a set of videos teaching customers newer features in each build. Customer success is uploading videos through FTP and some of the customer success team spends a large part of their day uploading files. Johnny The Automation Bot suggests that this process be automated through a script. The script should be intuitive enough that any member of the customer success team can run it and it should do the job of uploading and retrying the upload if any network latencies occur during the upload process. Betty shares a script that QA has written that can automate this process and be invoked to run as a background process.

Tony, a member of the customer success team, has now freed hours of work in his work day and can focus on the more important aspects of his job, such as setting customers up for success by creating fantastic videos. Tony has started working with the QA team and is going to start publishing videos and conducting user acceptance tests on parts of the product. QA is now able to better test features because the manual testing has been delegated to the customer success team. QA is focusing on automating the end-to-end test suite with a newer library that will help them write tests faster and, in turn, notify the developer team of broken functionality.

The marketing team has been embedding notes in PowerPoint slides and, at times, the notes are lost or overridden during presentations or company demonstrations. Johnny suggests that the developer team create a script that can convert the PowerPoint slides into markdown and that the markdown files can then be version controlled because markdown is just text files. This will have the added bonus that marketing can share information with the sales team to create more illustrative charts.

Victor has come to the realization that manual processes are destroying productivity and that manual processes have distinct disadvantages. Victor could introduce an automated system in the release cycle that be run be run by any developer in the team through a one-click deployment button. Instead of running all of the tests on a local workstation like Victor is currently doing, each software build can be pushed to a version control system such as GitHub, and all of the tests can be run on a CI environment, such as Jenkins, and developers can be notified automatically whether tests are passing or failing. Bruce, for example, is a newer developer on the team who can quickly read the developer documentation and start doing the next release with little or no guidance. Johnny The Automation Bot gives a big thumbs up for this practice.

Betty also has an opportunity to automate the manual testing process. Using a tool such as BrowserStack, Betty can write a series of testing scripts that test each part of the product. Within an hour, Betty can run a suite of acceptance tests in a testing environment and let Victor know the latest issues in the release. Victor can start assigning issues to the developer team and start writing regression test cases that ensure that there is no regressions in the current build. Victor, confident that the latest changes work as intended, can point Betty to a new URL where she can download the latest software release. Johnny The Automation Bot points out that the older practice of creating a ZIP file and emailing it was not a good practice because it requires an extra step each time and can be error prone if the wrong ZIP file is sent. Johnny suggests that the QA department has a dedicated URL where all the latest releases live, and that each release be versioned and state specific information such as a hotfix. For example, the latest hotfix could be v5.1.0-hotfix 1, and so, for each hotfix, the QA department would have a compressed file that has latest build and a specifier, such as hotfix. If this build was a regular build, then it could be named v5.1.0.

Victor finds out that the QA department has a Browser Stack account. Browser Stack provides access to an entire suite of browsers and mobile clients, and this can help automate the load testing of the UI. Load testing is being done with a custom server that the developer team uses for special scenarios, such as load testing. Johnny The Automation Bot suggests either using services like Browser Stack or having a custom service that can provision the necessary resources to conduct load testing.

Victor finds out that the QA team is having issues testing an email service written by the developer team. Johnny The Automation Bot suggests that the developer team make sure that QA has any scripts that can help work with the email service. Victor tells Betty that the new email service is proxying to the SendGrid service and that the developer team has already written a series of scripts that QA can use. The scripts help write a test email and can help QA test what happens under failure conditions.

The UI/UX team is uploading mockups to Sketch—Sketch is a prototyping tool—and embedding notes about possible validation states and flows in a page. The notes are extremely detailed and can be quite helpful to the developer team when starting feature work in company sprints. Johnny The Automation Bot suggests that the development team writes a plugin that can help the UI/UX team easily share this information. Victor decides to create a Sketch plugin that creates a PDF with the embedded notes that the UI/UX team can email to the developer team when the prototyping is done. This plugin is easy to install for the UI/UX team because all they have to do is double-click the file and it automatically installs the plugin. Access to the PDF and the embedded notes will help the developers understand use cases and UI flows for the newer feature.

Vincent, a lead sales engineer, has communicated to the developer team that he needs to be informed of process changes in the product, especially when talking to prospective clients about newer features on the company roadmap. Johnny The Automation Bot suggests that the developer team utilize the Git commit log which has detailed information about the latest feature changes. Victor has written a script that scrapes the Git commit log and writes a nice markdown document with all the latest feature work. In turn, the customer success team can work with the developer team and use the markdown file to create a nice blog entry on the company blog detailing all the latest features.

Let's reiterate some of the processes that were automated and enhanced by open collaboration. Victor helped QA automate an email-testing service issue by providing the scripts that the developer team created. QA helped customer success automate a video-uploading task by sharing a script that uploads the video and that has retry logic. The sales department expressed a need to have better visibility on newer features of the product; this caused the developer team to write a script that grabs information from the Git commit log to produce a markdown file, which the customer success team uses to write a nice blog entry in the company blog. The UI/UX team now has a plugin integrated into their Sketch application where they can simply click a button to generate a PDF document with notes taken during the prototype phase, which in turn helps the developer team work on newer features. The developer team finds out that QA is using a tool called BrowserStack and starts using it to conduct load testing on the product. The marketing team now has versioned copies of the marketing PowerPoint slides and is now sharing this information to the sales team to create newer charts for company demonstrations.

The UI/UX team has decided to create a style guide where developers can go for common UI patterns in the software product. The UI/UX team discovered that many different styles were being used in different pages, which was leading to confusion for a lot of customers. For instance, the Parts Supply page has a big blue save button on one page and a red cancel button, but on the Supplier Details page there is a big red save button and a blue cancel button. Customers are clicking the wrong buttons because the UI is not using colors universally. Sometimes, a page uses a confirmation modal to add and delete an item; other times, there is no confirmation modal. The UI/UX team has started working on the style guide and on creating a special URL in the company intranet where the live style guide will reside. The intent is to explicitly create and list all the usable hex colors for the page, design all the buttons in the product, and decide how a form should look and behave on the pages.

Also, there will be a special widget page that has the HTML markup and styling embedded for all the specialized widgets in the product:

Since Victor implemented a CI/CD pipeline into the the build, many time-consuming activities are now relegated to an automated pipeline. Whenever software is pushed upstream to a version control system (vcs), such as Git, an automated build is triggered in Jenkins that runs all of the unit and integration tests. A developer can quickly know whether the code that they wrote has introduced a defect. Remember that Victor had to incorporate all the software patches and manually run all of the tests on his local workstation. This was tedious, time-consuming, and unnecessary.

Since all software is pushed upstream, Victor has set up a code cutoff date for the release branch and has started versioning the software release binaries so that QA can more clearly delineate each build. Victor is immediately more effective as he can start delegating the release cycle to other developers in the team. Any issues encountered in the release cycle can be documented in the release by any developer. Victor now has more time to start planning for the next software cycle and mentoring junior developers in the team. David is now pleased because he can push his latest changes to source control and have all the tests run in a CI environment and be more confident that his changes work as intended.

Betty has set up an entire suite of acceptance tests that check each part of the software product. Any regressions in the product immediately surface in the CI environment and all of the tests can be run daily. The tests that QA are running are more time-consuming and resource intensive than the developer team's tests because they are end-to-end tests, but the advantage for the QA team is that all of the tests are run daily and QA gets a nightly report that details any test failures. Betty has written a collection of page objects that help other members in the QA team reuse other testing scripts and reduce testing cycles. Betty now has time in the QA cycle to mentor newer members in the QA department on testing practices and how to properly mark issues for the developer team so that they know where the issue is in the latest release:

David can now start helping Victor to mentor junior developers in the team, and the developer team has started doing a lunch-and-learn series whereby other developers can share knowledge throughout the team. The developer team quickly realizes that these lunch-and-learn sessions are also applicable to the QA department. In one of the lunch-and-learn sessions, QA proposes a change to the developer team about coordinating releases between QA and the developer team. Through this partnership, the release cycle shrinks from a week-long process to a 3-hour process. The developer team rotates the release work so that each developer in the team can learn how to do a release. The developer on duty makes sure that QA has a build available for QA to start testing with, and this build can be triggered automatically using a CI system, such as Jenkins, Travis, or CircleCI. In these CI environments, you can set build triggers that run at a specified date and time. The QA department notifies the developer team of any regressions in the release, and whenever the developer team is ready to push up a hot fix, the build is clearly delineated with the following pattern—vMAJOR.MINOR.PATH-[hotfix]-[0-9]*. For the sake of clarity, here is an example—v6.0.0-hotfix-1. This delineates major version 6, minor version 0, patch version 0, and hot fix number 1. This naming scheme is helpful to the QA department in differentiating regular builds with hotfix builds.

The customer success team has communicated to the developer team that some customers are having issues working with the Billy Bob's Machine Parts Application Programming Interface (API) service. The customer success team asks the developer team whether there is any way to help on board newer third-party API consumers. To clarify, an API consumer is someone who is consuming/using an existing API, whereas an API provider is the one who maintains the actual API service; so in this regard, Billy Bob's Machine Part is the API provider that provides a running API for third-party developers to use. The developer team tells the customer success team that they have been meaning to create a developer portal, which would help the API consumers easily work with the API. The developer team has had a hard time convincing upper management of the value of a developer portal, however, because no one has asked for this specific feature. The customer success team quickly convinces upper management that a developer portal would be a tremendous asset for Billy Bob's Machine Parts API consumers and that API consumers can start building nice dashboards using data from the API service.

In one of the developer meetings, it is discovered that the marketing team is using Google Docs to share documents, but that it is hard to find the uploads because you have to know what you are looking for. Victor quickly realizes that the developer team can help build a company intranet where the sales and marketing teams can share data in a more coherent manner. After a couple of months, the company intranet is exposed and the sales and marketing teams excitedly mention that the company intranet has helped them automate document sharing processes and that many meetings in the past between sales and marketing suffered countless wasted hours trying to find certain documents. The company intranet exposed a filtering mechanism whereby documents can be quickly found using a tag system. Another feature that the company intranet has enabled is the ability for the sales and marketing teams to edit the shared documents.

The sales team now has a great tool to utilize with the company blog, presenting new product features. Victor can now look at the company blog to find out the latest features in the product. This has all been enabled because of the script that Victor wrote that scrapes the Git commit log for commit messages, which in turn publishes a nice markdown file. The script is used on each release and has a listing of all the items worked on, and the developer team sends the newly created markdown file to the customer success team, and they in turn write a nice blog entry that discusses all the details of the latest release based on this markdown file.

The QA team started working on a ticket where a part limit causes a specific UI bug. In particular, if a customer has over 10,000 part listings on a product detail page, then the product detail page crashes with no helpful indication of what occurred. The developer team discovered that the QA team was manually creating new products in the new product page. The developer team helped the QA team by making QA aware of an admin endpoint that creates voicemails programmatically. The developer team helped write a script that generated new parts programmatically and thus saved the QA team from doing the time-consuming task of creating parts manually.

In order for automation to take place, communication barriers must be broken down between teams. At times, different parties can think they are on the same page but in reality they are talking about different things.

It is important for the channels of communication to be open in order to combat misunderstandings:

Interestingly, there is still more room for automation within the release cycle. Victor asks Betty about some of the acceptance tests that are in source control. Victor realizes that he could integrate the acceptance tests into a CI environment and create a secondary build where all of the acceptance tests are run each night and get QA would get a report detailing the latest failures each morning. The QA could then review the failing acceptance tests each morning and notify the developer team that feature x has broken—for example, the Parts Supply page—and that the developer working on this newer feature needs to recheck the new business logic.

David starts talking to the UI/UX team and finds that there is a bottleneck between newly exposed API endpoints and the construction of newer pages. The frontend developers are mocking out data in these pages and are constantly surprised by unexpected JSON payloads. The frontend developers are sometimes waiting for weeks for the API endpoint to be published, and instead of sitting by idly, they start mocking out data. This has the unintended consequence that they start assuming what the data model will look like, which in turn makes changing the page more difficult. David lets Victor know that there are tools that exist that can quickly scaffold a data model for an endpoint and give the frontend developers the current data model for an API endpoint. David starts using Swagger, which is an API design framework, as a tool for building newer APIs in the API service. Swagger helps reduce the friction between the developer team and the UI/UX team that was unnecessarily caused because the UI/UX team was waiting on the data model. Jason, a senior UI/UX developer, can now quickly start working on building newer pages because he knows exactly what type of payload to expect from newer API endpoints.

Amanda, a member of the QA team, has started working with the customer success team about load testing and user acceptance testing. Acceptance tests have been added during the user acceptance testing cycle that expose areas where UI/UX can be improved in the core product. The customer success team now has the added responsibility of testing newer pages and exposing possible UI issues. Acceptance tests are good for testing happy path scenarios, meaning when everything is done as expected, but user acceptance testing can expose unintuitive workflows in a UI. Larry, for example, started testing a newer filter feature in the Parts Supply page and discovered that in order for the filtering to start working you need to click on a checkbox. Larry asks QA why filtering cannot be done by default and why a checkbox is necessary; the developers then start working on adding filtering by default:

Justin, a member of the customer success team, asks Francis, a member of the QA team, whether he can borrow a video of a newer feature that was tested by the QA department. Justin realizes that the QA team has an extremely valuable set of videos that the customer success team can utilize to teach customers how to use the newest features. Francis creates an internal portal for the customer success team to use when newer videos are released from the QA department. The customer success team has been creating onboarding videos for newer customers and has designed a knowledge portal that explains how to set up, for instance, a newer Parts Supply page.

The sales team has been emailing notes to personal email accounts on discussions with clients and prospective leads. Victor discovers that Harry, a sales manager, recently lost some valuable notes because he accidentally deleted notes he took while having lunch with a prospective client. Victor tells Harry that there is a new company intranet and that there is a projects page that has cards that you can create, so the sales team can create a sales deck for each prospective client. Harry creates a new sales deck for a prospective client and shares it with Jim, the chief sales executive. Jim expresses great excitement because he also realizes that the company intranet can be used to create charts as well. Jim uses the following chart to showcase the latest sales leads to the chief sales executive:

The developer team has discovered that the customer success team has been using the UI for processes that are quite time consuming and error prone. Victor, the lead developer, sets an initiative to create a Command Line Interface (CLI) that the customer success team can use to automate many parts of their current workflow that have manual processes. The developer team explains how the CLI can save countless hours for the customer success team and that it can also help API consumers work better with the API service. The CLI can be used to quickly seed the UI with important page data. As each new release might expose newer endpoints, an initiative is created by the developer team to add additional commands to the CLI that work with newer API endpoints.

The CLI application will work in tandem with the developer portal initiative and will boost API consumer adoption. Along with this, the developer team decides to start an initiative on a Software Development Kit (SDK) that API consumers can utilize to work with the API. An SDK can greatly improve and enhance third-party adoption of an API provider and therefore increase API adoption. The SDK is especially useful because developers and QA work with different programming languages. The developers of the machine parts API are working with the Golang programming language while QA is utilizing Python for most of their work. The SDK will support many programming languages and will help API consumers get up and running quickly because they can pick a language of their choice to work with the API service.

In order for manual processes to be automated, there must be communication between different groups within an organization. There are bound to be manual processes throughout teams in an organization. The development leadership, the QA leadership, the customer success leadership, and the UI/UX leadership start meeting once a month to discuss newer practices and start finding other manual process within the company that are in need of automation.

Manual processes are not inherently bad, and user acceptance testing (UAT) is still effectively done in companies and can help expose issues that automated tests will not find. UAT is especially helpful to test edge-case scenarios that automated tests at times will not uncover, as the example shown earlier demonstrated, where the customer success team tested a newer feature and discovered that the Parts Details page only enabled filtering if a checkbox was checked.

Marketing, sales, and customer success teams often utilize spreadsheet applications, such as a Excel, to compute numbers, and presentation applications, such as PowerPoint, to create charts. Often, the numbers that are calculated in an Excel spreadsheet are saved in revisions, but members in a team must email copies to other team members. Developers can request that the marketing, sales, and customer success teams export values in Excel in comma-separated value (CSV) form, which is a text file and much easier to work with. This has the added value that the company intranet can use data visualization tools to create fancy HTML charts and presentations. A library such as D3 can be utilized to create many different types of visualizations that are quite powerful.

In order for teams to start collaborating and openly discussing issues, there must exist a spirit of openness. It is too easy for teams to be siloed, meaning that they are disconnected from what other teams are doing. The developer team can just as easily choose to stay disconnected from the QA department. The issue with this is that communication is the most important element to expose manual processes between teams. Without communication, teams will independently work on items that they deem important.

Social activities where different teams participate together can help break barriers and establish a friendly environment. Often, developers like going to conferences simply for the social aspect of interacting with other developers, and there is often a hallway track, where developers stand outside and simply talk to each other instead of attending active sessions in a conference.

Companies can sponsor social activities and help appoint representatives in different teams who help break the ice between teams. For instance, at a company bowling event, people may be intentionally put into different teams than they are used to working with. A small bowling team can be comprised of a developer, a customer success team member, a QA team member, a marketing team member, and a sales team member. This can inspire a working relationship where the team members get to know each other and communicate openly about issues they are encountering outside of the company event.

The Billy Bob Machine Parts company scheduled a baseball event and Victor, the lead developer, and Betty, the lead QA member, worked together with several marketing team members, sales team members, and customer success team members. Two teams were formed for the baseball event and a company barbecue was scheduled after the game so people could eat together and have conversations.

Another way to encourage active collaboration is to alter company floor plans to be more open. Many software companies have been adopting open floor plans because they remove natural divisions that cubicles create among people. The idea is that if you have an open floor plan, you are much more likely to approach different teams because you can easily walk up to different people and not feel like you are invading their space.

Communication is the key to finding manual processes, and it is important to find manual processes in order to automate these manual processes. Manual processes tend to be error prone and time consuming, as we illustrated with the various business scenarios. That's where automation, such as implementing a CI build and writing scripts to automate manual processes, comes in. Developers and QA can help develop automation scripts that can benefit many different departments, such as sales, marketing, and customer success. In this chapter, you have learned about the benefits of automation over manual processes and the value of open communication.

In the next chapter, we will learn about the basics of CI.

1. What is a manual process?
2. What is automation?
3. Why is it important to open up communication between departments?
4. What does CI/CD stand for?
5. Why are automation scripts useful?
6. What is the value of a company intranet?
7. Why should other departments share data?

To get more of an introduction to CI and to use a popular CI/CD tool, consider reading
Learning Continuous Integration with Jenkins – Second Edition (https://www.amazon.com/dp/1788479351/), by Packt Publishing.