Book Image

Mastering Jenkins

By : jmcallister -, Jonathan McAllister
Book Image

Mastering Jenkins

By: jmcallister -, Jonathan McAllister

Overview of this book

With the software industry becoming more and more competitive, organizations are now integrating delivery automation and automated quality assurance practices into their business model. Jenkins represents a complete automation orchestration system, and can help converge once segregated groups into a cohesive product development and delivery team. By mastering the Jenkins platform and learning to architect and implement Continuous Integration, Continuous Delivery, and Continuous Deployment solutions, your organization can learn to outmanoeuvre and outpace the competition. This book will equip you with the best practices to implement advanced continuous delivery and deployment systems in Jenkins. The book begins with giving you high-level architectural fundamentals surrounding Jenkins and Continuous Integration. You will cover the different installation scenarios for Jenkins, and see how to install it as a service, as well as the advanced XML configurations. Then, you will proceed to learn more about the architecture and implementation of the Jenkins Master/Save node system, followed by creating and managing Jenkins build jobs effectively. Furthermore, you'll explore Jenkins as an automation orchestration system, followed by implementing advanced automated testing techniques. The final chapters describe in depth the common integrations to Jenkins from third-party tools such as Jira, Artifactory, Amazon EC2, and getting the most out of the Jenkins REST-based API. By the end of this book, you will have all the knowledge necessary to be the definitive resource for managing and implementing advanced Jenkins automation solutions for your organization.
Table of Contents (18 chapters)
Mastering Jenkins
Credits
Foreword
About the Author
About the Reviewers
www.PacktPub.com
Preface
Index

Jenkins on Mac OS X


For Mac OS X users, the Jenkins community provides a native PKG installer. This installer has a similar guided installation wizard as the Microsoft Windows one, and allows us to specify the traditional installation details. For users who wish to alter the Jenkins listening port or memory options, some command-line magic will be required.

To alter the Jenkins listening port, we need to explicitly define the port in the following properties file:

/Applications/Jenkins/winstone.properties

To accomplish this we will need to create the winstone.properties file if it does not already exist on our host, and detail the httpPort parameter inside it. An example of how to set httpPort in the winstone.properties file is shown here:

httpPort=80

Once created, the winstone.properties file will automatically load and override the default Jenkins port with the one specified within.

The winstone.properties file is not limited to simply altering the Jenkins listening port. There are other options available for customization as well. These options include logfile, httpListenAddress, and more. To obtain a complete list of the available override options, you can run the following commands from your OS X terminal:

cd /Applications/Jenkins java –jar Jenkins.war --help

The JVM runtime memory settings (Heap memory, PermGen, and so on) are stored in a standard properties file format (org.jenkins-ci.plist). The launch daemon retrieves the values stored in this properties file. If no such file exists, the system will use the built-in defaults. On Mac OS X, this plist file will typically reside in the following location:

/Library/Preferences/org.jenkins-ci.plist

Adjusting the Java memory options for Jenkins involves modifying the appropriate property entries inside the org.jenkins-ci.plist file. To modify this file, we can use the OS X defaults command. This command allows us to read and write entries in the plist file without fear of corruption or improper formatting. A few example descriptions and use cases for this command are detailed in the sections below.

To view all settings in the plist file, execute the following command in the command-line terminal:

sudo defaults read /Library/Preferences/org.jenkins-ci

The output of the preceding command will look like this:

{
    heapSize = 512m;
    minHeapSize = 256m;
    minPermGen = 256m;
    permGen = 512m;
    tmpdir = "/Users/Shared/Jenkins/tmp";
}

To retrieve the value of the heapSize setting from the plist file, execute the following command in the command-line terminal:

sudo defaults read /Library/Preferences/org.jenkins-ci heapSize

The output of the preceeding command will look like this:

512m

To set the value of the heapSize setting in the plist file, execute following command in the command-line terminal:

sudo defaults write /Library/Preferences/org.jenkins-ci heapSize 1024m

If any alterations are made to the org.jenkins-ci.plist file, make sure you restart Jenkins for them to take effect. To restart Jenkins from the OS X command line terminal enter the following commands in the terminal:

sudo launchctl unload /Library/LaunchDaemons/org.jenkins-ci.plist
sudo launchctl load /Library/LaunchDaemons/org.jenkins-ci.plist

The Jenkins LTS release line

The Jenkins community recognizes that installing an edge release may be risky, and upgrading weekly (the default option) may pose a bit of an overhead in maintenance. For the more conservative users, the Jenkins Long-Term Support (LTS) release may be a more viable option. The Jenkins LTS release is delivered once every 12 weeks (instead of every week) and is selected by community vote. The Jenkins LTS release represents a community-voted selection of the most stable Jenkins release within the past 12 weeks.

In this section of Mastering Jenkins, you will learn about the Jenkins LTS release and understand how to convert an existing edge Jenkins installation over to the LTS release line. The Jenkins platform features a streamlined upgrade process, and typically provides all upgrades through the Jenkins UI. To convert a Jenkins installation over to the LTS line, there are two options available:

  • Uninstall and replace the existing latest and greatest installation with the LTS package (immediate, but nuclear, option)

  • Convert an existing installation and point it to the LTS update URL (waits for the next LTS release)

This section will focus on converting an existing installation over to the LTS release line. We can do this by pointing our Jenkins instance to the LTS update URL. This is because uninstalling and reinstalling the Jenkins platform is a straightforward process and is already documented in a number of places.

To migrate our Jenkins installation to the LTS release line, we will need to modify hudson.model.UpdateCenter.xml, located in $JENKINS_HOME, to point our Jenkins instance to the LTS release update center URL. The Hudson.model.UpdateCenter.xml file is what Jenkins uses to determine where it should look for updates to the system. The contents of this XML file are shown here:

<?xml version='1.0' encoding='UTF-8'?>
<sites>
  <site>
    <id>default</id>
    <url>http://updates.jenkins-ci.org/stable/update-center.json</url>
  </site>
</sites>

As you may have guessed already, the node in the XML that we will need to alter is the <url> node. The Jenkins LTS release has its own update center URL. Let's replace the existing update center URL with the one shown here:

http://updates.jenkins-ci.org/stable/update-center.json

Once the file is modified and saved, we need to restart the Jenkins service to complete the switchover to the LTS release line for all future updates.

Note

The LTS release comes out every 12 weeks. We will need to wait for this cycle to complete before our Jenkins instance is completely switched over.

The Jenkins LTS release is also available as a Docker container. This means that if the target setup is new, we can leverage the LTS Docker container (if desired) to perform the duties of the Jenkins master. Details on the Jenkins official LTS Docker container can be found at the following URL:

http://jenkins-ci.org/content/official-jenkins-lts-docker-image

Jenkins XML configuration files

Configuration data in Jenkins is persisted to disk via XML files. These XML files contain information describing how the Jenkins instance will behave. Understanding how Jenkins implements configuration XML files and manages the data they contain can prove to be valuable in debugging issues and keeping the system stable.

In Jenkins, persistent configuration data is serialized into XML and subsequently written to disk. The primary Jenkins subsystem serializes its data into config.xml files. These config.xml files govern the overall Jenkins system and describe how Jenkins will behave upon startup. The primary config.xml configuration file can be found in the following location:

$JENKINS_HOME/config.xml

An example of this configuration file is provided here (taken from an Apple OS X installation of Jenkins):

<?xml version='1.0' encoding='UTF-8'?>
<hudson>
  <disabledAdministrativeMonitors/>
  <version>1.0</version>
  <numExecutors>2</numExecutors>
  <mode>NORMAL</mode>
  <useSecurity>true</useSecurity>
  <authorizationStrategy class="hudson.security.AuthorizationStrategy$Unsecured"/>
  <securityRealm class="hudson.security.SecurityRealm$None"/>
  <disableRememberMe>false</disableRememberMe>
  <projectNamingStrategy class="jenkins.model.ProjectNamingStrategy$DefaultProjectNamingStrategy"/>
  <workspaceDir>${ITEM_ROOTDIR}/workspace</workspaceDir>
  <buildsDir>${ITEM_ROOTDIR}/builds</buildsDir>
  <markupFormatter class="hudson.markup.EscapedMarkupFormatter"/>
  <jdks/>
  <viewsTabBar class="hudson.views.DefaultViewsTabBar"/>
  <myViewsTabBar class="hudson.views.DefaultMyViewsTabBar"/>
  <clouds/>
  <slaves>
    <slave>
      <name>Windows 2012</name>
      <description></description>
      <remoteFS></remoteFS>
      <numExecutors>1</numExecutors>
      <mode>NORMAL</mode>
      <retentionStrategy class="hudson.slaves.RetentionStrategy$Always"/>
      <launcher class="hudson.plugins.sshslaves.SSHLauncher" plugin="[email protected]">
        <host></host>
        <port>22</port>
        <credentialsId>0bb868e0-2cd6-4ab2-9781-a373d914cb85</credentialsId>
        <maxNumRetries>0</maxNumRetries>
        <retryWaitTime>0</retryWaitTime>
      </launcher>
      <label>Windows Build Pool</label>
      <nodeProperties/>
      <userId>anonymous</userId>
    </slave>
  </slaves>
  <scmCheckoutRetryCount>0</scmCheckoutRetryCount>
  <views>
    <hudson.model.AllView>
      <owner class="hudson" reference="../../.."/>

    <name>All</name>
      <filterExecutors>false</filterExecutors>
      <filterQueue>false</filterQueue>
      <properties class="hudson.model.View$PropertyList"/>
    </hudson.model.AllView>
    <listView>
      <owner class="hudson" reference="../../.."/>
      <name>Build.TestApp</name>
      <filterExecutors>false</filterExecutors>
      <filterQueue>false</filterQueue>
      <properties class="hudson.model.View$PropertyList"/>
      <jobNames>
        <comparator class="hudson.util.CaseInsensitiveComparator"/>
      </jobNames>
      <jobFilters/>
      <columns>
        <hudson.views.StatusColumn/>
        <hudson.views.WeatherColumn/>
        <hudson.views.JobColumn/>
        <hudson.views.LastSuccessColumn/>
        <hudson.views.LastFailureColumn/>
        <hudson.views.LastDurationColumn/>
        <hudson.views.BuildButtonColumn/>
      </columns>
      <recurse>false</recurse>
    </listView>
  </views>
  <primaryView>All</primaryView>
  <slaveAgentPort>0</slaveAgentPort>
  <label></label>
  <nodeProperties/>
  <globalNodeProperties/>
</hudson>

As we can see, the nodes defined in the XML file provide configuration definitions for the overall Jenkins system. The nodes govern the overall behavior of the Jenkins system. Some of the configuration highlights include:

  • Number of executors on the master

  • Workspace folder definitions

  • Security authorization strategy

  • Master/slave definitions

  • View definitions (the tabs on the main Jenkins dashboard)

  • Slave agent ports

The second configuration XML we will investigate is dedicated to Jenkins jobs. These configuration files are located in $JENKINS_HOME/jobs/<JOBNAME>/config.xml. Each config.xml file belongs to a unique job defined in Jenkins. An XML DOM derived from the JenkinsExample job is provided here:

<?xml version='1.0' encoding='UTF-8'?>
<project>
  <actions/>
  <description></description>
  <keepDependencies>false</keepDependencies>
  <properties/>
  <scm class="hudson.scm.NullSCM"/>
  <canRoam>true</canRoam>
  <disabled>false</disabled>
  <blockBuildWhenDownstreamBuilding>false</blockBuildWhenDownstreamBuilding>
  <blockBuildWhenUpstreamBuilding>false</blockBuildWhenUpstreamBuilding>
  <triggers/>
  <concurrentBuild>false</concurrentBuild>
  <builders/>
  <publishers/>
  <buildWrappers/>
</project>

As we can see from the sample config.xml provided, the Project XML DOM contains persistent data about a given job, its build steps, and any related automation. This includes information related to SCM solutions, triggers, builders, publishers, buildWrappers, and more.

Tip

It is highly recommended that all Jenkins configuration files are committed to source control. This will ensure that changes and history are preserved properly. This solution will also provide the ability to revert changes to a Jenkins job when needed.