Overview of this book
<p>Jenkins is an award-wining and one of the most popular Continuous Integration servers in the market today. It was designed to maintain, secure, communicate, test, build, and improve the software development process.</p>
<p>This book starts by examining the most common maintenance tasks. This is followed by steps that enable you to enhance the overall security of Jenkins. You will then explore the relationship between Jenkins builds and Maven pom.xml. Then, you will learn how to use plugins to display code metrics and fail builds to improve quality, followed by how to run performance and functional tests against a web application and web services. Finally, you will see what the available plugins are, concluding with best practices to improve quality.</p>
<p> </p>
<div class="book-toc-chapter"> </div>
<h2>Read an Extract from the book</h2>
<h2><span lang="EN-GB" style="color: windowtext;">Analyzing project data with the R plugin</span></h2>
<p class="NormalPACKT">This recipe describes how to use R to process metrics on each file in your project workspace. The recipe does this by traversing the workspace and collecting a list of files of a particular extension such as Java. The R script then analyzes each file individually and finally plots the results in a graphical format to a PDF file. The workflow is common to almost all quality-related analysis of software projects. This recipe is easily customized for tasks that are more complex.</p>
<p class="NormalPACKT" style="margin-bottom: .0001pt;"><span style="letter-spacing: -.1pt;">In this example, we are looking at the size in words of the text files, printing to the console the names of large files and plotting the sizes of all files. From the visual representation, you can easily see which files are particularly large. If your property file is much larger than the other property files, it is probably corrupt. If a Java file is too large, it is difficult to read and understand.</span></p>
<h2><span lang="EN-GB" style="color: windowtext;">How to do it...</span></h2>
<ol>
<li>Create a free-style job with the name <em>ch5.R.project.data</em>.</li>
<li>In the <strong>Source Code Management </strong>section, select <strong>Subversion</strong>.</li>
<li>Add the<strong> Repository URL</strong> as <a href="https://source.sakaiproject.org/svn/profile2/trunk">https://source.sakaiproject.org/svn/profile2/trunk</a>.</li>
<li>In the <strong>build</strong> section, under <strong>Add build step</strong>, select <strong>Execute R script</strong>.</li>
<li class="line-numbers">In the <strong>Script text</strong> area add the following code:
<pre class="line-numbers"><code class="language-java">processFile
<- function(file){
text <- readLines(file,encoding="UTF-8")
if (length(text)> 500) print(file)
length(text)
}
javaFiles <-
list.files(Sys.getenv('WORKSPACE'), recursive
= TRUE, full.names = TRUE, pattern = "\\.java$")
propertiesFiles
<- list.files(Sys.getenv('WORKSPACE'),
recursive = TRUE, full.names = TRUE, pattern = "\\.properties$")
resultJava <-
sapply(javaFiles, processFile)
resultProperties
<- sapply(propertiesFiles,processFile)
warnings()
filename
<-paste('Lengths_JAVA_',Sys.getenv('BUILD_NUMBER'),'.pdf',sep ="")
pdf(file=filename)
hist(resultJava,main="Frequency
of length of JAVA files")
filename <-
paste('Lengths_Properties_',Sys.getenv('BUILD_NUMBER'),'.pd
f',sep="")
pdf(file=filename)
hist(resultProperties,main="Frequency
of length of Property files")</code></pre>
</li>
<li>Click on the <strong>Save</strong> button.</li>
<li>Click on the <strong>Build Now</strong> icon.</li>
<li>Review the console output from the build. It should appear similar to the following:
<pre class="line-numbers"><code class="language-java">At revision 313948
no change for
https://source.sakaiproject.org/svn/profile2/trunk since
the previous build
[ch5.R.project.data] $ Rscript /tmp/hudson7641363251840585368.R
[1] "/var/lib/jenkins/workspace/ch5.project.data/api/src/java/o
rg/sakaiproject/profile2/logic/SakaiProxy.java"
[1]
"/var/lib/jenkins/workspace/ch5.project.data/impl/src/java/org/sakaiproject/profile2/conversion/ProfileConverter.java"
[1]
"/var/lib/jenkins/workspace/ch5.project.data/impl/src/java/
org/sakaiproject/profile2/dao/impl/ProfileDaoImpl.java"
14: In readLines(file, encoding = "UTF-8") :
incomplete final line found on
'/var/lib/jenkins/workspace/ch5.project.data/tool/src/java/
org/apache/wicket/markup/html/form/upload/MultiFileUploadFi
eld_ca_ES.properties'
Finished: SUCCESS</code></pre>
</li>
<li>Visit the workspace and review the files <em>Lengths_Properties_1.pdf</em>, <em>Lengths_JAVA_1.pdf</em>.</li>
<li><span style="letter-spacing: -.1pt;">Notice the straggler files with a large number of lines. Property files should be of roughly similar length, as they contain, in this case, the international translations for the GUI.</span></li>
</ol>
<p class="NormalPACKT" style="margin-bottom: .0001pt;"><span lang="EN-GB"> </span>This feels like a well-balanced project, as there are only a few files that have a large number of lines of code.</p>
<h2><span lang="EN-GB" style="color: windowtext;">How it works...</span></h2>
<p class="NormalPACKT">You loaded in the profile2 tool from subversion <a href="https://source.sakaiproject.org/svn/profile2/trunk">https://source.sakaiproject.org/svn/profile2/trunk</a>. This code is used by millions of students around the world and represents mature, realistic production code.</p>
<p class="NormalPACKT">Within your R script, you defined a function that takes a filename as input and then reads the file into a text object. The function then checks to see whether the number of lines is greater than 500. If it is greater than 500 lines then the filename is printed to the console output. Finally, the function returns the number of lines in the text file.</p>
<pre class="line-numbers"><code class="language-java">processFile <- function(file){
text <- readLines(file,encoding="UTF-8")
if (length(text)> 500) print(file)
length(text)
}</code></pre>
<p class="NormalPACKT">Next, the script discovers the property and Java files under the workspace. The file search is filtered by the value defined in the <em>pattern</em> argument. In this case, <em>.java</em>:</p>
<pre class="line-numbers"><code class="language-java">javaFiles <- list.files(Sys.getenv('WORKSPACE'), recursive = TRUE,
full.names = TRUE, pattern = "\\.java$")
propertiesFiles <- list.files(Sys.getenv('WORKSPACE'), recursive =
TRUE, full.names = TRUE, pattern = "\\.properties$")</code></pre>
<p class="NormalPACKT">The list of filenames is passed one name at a time to the <em>processFile</em> function you have previously defined. The results are a list of file lengths that are stored in the <em>resultJava </em>and <em>resultProperties</em> objects:</p>
<pre class="line-numbers"><code class="language-java">resultJava <- sapply(javaFiles, processFile)
resultProperties <- sapply(propertiesFiles,processFile)</code></pre>
<p class="NormalPACKT">The <em>warnings()</em> function produces a list of issues generated while running the <em>sapply</em> command:</p>
<pre class="line-numbers"><code class="language-java">14: In readLines(file, encoding = "UTF-8") :
incomplete final line found on
'/var/lib/jenkins/workspace/ch5.project.data/tool/src/java/org/apa
che/wicket/markup/html/form/upload/MultiFileUploadField_ca_ES.prop
erties'</code></pre>
<p class="NormalPACKT">This is stating that a new line was expected at the end of the file. It is not a critical issue. Showing the warnings is a helpful approach to discovering corrupted files.</p>
<p class="NormalPACKT">Finally, we generate two histograms of the results, one for the Java file and the other for the properties files. The filename is created from a constant string followed by the <em>BUILD_NUMBER</em> environment variable that is set uniquely for each build. The <em>pdf</em> function tells R that the output is to be stored in a PDF file and the <em>hist</em> function draws a histogram of the results:</p>
<pre class="line-numbers"><code class="language-java">filename <-paste('Lengths_JAVA_',Sys.getenv('BUILD_NUMBER'),'.pdf',sep="")
pdf(file=filename)
hist(resultJava,main="Frequency of length of JAVA files")</code></pre>
<h2><span lang="EN-GB" style="color: windowtext;">There's more...</span></h2>
<p class="NormalPACKT">When writing R code for processing your files, don't reinvent the wheel. R has many libraries for manipulating text. The stringi library is one example (<a href="http://cran.r-project.org/web/packages/stringi/stringi.pdf">http://cran.r-project.org/web/packages/stringi/stringi.pdf</a>). Here is some example code that counts the number of words in a text file:</p>
<pre class="line-numbers"><code class="language-java">library(stringi)
processFile <-function(file){
stri_stats_latex(readLines(file,encoding="UTF-8"))
}
results<-processFile(file.choose())
paste("Number of words in file:", results[4])</code></pre>
<p class="NormalPACKT">The script defines the function <em>processFile</em>. The function requires a filename. The file is read into the <em>stri_stats_latex</em> function. This function is included in the <em>stringi</em> library. It returns a summary of the file as a vector (a series of numbers).</p>
<p class="IgnorePACKT" style="margin: 0in 0in 6.0pt 0in;">The <em>file.choose()</em> function pops up a dialog that allows you to browse your file system and choose a file. The call returns the fully qualified path to the file. It passes the value to the <em>processFile</em> function call. The results are stored in the results vector. The script then prints out the fourth number that is the number of words in the file.</p>
<blockquote>
<p class="InformationBoxPACKT" style="margin: 9.0pt 0in 9.0pt 0in;"><em>Another interesting R package for text mining is tm: (<a href="http://cran.r-project.org/web/packages/tm/tm.pdf">http://cran.r-project.org/web/packages/tm/tm.pdf</a>). The tm package has the ability to load a set of text files and analyze them in many different ways.</em></p>
</blockquote>
<div class="book-toc-chapter"> </div>
