Continuous integration with Jenkins – Tutorial

This Tutorial will see how to use the Jenkins continuous integration build server.

1. Using the Jenkins build server

Continuous integration is a process in which all development work is integrated as early as possible. The resulting artifacts are automatically created and tested. This process allows to identify errors as early as possible.

Jenkins is a popular open source tool to perform continuous integration and build automation. The basic functionality of Jenkins is to execute a predefined list of steps, e.g. to compile Java source code and build a JAR from the resulting classes. The trigger for this execution can be time or event based. For example, every 10-30 minutes or after a new commit in a Git repository.

Jenkins can,

• perform a software build using a build system like Apache Maven or Gradle
• execute a shell script
• archive a build result
• running software tests

Jenkins monitors the execution of the steps and allows to stop the process, if one of the steps fails. Jenkins can send out notification in case of a build success or failure based on our configuration on the project base.

Jenkins can be extended by additional plug-ins. For example, you can install plug-ins to support building and testing Android applications, docker, etc.

2. Installation and setup of Jenkins

Jenkins available on most of the platform, as it doesn’t have any limitations. For more details Refer Jenkins Homepage.

2.1 Installing of the Jenkins server on Ubuntu

Jenkins provides Debian/Ubuntu packages which install Jenkins and register Jenkins as start service. See the Install Jenkins on Ubuntu description The Linux installation creates a /etc/init.d/jenkins script which starts Jenkins automatically at boot time.

Jenkins stores all the settings, logs and build artifacts in its home directory. The default installation directory is /var/lib/jenkins under Ubuntu.

2.2 Using the .war file to start Jenkins

Download the jenkins.war file from Jenkins Homepage. From this file you can start Jenkins directly via the command line with java -jar jenkins*.war.

If you start it locally, you find it running under the following URL: http://localhost:8080/

To run it in your Tomcat server, put the .war file into the webapps directory. If you start Tomcat, your Jenkins installation will be available under

http://localhost:8080/jenkins

If the jenkins.war is deployed in your webapps directory, but cannot be started and the tomcat manager says FAIL – Application at context path /jenkins could not be started, you may need to grant the permissons for JENKINS_HOME.

# mkdir .jenkins
# chown tomcat7:nogroup .jenkins

This makes the .jenkins folder writable and Jenkins can use it.

2.3 Connect to Jenkins for the initial configuration

After installation, open a browser and connect to it. The default port of Jenkins is :8080, therefore on your local machine you find it under the following URL:

http://localhost:8080/

or

http://<<Your-Server-IP>>:8080

First time, it will ask you initial password to setup Jenkins, you have to copy and paste on the required place and press next.

Once password password screen,  you can select to install plug-ins. Select the Install suggested plug-ins to get a typical configuration.

Then create an admin user and press Save and Finish.

3. Configure Jenkins

3.1 Jenkins Home Directory

Jenkins needs some disk space to perform builds and keep archives. One can check this location from the configuration screen of Jenkins. By default, this is set to ~/.jenkins, and this location will initially be stored within your user profile location. In a proper environment, you need to change this location to an adequate location to store all relevant builds and archives. Once can do this in the following ways

• Set “JENKINS_HOME” environment variable to the new home directory before launching the servlet container.
• Set “JENKINS_HOME” system property to the servlet container.
• Set JNDI environment entry “JENKINS_HOME” to the new directory.

The following example will use the first option of setting the “JENKINS_HOME” environment variable.

First create a new folder D:\project\1\Jenkins. Copy all the contents from the existing ~/.jenkins to this new directory.

Set the JENKINS_HOME environment variable to point to the base directory location where Java is installed on your machine. For example,

In the Jenkins dashboard, click Manage Jenkins from the left hand side menu. Then click on ‘Configure System’ from the right hand side.

In the Home directory, you will now see the new directory which has been configured.

3.2 # of executors

This refers to the total number of concurrent job executions that can take place on the Jenkins machine. This can be changed based on requirements. Sometimes the recommendation is to keep this number the same as the number of CPU on the machines for better performance.

3.3 Environment Variables

This is used to add custom environment variables which will apply to all the jobs. These are key-value pairs and can be accessed and used in Builds wherever required.

3.4 Jenkins URL

By default, the Jenkins URL points to localhost. If you have a domain name setup for your machine, set this to the domain name else overwrite localhost with IP of machine. This will help in setting up slaves and while sending out links using the email as you can directly access the Jenkins URL using the environment variable JENKINS_URL which can be accessed as ${JENKINS_URL}.

3.5 Email Notification

In the email Notification area, you can configure the SMTP settings for sending out emails. This is required for Jenkins to connect to the SMTP mail server and send out emails to the recipient list.

4 Secure Jenkins

By default, It is recommended to secure Jenkins. Select Manage Jenkins and then Configure Global Security. Select the Enable security flag. The easiest way is to use Jenkins own user database. Create at least the user “Anonymous” with read access. Also create entries for the users you want to add in the next step.

On the login page, select Create an account to create the users you just gave access.

4.1 Create a new user

Go to Manage Jenkins, Manage and Assign Roles and then Assign Roles to grant the newly created user additional access rights.

Navigate to Manage Roles to define access restrictions in detail. Pattern is a regex value of the job name. The following grants unregistered users read-only access to your build jobs that start with the C-MASTER or M-MASTER prefix and only those.

4.2 Generate ssh key for Jenkins user

If you want to access a private Git repo, for example at Github, you need to generate an ssh key-pair. Create a SSH key with the following command.

# sudo -u jenkins ssh-keygen

The public key must be uploaded to the service you are using, e.g., Github.

5 Jenkins management

5.1 Plug-in management

Jenkins can be extended via additional plug-ins with more functionality. You can configure your plug-ins via the Manage Jenkins ▸ Manager Plugins link.

To install plugins in Jenkins select use the Manage Jenkins ▸ Manager Plugins link and search for the plugin you want to install. Select it from the list and select to install it and restart Jenkins.

The following table is a summary of commonly used plug-ins.

5.2 Uninstalling Plugins

To uninstall a plugin, Go to Manage Jenkins → Manage plugins. Click on the Installed tab. Some of the plugins will have the Uninstall option. You can click these buttons to uninstall the plugins. Ensure to restart your Jenkins instance after the uninstallation.

5.3 Installing another Version of a Plugin

Sometimes it may be required to install an older version of a plugin, in such a case, you can download the plugin from the relevant plugin page on the Jenkins web site. You can then use the Upload option to upload the plugin manually.

6. Server maintenance

6.1 URL Options

The following commands when appended to the Jenkins instance URL will carry out the relevant actions on the Jenkins instance.

http://localhost:8080/jenkins/exit − shutdown jenkins

http://localhost:8080/jenkins/restart − restart jenkins

http://localhost:8080/jenkins/reload − to reload the configuration

You can manually restart Jenkins by adding restart as URL parameter.

7. Setting up a Jenkins job

The build of a project is handled via jobs in Jenkins. Select New Item. Afterwards, enter a name for the job and select Freestyle Job and press OK.

Enter a description for the job and configure how many old jobs should be retained.

Configure how the source code can be retrieved. If you for example using Git, enter the URL to the Git repository. If the repository is not public, you may also need to configure the credentials.

Specify when and how your build should be triggered. The following example polls the Git repository every 1 hours. It triggers a build, if something has changed in the repo.

I typically delete the workspace before a build to avoid any side-effect. In the Build section you can add a build step, e.g., a Docker image build

Press Save to finish the job definition. Press Build Now on the job page to validate the job works as expected.

After a while the job should go to green or blue (depending on your configuration), if successful. Click on the job and afterwards on Console Output to see the log file. Here you can analyze the build errors.

6. Jenkins backup and copying files

Jenkins stores all the settings, logs and build artifacts in its home directory. For example, in /var/lib/jenkins under the default install location of Ubuntu.

To create a backup of your Jenkins setup, just copy this directory.

The jobs directory contains the individual jobs configured in the Jenkins install. You can move a job from one Jenkins installation to another by copying the corresponding job directory. You can also copy a job directory to clone a job or rename the directory.

You only need to copy the config.xml file. If you are using Git in the jobs directory you can use the following . gitignore file to exclude everything except this file.

# Ignore everything in /jobs
*
# Reinclude all folders in /jobs
!*/
# Ignore everything in the subfolders of /jobs
*/*
# Reinclude config.xml files in the first-level subfolders of /jobs
!*/config.xml

Click Reload Configuration from Disk button in the Jenkins web user interface to force Jenkins to reload configuration from the disk.

See the following link for details: https://wiki.jenkins-ci.org/display/JENKINS/Administering+Jenkins

7. Deleting Jenkins log file

Since you can’t delete a file that is currently opened by a program you need to stop all applications accessing the log file. If you are running Ubuntu the applications are Jenkins itself and the rsyslog service.

# service jenkins stop
# rm /var/log/jenkins/jenkins.log
# service rsyslog restart
# service jenkins start

If after restarting your services is not an opening, you can truncate the deleted file.

# rm /var/log/jenkins/jenkins.log
# lsof | grep deleted | grep jenkins.log
daemon  32268  jenkins   5w  REG  8,1 25985776 30210676 /var/log/jenkins/jenkins.log (deleted)
# su root
# cd /proc/32268/fd
# ls -l | grep deleted
lrwx------ 1 jenkins jenkins 64 Aug 17 19:30 3 -> /run/jenkins/jenkins.pid (deleted)
l-wx------ 1 jenkins jenkins 64 Aug 17 19:30 4 -> /var/log/jenkins/jenkins.log (deleted)
l-wx------ 1 jenkins jenkins 64 Aug 17 19:30 5 -> /var/log/jenkins/jenkins.log (deleted)
# > 4
# > 5