Jenkins Command Line Interface

Jenkins contains a built-in command line interface that allows us to access Jenkins from a Script or from our shell. This is convenient for automation of routine tasks. The interface is accessed via the Jenkins CLI client. The Client is a jar file distributed with Jenkins that we can download and use to communicate with the Jenkins Server. My Local Jenkins runs on

http://puppet.jas.com:8080/

In order to download Jenkins-cli.jar we need to use the below url ( where Jenkins is running locally or remotely) like http://puppet.jas.com:8080/jnlpJars/jenkins-cli.jar. But before using the Jenkins cli jar we have some pre requisites that we do to use the Jenkins system on cli. Here are the checks that needs to be done before using the cli,

1. Jenkins service is running.

2.  Enable security option under “Configure Global Security”

Go to jenkins dashboard in Home page ( e.g http://localhost:8080/ ) -> Manage Jenkins

-> Configure Global Security -> Click on “Enable security” checkbox

You can also configure “Access Control” and “Authorization” option in Global Security page.

Jenkins allow us to trigger Jenkins build with any specific user, For that we have to pass username and password in command line.

Once we download the Jnekins jar file we can get the help as,

java -jar jenkins-cli.jar -s http://puppet.jas.com:8080/ help

 add-job-to-view

      Adds jobs to view.

 build

          Builds a job, and optionally waits until its completion.

 cancel-quiet-down

          Cancel the effect of the "quiet-down" command.

****************

Version of the Jenkins – In Order to find the version of the Jenkins running, We can use the command line as,

java -jar jenkins-cli.jar -s http://puppet.jas.com:8080/ version
1.651.3

Login to the Jenkins system – In order to perform some Operations  like Running builds, Installing plugin etc, a user must be logged in. This can done from command line as,

java -jar jenkins-cli.jar -s http://puppet.jas.com:8080 login --username jagadish12 --password jagadesh1982

This will not display any output but we can check whether the user logged in or not by using

java -jar jenkins-cli.jar -s http://puppet.jas.com:8080/ who-am-i
Authenticated as: jagadish12
Authorities:
  authenticated

The above output tells us that the User is logged in to the Jenkins system.

Dealing with Jobs – Build Jobs are the most important one that we use in Jenkins. Many of the times we automate things like building jobs

Available Jobs - In order to find out the available jobs we can use the list-jobs as,

java -jar jenkins-cli.jar -s http://puppet.jas.com:8080/ list-jobs

jenkinsnexustesting

Mint

The above command will display the jobs currently available in Jenkins System.

Copy Job – In order to copy a Job from an existing one we can use the below command as,

java -jar jenkins-cli.jar -s  http://puppet.jas.com:8080/ copy-job jenkinsnexustesting sampleTest

This will create a job named sampleTest from existing job jenkinsnexustesting

Now run the list-jobs to see if the job has created successfully as,

java -jar jenkins-cli.jar -s http://puppet.jas.com:8080/ list-jobs

jenkinsnexustesting

Mint

sampleTest

We can see that the new job is created successfully.

Build Job – In order to build a job we can use the build command line as ,

java -jar jenkins-cli.jar -s  http://puppet.jas.com:8080/  build sampleTest

This will starting running the job sampleTest.

Build Job with output - In order to check the Output of the Build job we can use the command as below,

java -jar jenkins-cli.jar -s  http://puppet.jas.com:8080/  console sampleTest

Started from command line by ha:AAAAnB+LCAAAAAAAAP9b85aBtbiIQTGjNKU4P08vOT+vOD8nVc83PyU1x6OyILUoJzMv2y+/JJUBAhiZGBgqihhk0NSjKDWzXb3RdlLBUSYGJk8GtpzUvPSSDB8G5tKinBIGIZ+sxLJE/ZzEvHT94JKizLx0a6BxUmjGOUNodHsLgAyuEgYB/dLi1CL9rMT0xJTM4gxDIwCWUQEexQAAAA==jagadish12

Building on master in workspace /var/lib/jenkins/jobs/sampleTest/workspace

Cloning the remote Git repository

**********************************

 [INFO] Scanning for projects...

[INFO]                                                                              

[INFO] ------------------------------------------------------------------------

[INFO] Building javaee7-simple-sample 1.11-SNAPSHOT

[INFO] ------------------------------------------------------------------------

[INFO]

ha:AAAAYB+LCAAAAAAAAP9b85aBtbiIQSmjNKU4P0+vJLE4u1gvPjexLDVPzxdEGvvmZ+X75ZekLlOVfvTjc8FPJgaGiiIGKaiG5Py84vycVD1nCA1SyAABjCCFBQCV27OjYAAAAA==[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ javaee7-simple-sample ---

Started calculate disk usage of build

Finished Calculation of disk usage of build in 0 seconds

Started calculate disk usage of workspace

Finished Calculation of disk usage of workspace in 0 seconds

[INFO] ------------------------------------------------------------------------

[INFO] BUILD SUCCESS

[INFO] ------------------------------------------------------------------------

[INFO] Total time: 1.386 s

[INFO] Finished at: 2016-10-21T01:02:13-04:00

[INFO] Final Memory: 9M/155M

[INFO] ------------------------------------------------------------------------

Get Job Configuration – Some times there may be cases where we need to check the Configuration settings for the Job as,

java -jar jenkins-cli.jar -s  http://puppet.jas.com:8080/  get-job sampleTest > config.xml

[puppet@root$:~/Downloads/jenkins]$  ll

total 816

-rw-r--r-- 1 root root   2428 Oct 21 05:04 config.xml

The Configuration is saved as a XML file with a name (config.xml) that we pass as argument.

We can also run the get-job command to simply display the configuration details on the console as

java -jar jenkins-cli.jar -s http://puppet.jas.com:8080/ get-job jenkinsnexustesting

Restore a Job – In order to Restore a saved job to a job named test3 from an XML file named config.xml we can use the command

java -jar jenkins-cli.jar -s  http://puppet.jas.com:8080/   create-job test3 < config.xml  

Disable a Job – In order to disable a job we can use

java -jar jenkins-cli.jar -s http://puppet.jas.com:8080/   disable-job test3

Script Execution -  Jenkins cli not just allows us to run command provided by cli. It also allows us to execute scripts. The script needs to be written using the groovy language and the invocation is done using groovy command line.

Below is a script that will display the last successfully build job – succ.gsh

// Get the list of jobs that were SUCCESSFUL

// -- Get Active Jobs with 'Buildable' state

activeJobs = hudson.model.Hudson.instance.items.findAll{job -> job.isBuildable()}

// -- Get a subset of buildable jobs with result status of 'SUCCESSFUL'

goodRuns = activeJobs.findAll{job -> job.lastBuild != null && job.lastBuild.result == hudson.model.Result.SUCCESS}

// Do something with them - e.g. listing them

goodRuns.each{run -> println "Successful Job Name: ${run.name}"}

This can be run as,

java -jar jenkins-cli.jar -s http://172.16.202.95:8080 groovy succ.gsh

Successful Job Name: jenkinsnexustesting

We can see that the last successfully job is jenkinsnexustesting

Below is a another sample script which will create Nodes - create_node.gsh

import jenkins.model.*

import hudson.model.*

import hudson.slaves.*

Jenkins.instance.addNode(new DumbSlave("test-script","test slave description","/tmp","1",Node.Mode.NORMAL,"test-slave-label",new JNLPLauncher(),new RetentionStrategy.Always(),new LinkedList()))

Now In order to run the script we can run the command line as ,

java -jar jenkins-cli.jar -s http://localhost:8080 groovy create_node.gsh

Creating Slave Node - In order to create Slave nodes we can use

cat <<EOF | java -jar jenkins-cli.jar -s http://puppet.jas.com:8080/ create-node $2

> <slave>

>   <name>MasterNode</name>

>   <description></description>

>   <remoteFS>/tmp</remoteFS>

>   <numExecutors>1</numExecutors>

>   <mode>NORMAL</mode>

>   <retentionStrategy class="hudson.slaves.RetentionStrategy$Always"/>

>   <launcher class="hudson.plugins.sshslaves.SSHLauncher" plugin="ssh-slaves@1.5">

>       </launcher>

>   <label>mastertesting</label>

>   <nodeProperties/>

>   <userId>jagadish12</userId>

>  

> </slave>

> EOF

In the above command line we are using the underlying shell with Jenkins cli as above. We passed the Slave node configuration as an argument to the shell cat command. Now to

check whether our Node has been created or not we can use the Jenkins GUI or we can get the configuration of the created node as,

java -jar jenkins-cli.jar -s  http://172.16.202.95:8080/ -noKeyAuth get-node MasterNode

<?xml version="1.0" encoding="UTF-8"?>

<slave>

  <name>MasterNode</name>

  <description></description>

  <remoteFS>/tmp</remoteFS>

  <numExecutors>1</numExecutors>

  <mode>NORMAL</mode>

  <launcher class="hudson.plugins.sshslaves.SSHLauncher" plugin="ssh-slaves@1.9">

          <port>0</port>

  </launcher>

  <label>mastertesting</label>

  <nodeProperties/>

  <userId>jagadish12</userId>

</slave>

Installing a plugin – To install a plugin we can use the command line as,

java -jar jenkins-cli.jar -s  http://puppet.jas.com:8080/ install-plugin 
http://updates.jenkins-ci.org/latest/build-monitor-plugin.hpi  -restart

Hope this helps you in using Jenkins CLI. More articles to come. Happy learning J

Read More

Integrating Jenkins With Sonar Qube

SonarQube (formerly Sonar) is an open source platform for continuous inspection of code quality. In this article we will see how we can integrate Sonar Qube in Jenkins. We will also build sample application which will use the integrated sonar Qube to test the code.

1. Sonar Qube Installation

a. Download the Sonar Qube with version 5.6.3

b. Unzip the sonar Qube

c. Go to <Sonar Qube>/bin/<OS-version>/

d. Run “sh sonar.sh console” command

e. Once the server is started, access the server from http://localhost:9000/ ( if you have a name to your system use that). This will be used in configuring the sonar qube servers in jenkins

2. Sonar Runner Installation

a. Download the Sonar-runner-2.4

b. Unzip the sonar runner

c. Don’t run the sonar-runner. This will be started from Jenkins.

Sonar Qube vs Sonar runner

SonarQube (formerly just "Sonar") is a server-based system. Of course you can install it on your local machine (the hardware requirements are minimal). But it is a central server with a database.

Analyses are performed by some Sonar "client" software, which could be the sonar runner, the sonar ant task, the sonar Eclipse plugin etc. The analysis results can be automatically uploaded to the server, where they can be accessed via the sonar Web application.

Once the Sonar Qube and Sonar runner are installed. We then need to install Sonar Plugin in the Jnekins

3. Download Jenkins Plugin

a. Download the Sonar Qube plugin 2.4.4 from Manage Jenkins => Manage Plugins

4. Configure the Jenkins – Sonar Qube Server

a. Configure System =>

    Under the Sonar Qube Server as,

Enter the Name.

Server URL – the one that we tried to access above when we started the Sonar qube

Save

Note – Make sure to check the “Enable injection of sonarqube server configuration as build environment variables

5. Configure the Jenkins – Sonar Qube Scanner

 Under the Sonar Qube Scanner,

 Choose a new Sonar Qube Scanner Installation as,

Add the name and in the SONAR_RUNNER_HOME and pass the location of the unzip sonar-runner path above.

NOTE – also create a environment variable with SONAR_RUNNER_HOME variable with the location of the sonar-runner path

Save and Apply

6. Configure the Jenkins – Sonar Qube Project Properties file.

Every project that we want to check needs a sonar-project.properties file. The file can be a simple one by just passing the src location path. Here is a sample of the that I have

cat sonar-project.properties
# must be unique in a given SonarQube instance
sonar.projectKey=my:project
# this is the name and version displayed in the SonarQube UI. Was mandatory prior to SonarQube 6.1.
sonar.projectName=javaee7-simple-sample
sonar.projectVersion=1.0

sonar.sources=.

# Encoding of the source code. Default is default system encoding
#sonar.sourceEncoding=UTF-8

Save the file along with your source directory.

 7. Configure the Jenkins – Job

 Once the file is created, create a Job with the sonar qube as,

Check the “prepare SonarQube Scanner Environment”

In the Add pre-build Step , Choose the “Execute SonarQube Scanner”.

A new Item will be added and provide the path of the project’s “sonar-project.properties” file path as above.

Save and Run the Build. We can see the project Analysis at the sonar Qube Dashboard at http://localhost:9000/

That is all you need to do when configuring a Sonar Qube with Jenkins. More to Come. Happy Learning J

Read More

Nexus Rest API

Nexus is a collection of services available for us to automate. With the services available we can integrate them in the work flow that we need. Nexus also exposes the services as Rest services so that we can use them to automate things. In this article we will see how we can use the nexus rest api to automate things.

While nexus has a rich set of Rest end points for every thing from core actions like storing artifacts in the repository to creating users and repositories, but the documentation is little touch to find. Here are the steps you need to take to access this documentation:

1. Login as an Administrator
2. In the Administration section of the left-hand menu, click on Plugin Console to open the Plugin Console.
3. Once in the Plugin Console you will see a list of Nexus Plugins. Click on a plugin to view the APIs it provides.
4. Once you select a plugin, you should see a list of APIs. For example, the video below shows the Core API, click on the link in the Plugin Console to view the REST API documentation.

We will use the CURL command available in Linux to make the rest calls.  Nexus uses a single end point with changing services to communicate on the rest.

/nexus/service/local is the endpoint and by adding the service after this we can make a rest call to automate different things. Below are the few nexus services that we can use to communicate

nexus_status_service: status
nexus_users_service: users
nexus_repositories_service: repositories
nexus_roles_service: roles

Lets see a few examples on making use of the nexus rest API

Simple Get calls

Retrieve the status of the Nexus Server -

curl http://localhost:8081/nexus/service/local/status

Retrieve the available nexus repositories

curl http://localhost:8081/nexus/service/local/repositories

Authenticated REST - GET requests  

User Details– In order to retrieve some details we need to send the credentials details. Curl Command also allows us to pass the credentials. In order to retrieve a user details the Rest call needs the credentials and this can be done as,

curl -X GET -u admin:admin123 http://localhost:8081/nexus/service/local/users

Delete Repositories

We can also pass other HTTP parameter like DELETE using the curl command and make a Rest call as,

curl -X DELETE -v -u admin:admin123 http://localhost:8081/nexus/service/local/repositories/my-releases-xml

Post Calls – The most important call will be the Post call since we will be using this to create things. Lets see on creating a repository using the CURL post call along with the XML payload

Create a XML file new_user.xml with user details as,

<?xml version="1.0" encoding="UTF-8"?>

<user-request>

  <data>

    <userId>master</userId>

    <email>master@gmail.com</email>

    <status>active</status>

    <firstName>mind</firstName>

    <resourceURI>http://puppet.jas.com:8081/nexus/service/local/roles/repository-any-full</resourceURI>

    <roles>

      <role>npm-all-view</role>

    </roles>

    <lastName>minder</lastName>

    <password>test123</password>

  </data>

</user-request>

In the above snippet we are creating a new user master. In order to make the call we can call the CURL command as,

curl -i -H "Accept: application/xml" -H "Content-Type: application/xml; charset=UTF-8"  -v -d "@new_user.xml" -u admin:admin123 http://puppet.jas.com:8081/nexus/service/local/users

In the above command we are using the users service to make the rest call and create a user by passing the xml payload that we created in the file. Once we ran the curl command we can see the output as below which includes the necessary return code details.

* About to connect() to puppet.jas.com port 8081 (#0)

*   Trying 172.16.202.95...

* Connected to puppet.jas.com (172.16.202.95) port 8081 (#0)

* Server auth using Basic with user 'admin'

> POST /nexus/service/local/users HTTP/1.1

> Authorization: Basic YWRtaW46YWRtaW4xMjM=

> User-Agent: curl/7.29.0

> Host: puppet.jas.com:8081

> Accept: application/xml

> Content-Type: application/xml; charset=UTF-8

> Content-Length: 424

>

* upload completely sent off: 424 out of 424 bytes

< HTTP/1.1 201 Created

HTTP/1.1 201 Created

< Date: Wed, 31 Aug 2016 08:27:23 GMT

Date: Wed, 31 Aug 2016 08:27:23 GMT

< X-Frame-Options: SAMEORIGIN

X-Frame-Options: SAMEORIGIN

< X-Content-Type-Options: nosniff

X-Content-Type-Options: nosniff

< Server: Nexus/2.13.0-01 Noelios-Restlet-Engine/1.1.6-SONATYPE-5348-V8

Server: Nexus/2.13.0-01 Noelios-Restlet-Engine/1.1.6-SONATYPE-5348-V8

< Content-Type: application/xml; charset=UTF-8

Content-Type: application/xml; charset=UTF-8

< Content-Length: 388

Content-Length: 388

<

<user-response>

  <data>

    <resourceURI>http://puppet.jas.com:8081/nexus/service/local/users/master</resourceURI>

    <userId>master</userId>

    <password>test123</password>

    <firstName>mind</firstName>

    <lastName>minder</lastName>

    <status>active</status>

    <email>master@gmail.com</email>

    <roles>

      <role>npm-all-view</role>

    </roles>

  </data>

* Connection #0 to host puppet.jas.com left intact

</user-response>

Besides passing the XML file as payload , nexus also accepts the JSON content as payload even and CURL command allows us to do so. Create a file repo.json with the repository details as,

[puppet@root$:/test]$  cat repo.json

{

    "data": {

        "repoType": "proxy",

        "id": "somerepo",

        "name": "Some Repo Name",

        "browseable": true,

        "indexable": true,

        "notFoundCacheTTL": 1440,

        "artifactMaxAge": -1,

        "metadataMaxAge": 1440,

        "itemMaxAge": 1440,

        "repoPolicy": "RELEASE",

        "provider": "maven2",

        "providerRole": "org.sonatype.nexus.proxy.repository.Repository",

        "downloadRemoteIndexes": true,

        "autoBlockActive": true,

        "fileTypeValidation": true,

        "exposed": true,

        "checksumPolicy": "WARN",

        "remoteStorage": {

            "remoteStorageUrl": "http://puppet.jas.com:8081/local",

            "authentication": null,

            "connectionSettings": null

        }

    }

}

Now we can run the CURL command as,

curl -H "Content-Type: application/json" -X POST -d @virtual.json -u admin:admin123 http://puppet.jas.com:8081/nexus/service/local/repositories

In the above command we have changed the content-type to json unlike xml.

That’s all about using the nexus Rest api and curl command.  Below are the few examples on using nexus Rest api

Get Single Repository Details

curl -k http://puppet.jas.com:8081/nexus/service/local/repositories/releases -u

Get Repository Status

curl -k http://puppet.jas.com:8081/nexus/service/local/repository_statuses -u admin:admin123

Getting a list of all users, now with authenticating as admin user

curl -X GET -u admin:admin123 http://localhost:8081/nexus/service/local/users

Retive the account Details

curl -k http://puppet.jas.com:8081/nexus/service/local/user_account/admin -u admin:admin123

Get the list of users available

curl -X GET -u admin:admin123 http://172.16.202.96:9091/nexus/service/local/users

Get the list of repositories available

curl http://172.16.202.96:9091/nexus/service/local/repositories

Hope this Helps.

Read More

Ansible Rest Calls

Rest API’s calls are now a days the main API to integrate with different applications. If you want to connect to an existing infrastructure we can expose the infrastructure as rest API and other applications can you the API to make calls.

Ansible also provides us a way to make the Rest calls using URI module. The URI module allows us to send XML or JSON payload and get the necessary details. In this article we will see how we can use the URI module and make the Rest calls. As for the article I will be using the Nexus artifactory to connect which run on the 8081 Port. The URL are specified in the vars/main.yml file.

Make a Get Call - Using the Ansible URI module to make get call is pretty easy. We can use

- name: Maker a Call

  uri:

   url: "{{nexus_url}}/repositories"

   method: GET

   user: admin

   password: admin123

   force_basic_auth: yes

   return_content: yes

In the above snippet, I am making a Rest call to the “nexus_url/respositories” URL by passing them the user name and password along with the method as “Get” and the return_content.  We can use the Ansible debug to check the obtained response.

Make a Post call – Making a Post call needs a little extra details. Now for making a post call we need the JSON data. We can both create a file with JSON data and pass that to the file or we can include the JSON data at the same place. Lets create a JSON file first as

{

    "data": {

        "repoType": "proxy",

        "id": "somerepo1",

        "name": "Some Repo Name1",

        "browseable": true,

        "indexable": true,

        "notFoundCacheTTL": 1440,

        "artifactMaxAge": -1,

        "metadataMaxAge": 1440,

        "itemMaxAge": 1440,

        "repoPolicy": "RELEASE",

        "provider": "maven2",

        "providerRole": "org.sonatype.nexus.proxy.repository.Repository",

        "downloadRemoteIndexes": true,

        "autoBlockActive": true,

        "fileTypeValidation": true,

        "exposed": true,

        "checksumPolicy": "WARN",

        "remoteStorage": {

            "remoteStorageUrl": "http://puppet.jas.com:8081/local",

            "authentication": null,

            "connectionSettings": null

        }

    }

}

Save the file as proxy.json and move the file to the files location. Now we can use the proxy.json using the lookup in ansible as,

- name: Make a Proxy Repository

  uri:

    url: "{{ nexus_url }}/repositories"

    method: POST

    body: "{{ lookup('file','proxy.json') }}"

    user: admin

    password: admin123

    force_basic_auth: yes

    body_format: json

    HEADER_Content-Type: application/json

    HEADER_Accept: application/json,version=2

    return_content: yes

    status_code: 201

IN the above snippet I passed the JSON file an argument to the  Ansible lookup. Besides this I passed the user name , password and most importantly the body_format , Header fields etc. We also need to pass the return code of the Rest call so that Ansible will compare the return status code with the status code that we defined. In the above example I passed the status code as 201. In this case the Ansible will check the rest call status code and passed status code to check if that is success or not. Even though the rest call is success if the status codes are not same Ansible will throw an error.

If we want to pass the JSON format in the body itself without using the file, we can use

- name: Make a Hosted Repository
  uri:
    url: "{{nexus_url}}/{{ nexus_endpoint }}/{{ nexus_repositories_service }}"
    method: POST
    body: >
     {
      data: {
        "repoType": "{{ hosted_repo_data.repoType }}",
        "id": "{{ repoId }}",
        "name": "{{ repoName }}",
        "provider": "{{ repoProvider}}",
        "browseable": {{ hosted_repo_data.browseable }},
        "repoPolicy": {{ hosted_repo_data.repoPolicy }},
        "providerRole": {{ hosted_repo_data.providerRole }}
           }
     }
    user: "{{ nexus_admin }}"
    password: "{{ nexus_admin_password }}"
    force_basic_auth: yes
    body_format: json
    HEADER_Content-Type: application/json
    HEADER_Accept: application/json,version=2
    return_content: yes
    status_code: 201

The body can be added directory to the Body method in the Ansible JSON call.

That’s all about Ansible URL module and rest calls. Hope this helps. 

Read More