Jenkins – SMTP setup

There is always a need for getting the Status of the nightly Build that we configure. Jenkins does provide a Email Notification option which allows us to configure Email Notifications that can be used to get the status of the Build. In this article we will see how we can configure the SMTP Configuration in Jenkins and get a mail for the Failed Build.

1. Configure the SMTP Setup

Configure System -> Email Notification ( Enter the Values accordingly )

SMTP Server – smtp.gmail.com (Click Advanced)

Check “Use SMTP Authentication”

User Name –Mail ID you want to Send mail from

Password – Enter the Password for the above configured Mail ID

SMTP Port – 587

Reply-To- Address – Same above mail

Charset – UTF-8

Check the Test Configuration by sending a test E-mail, it will send a test email.

Note – At this moment there may be case where we see some errors saying

a) Must Issue a STARTLS Command first.

For this Error, please add the startls command to the JAVA Command line that start Jenkins. For me the Jenkins file is located at “/etc/sysconfig/Jenkins”

Add the “-Dmail.smtp.starttls.enable='true'” to the existing JENKINS_JAVA_OPTSIONS variable as

JENKINS_JAVA_OPTIONS="-Djava.awt.headless=true -Dmail.smtp.starttls.enable='true'"

b) javax.mail.AuthenticationFailedException: 534-5.7.14 <https://accounts.google.com/signin/continue?sarp=1&scc=1&plt=AKgnsbu9

If the above error, comes login to you Gmail account that you want to send mail and browse the below link “https://www.google.com/settings/security/lesssecureapps“

Click the “Turn ON” Check box and error will go off.

This feature is disabled to block less secure apps from accessing Google Apps accounts

2) Once the SMTP configuration is success, we now need to configure email settings for the Build Job. Click the build job and in the Build Setting , check the Email Notification.

Enter the Recipients main in the Recipients Box and Check the “Send e-mail for every unstable build”

This will make sure that for every failed build a mail alert is sent to the people in the Recipients list.

That’s all you need to do for configuring the Mail alerts in Jenkins. More to Come. Happy Coding.

Read More

Jenkins Slave – Remote Host With SSH

In the previous article we have seen how we can configure a Jenkins slave on a remote machine. In this article we will see how we can configure Slave on the remote machine using SSH configuration.

1. On the Remote machine (Ex- 172.16.202.95) create a SSH Key. Use the Same Jenkins User as we configured in the last article. Login as Jenkins and run the ssh key generation commands as,

[jenkins@devm ~]$ ssh-keygen -t rsa -C "this is access to Jenkins Slave"

Generating public/private rsa key pair.

Enter file in which to save the key (/home/jenkins/.ssh/id_rsa):

Created directory '/home/jenkins/.ssh'.

Enter passphrase (empty for no passphrase):

Enter same passphrase again:

Your identification has been saved in /home/jenkins/.ssh/id_rsa.

Your public key has been saved in /home/jenkins/.ssh/id_rsa.pub.

The key fingerprint is:

af:e3:74:ae:24:96:53:0a:c6:fd:43:29:e7:e6:47:f3 this is access to Jenkins Slave

The key's randomart image is:

+--[ RSA 2048]----+

|                 |

|                 |

|                 |

|   . .   .       |

|    + o S        |

|   . . X .o      |

|      * B.oo     |

|     . B.=. E    |

|       .=+.      |

+-----------------+

Save the Keys in the default location. Once keys are created we can see the public and private keys in the /home/Jenkins/.ssh location. Copy the Public key ( id_rsa.pub ) to authorized_keys file.  

[jenkins@devm ~]$ cd /home/jenkins/

[jenkins@devm ~]$ cd .ssh/

[jenkins@devm .ssh]$ ll

total 8

-rw-------. 1 jenkins jenkins 1675 Aug 25 09:50 id_rsa

-rw-r--r--. 1 jenkins jenkins  413 Aug 25 09:50 id_rsa.pub

[jenkins@devm .ssh]$ cat id_rsa.pub >> authorized_keys

Make Sure the file permissions on the authorized_keys file is 0644.

2.  Once the key generation is done, we need to copy the private key to the Jenkins so that Jenkins can communicate with the host.

On the Jenkins Side -> Credentials -> System -> Global Credentials -> Add Credentials

ON the Right Hand ,

Select Kind -> “SSH Username and Private Key”

Scope -> Global

User name -> Jenkins

Private Key -> Enter Directly ( This will give you a text box at bottom , enter the Contents that are copied from id_rsa file in the remote machine that we created the keys for)

Save

By this we have configured the SSH Private Key to the Credentials.

3. Create a Slave but Choose different options in the Launch method

Launch method -> Launch Slave agents on Unix machine via SSH. This gives you a Host and Credentials elements under this

Enter the Host Name (Ex- 172.16.202.95)

Select the newly Created user name Jenkins that we created above from the Credentials List Box.

Once saved, the salve will be automatically stared. Make sure JAVA is configured and is available for the Jenkins to start or else we will see Java Not found exceptions

More To Come, Happy learning J

Read More

Jenkins Slave – Remote Host

As we already know that Jenkins besides acting as a continuous integration tool allows other configuration options including master – slave configuration. We already discussed master-slave configuration (Check Here for details). In this article we will see how we can configure Slave on a Remote Host using the Username and password.

1. Create a user name “Jenkins” on the remote machine with Password on the remote Machine (Ex- 172.16.202.95)

[root@devm local]# adduser jenkins

[root@devm local]# passwd jenkins

Changing password for user jenkins.

New password:

BAD PASSWORD: The password is shorter than 8 characters

Retype new password:

passwd: all authentication tokens updated successfully.

2. Create a home directory for the Jenkins Slave

[root@devm local]# cd /home/jenkins/

[root@devm jenkins]# mkdir jenkins-slave

3. Create a New Node in the Jenkins Master server with the below configuration,

a. Enter the Name of the Slave “JenkinsSlave1”

b. In the Remote root Directory – (enter the location that we created in the remote machine) /home/Jenkins/Jenkins-salve

c. In the Launch Method – Launch slave agents on Unix Machine via SSH

This gives Host and Credentials boxes under this where we need to enter the Host name and Select the User name and password that are added in the global credentials

The credentials are same as the one that we created on the remote machine above. Once saved, the Slave will automatically gets started and we can see the slave running on the remote machine. Make sure JAVA is configured and is available for the Jenkins to start or else we will see Java Not found exceptions.

In the next article we will see how we can configure a Jenkins Slave on a remote Host using SSH keys

More to Come, happy learning J

Read More

Sonar Integration with maven

In the next series of sonar, we will see how we can integrate sonar with maven and use maven command in analyzing the source code of a project and display the results on the sonarqube console.

1. Make sure JDK,Maven and Sonar are available ( with sonar running ).

2. Make the changes to the global maven setting.xml file available at location ~/.m2

3. Add the pluginGroups as,

    <pluginGroups>

        <pluginGroup>org.sonarsource.scanner.maven</pluginGroup>

    </pluginGroups>

4. Add the Profile pointing the location of the sonar console as,

   <profile>

        <id>sonar</id>

         <activation>

              <activeByDefault>true</activeByDefault>

          </activation>

          <properties>

              <sonar.host.url>http://puppet.jas.com:9000</sonar.host.url>

          </properties>

    </profile>

4. Once the changes are done, Move to you java application and run,

[puppet@root$:/work/testing/javaee7-simple-sample]$ mvn clean verify sonar:sonar

[INFO] Scanning for projects...

[INFO]                                                                        

[INFO] ------------------------------------------------------------------------

[INFO] Building javaee7-simple-sample 1.11-SNAPSHOT

[INFO] ------------------------------------------------------------------------

[INFO]

[INFO] --- maven-war-plugin:2.3:war (default-war) @ javaee7-simple-sample ---

[INFO] Packaging webapp

[INFO] Assembling webapp [javaee7-simple-sample] in [/work/testing/javaee7-simple-sample/target/javaee7-simple-sample]

[INFO] Processing war project

[INFO] Copying webapp resources [/work/testing/javaee7-simple-sample/src/main/webapp]

[INFO] Webapp assembled in [63 msecs]

[INFO] Building war: /work/testing/javaee7-simple-sample/target/javaee7-simple-sample.war

[INFO]                                                                        

[INFO] ------------------------------------------------------------------------

[INFO] Building javaee7-simple-sample 1.11-SNAPSHOT

[INFO] ------------------------------------------------------------------------

[INFO]

[INFO] --- sonar-maven-plugin:3.0.2:sonar (default-cli) @ javaee7-simple-sample ---

 [INFO] User cache: /root/.sonar/cache

[INFO] SonarQube version: 4.5.7

[INFO] Default locale: "en_US", source code encoding: "UTF-8"

[INFO] [02:21:17.060] Load global referentials...

[INFO] [02:21:17.393] Load global referentials done: 334 ms

[INFO] [02:21:17.411] User cache: /root/.sonar/cache

[INFO] [02:21:17.420] Install plugins

[INFO] [02:21:17.739] Install JDBC driver

[INFO] [02:21:17.743] Create JDBC datasource for jdbc:h2:tcp://localhost/sonar

[INFO] [02:21:18.596] Initializing Hibernate

[INFO] [02:21:19.930] Load project referentials...

[INFO] [02:21:20.315] Load project referentials done: 385 ms

[INFO] [02:21:20.315] Load project settings

[INFO] [02:21:21.131] Loading technical debt model...

[INFO] [02:21:21.151] Loading technical debt model done: 20 ms

[INFO] [02:21:21.153] Apply project exclusions

[INFO] [02:21:21.395] -------------  Scan javaee7-simple-sample

[INFO] [02:21:21.398] Load module settings

[INFO] [02:21:21.720] Loading rules...

[INFO] [02:21:22.089] Loading rules done: 369 ms

[INFO] [02:21:22.113] Configure Maven plugins

[INFO] [02:21:22.170] No quality gate is configured.

[INFO] [02:21:22.329] Initializer FindbugsMavenInitializer...

[INFO] [02:21:22.330] Initializer FindbugsMavenInitializer done: 1 ms

[INFO] [02:21:22.330] Base dir: /work/testing/javaee7-simple-sample

[INFO] [02:21:22.330] Working dir: /work/testing/javaee7-simple-sample/target/sonar

[INFO] [02:21:22.330] Source paths: src/main/webapp, pom.xml, src/main/java

[INFO] [02:21:22.330] Binary dirs: target/classes

[INFO] [02:21:22.331] Source encoding: UTF-8, default locale: en_US

[INFO] [02:21:22.331] Index files

[INFO] [02:21:22.363] 4 files indexed

[INFO] [02:21:22.414] Quality profile for java: Sonar way

[INFO] [02:21:22.427] Sensor JavaSquidSensor...

[INFO] [02:21:22.600] Java Main Files AST scan...

[INFO] [02:21:22.603] 4 source files to be analyzed

[INFO] [02:21:22.765] 4/4 source files analyzed

[INFO] [02:21:22.767] Java Main Files AST scan done: 167 ms

[INFO] [02:21:22.768] Java bytecode scan...

[INFO] [02:21:22.790] Java bytecode scan done: 22 ms

[INFO] [02:21:22.790] Java Test Files AST scan...

[INFO] [02:21:22.790] 0 source files to be analyzed

[INFO] [02:21:22.791] Java Test Files AST scan done: 1 ms

[INFO] [02:21:22.791] 0/0 source files analyzed

[INFO] [02:21:22.795] Package design analysis...

[INFO] [02:21:22.799] Package design analysis done: 4 ms

[INFO] [02:21:22.822] Sensor JavaSquidSensor done: 395 ms

[INFO] [02:21:22.822] Sensor QProfileSensor...

[INFO] [02:21:22.826] Sensor QProfileSensor done: 4 ms

[INFO] [02:21:22.826] Sensor InitialOpenIssuesSensor...

[INFO] [02:21:22.863] Sensor InitialOpenIssuesSensor done: 37 ms

[INFO] [02:21:22.863] Sensor ProjectLinksSensor...

[INFO] [02:21:22.866] Sensor ProjectLinksSensor done: 3 ms

[INFO] [02:21:22.867] Sensor VersionEventsSensor...

[INFO] [02:21:22.872] Sensor VersionEventsSensor done: 4 ms

[INFO] [02:21:22.872] Sensor FileHashSensor...

[INFO] [02:21:22.873] Sensor FileHashSensor done: 1 ms

[INFO] [02:21:22.873] Sensor SurefireSensor...

[INFO] [02:21:22.874] parsing /work/testing/javaee7-simple-sample/target/surefire-reports

[WARN] [02:21:22.874] Reports path not found: /work/testing/javaee7-simple-sample/target/surefire-reports

[INFO] [02:21:22.874] Sensor SurefireSensor done: 1 ms

[INFO] [02:21:22.874] Sensor Maven dependencies...

[INFO] [02:21:22.897] Sensor Maven dependencies done: 23 ms

[INFO] [02:21:22.897] Sensor CPD Sensor (wrapped)...

[INFO] [02:21:22.897] JavaCpdEngine is used for java

[INFO] [02:21:22.898] Cross-project analysis disabled

[INFO] [02:21:22.917] Sensor CPD Sensor (wrapped) done: 20 ms

[INFO] [02:21:23.018] Execute decorators...

[INFO] [02:21:23.121] Store results in database

[INFO] [02:21:23.208] ANALYSIS SUCCESSFUL, you can browse http://puppet.jas.com:9000/dashboard/index/org.javaee7.sample:javaee7-simple-sample

[INFO] [02:21:23.255] Executing post-job class org.sonar.plugins.core.issue.notification.SendIssueNotificationsPostJob

[INFO] [02:21:23.255] Executing post-job class org.sonar.plugins.core.batch.IndexProjectPostJob

[INFO] [02:21:23.272] Executing post-job class org.sonar.plugins.dbcleaner.ProjectPurgePostJob

[INFO] [02:21:23.283] -> Keep one snapshot per day between 2016-07-27 and 2016-08-23

[INFO] [02:21:23.284] -> Keep one snapshot per week between 2015-08-26 and 2016-07-27

[INFO] [02:21:23.284] -> Keep one snapshot per month between 2011-08-31 and 2015-08-26

[INFO] [02:21:23.284] -> Delete data prior to: 2011-08-31

[INFO] [02:21:23.288] -> Clean javaee7-simple-sample [id=1]

[INFO] ------------------------------------------------------------------------

[INFO] BUILD SUCCESS

[INFO] ------------------------------------------------------------------------

[INFO] Total time: 38.600 s

[INFO] Finished at: 2016-08-24T02:21:23-04:00

[INFO] Final Memory: 30M/319M

[INFO] ------------------------------------------------------------------------

Once the Build is success we can see the results in the console as,

More to Come, Happy learning J

Read More

SonarQube - Static Code Analysis

Testing is one of the important aspects of code to identify various issues that can occur while running the code. At the same time it is necessary to examine the code quality while moving the code to production.

Static analysis also called as static code analysis is a method of debugging that is done by examining the code without executing that. This allows developers a better understanding of the code structure and can help developers to ensure that the code adheres to industry standards. The main advantage of the static analysis is that it reveals errors that do not occur until months and years of application running. It is said that the static analysis is only a first step in a comprehensive software quality-control regime. Sonar is one such tool which provides us the static code analysis.

Sonar is an open source web-based application to manage code quality which covers seven axes of code quality as: Architecture and design, comments, duplications, unit tests, complexity, potential bugs and coding rules. Developed in Java and can cover projects in Java, Flex, PHP, PL/SQL,  Visual Basic 6. It's very efficient to navigate; offering visual reporting and you can follow metrics evolution of your project and combine them.

In this article we will see how we can install SONAR tool and use that.

1. Download SonarQube from here

2. Extract the tar file to /op/sonarqube.

Once extracted move to the /opt/sonarqube/bin/linux-x86-64.And run the “sonar.sh  start”.

That’s all you need to do in starting the sonarqube. Access the sonarqube console using the “localhost:9000” and we can see the web console as below,

 The default credentials for login are admin and admin. More to come using this tool. Stay learning J

Read More

PSSH – Parallel SSH

Many times we use SSH to login to the remote machines, Copy files, perform administrative tasks. But in all the cases we do these operations machine by machine i.e. we log to one machine perform the operations and then log out to go to the next machine. If we want to increase the productivity with SSH, we can use Tools that allow commands to be executed on multiple remote machines simultaneously. Parallel SSH is one such tool which allows running commands on multiple servers at the same time.

This article will give you a brief introduction about the Parallel SSH tool.

1. Install Parallel SSH

Parallel SSH tool is based on Python. PSSH is supported on Python 2.4 or more.In order to install use the PIP tool provided by Python. Install the parallel tool using

Pip install pssh

[puppet@root$:~/.pip]$  pip install pssh

Collecting pssh

  Downloading pssh-2.3.1.tar.gz

Building wheels for collected packages: pssh

  Running setup.py bdist_wheel for pssh ... done

Stored in directory: /root/.cache/pip/wheels/b6/98/92/eab367fee5ded0129e0d91feb3207e76fcb282ccc87507a6d0

Successfully built pssh

Installing collected packages: pssh

Successfully installed pssh-2.3.1

A version of pssh with 2.3.1 version is installed.

Note – pip is a tool by Python to install python packages. In order to install pip use the command “yum install python-pip”. In most linux version python will be already installed and hence use the “pip install pssh” command to install the pssh tool.

2. PSSH tools

Pssh includes parallel versions of OpenSSH and there are many other related tools like

• pssh – is a program for running ssh in parallel on a multiple remote hosts.
• pscp – is a program for copying files in parallel to a number of hosts.
• prsync – is a program for efficiently copying files to multiple hosts in parallel.
• pnuke – kills processes on multiple remote hosts in parallel.
• pslurp – copies files from multiple remote hosts to a central host in parallel.

3. Starting PSSH

In order to see the help for the pssh, we can use

[puppet@root$:~/.pip]$  pssh --help

Usage: pssh [OPTIONS] command [...]

Options:

  --version             show program's version number and exit

  --help                show this help message and exit

  -h HOST_FILE, --hosts=HOST_FILE

                        hosts file (each line "[user@]host[:port]")

  -H HOST_STRING, --host=HOST_STRING

                        additional host entries ("[user@]host[:port]")

  -l USER, --user=USER  username (OPTIONAL)

  -p PAR, --par=PAR     max number of parallel threads (OPTIONAL)

  -o OUTDIR, --outdir=OUTDIR

                        output directory for stdout files (OPTIONAL)

  -e ERRDIR, --errdir=ERRDIR

                        output directory for stderr files (OPTIONAL)

  -t TIMEOUT, --timeout=TIMEOUT

                        timeout (secs) (0 = no timeout) per host (OPTIONAL)

  -O OPTION, --option=OPTION

                        SSH option (OPTIONAL)

  -v, --verbose         turn on warning and diagnostic messages (OPTIONAL)

  -A, --askpass         Ask for a password (OPTIONAL)

  -x ARGS, --extra-args=ARGS

                        Extra command-line arguments, with processing for

                        spaces, quotes, and backslashes

  -X ARG, --extra-arg=ARG

                        Extra command-line argument

  -i, --inline          inline aggregated output and error for each server

  --inline-stdout       inline standard output for each server

  -I, --send-input      read from standard input and send as input to ssh

  -P, --print           print output as we get it

In order to use pssh tool,we need to pass the host file as argument and this is done by using “-h <host File>” or “—hosts <host File>”

We can also pass the user name using “-l <user Name>” or “—user <user Name>”

Standard Error and Standard message on each host executions can be seen by passing the “-i or -–inlineoption” and we can use the “-o <path to Dir>” to save the standard Output.

If we want the pssh to ask for the password we can pass “-A” Option.

Since we will be doing a SSH to the remote machine we need to specify a Timeout to check how long a command takes. It defaults to 60 seconds. This means that if your command fails to complete within 60 seconds on a host, pssh will consider it an error and report it. By default, pssh uses at most 32 ssh processes in parallel to ssh to the various nodes. By default, it also uses a timeout of one minute to ssh to a node and obtain a result.

4. Using PSSH

Create a file containing the hosts that we need to ssh and run the commands. I created the file “pssh-hosts” and added the remote machines that I want to ssh

[puppet@root$:/work]$ cat  pssh-hosts 

vagrant@devm.foohost.vm

vagrant@devs.foohost.vm

During this point, we need to make sure both the above machines have the user vagrant available and both machines have the ssh keys configured for the root machine that we are running pssh commands. We can add the arguments which will ask for the password while executing the pssh command. Passing of the user vagrant is not mandatory as we can pass the user while running the command too.

Once the file is created lets run a sample command as,

[puppet@root$:/work]$  pssh -h pssh-hosts -l vagrant echo "hello World"

[1] 08:09:14 [SUCCESS] devs.foohost.vm

[2] 08:09:14 [SUCCESS] devm.foohost.vm

[puppet@root$:/work]$  pssh -h pssh-hosts -l vagrant "touch /tmp/hello"

[1] 08:10:47 [SUCCESS] devm.foohost.vm

[2] 08:10:47 [SUCCESS] devs.foohost.vm

If we want to see the output, we can use “-I” argument as,

[puppet@root$:/work]$  pssh -h pssh-hosts -l vagrant -i "df -hT"

[1] 08:11:54 [SUCCESS] devm.foohost.vm

Filesystem              Type      Size  Used Avail Use% Mounted on

/dev/mapper/centos-root xfs        50G  1.8G   49G   4% /

devtmpfs                devtmpfs  235M     0  235M   0% /dev

/dev/mapper/centos-home xfs        29G   49M   29G   1% /home

/dev/sda1               xfs       497M  148M  350M  30% /boot

vagrant                 vboxsf     49G   14G   36G  28% /vagrant

[2] 08:11:54 [SUCCESS] devs.foohost.vm

Filesystem              Type      Size  Used Avail Use% Mounted on

/dev/mapper/centos-root xfs        50G  1.1G   49G   3% /

devtmpfs                devtmpfs  235M     0  235M   0% /dev

/dev/mapper/centos-home xfs        29G   33M   29G   1% /home

vagrant                 vboxsf     49G   14G   36G  28% /vagrant

These are some of the basic usages of the Pssh command. As we already discussed we do have other tool available with the pssh command.

PSCP – This is much similar to the SCP command available in Linux but the only difference is that this will do the scp on multiple machines at same time. we can use the command as,

Basic Usage - pscp -h ip remote file on the local file directory

[puppet@root$:/work]$  pscp -h pssh-hosts -l root /work/roles.zip /tmp

[1] 01:13:22 [SUCCESS] devm.foohost.vm

[2] 01:13:23 [SUCCESS] devs.foohost.vm

Pnuke - The pnuke command is useful when you want to kill a bunch of processes on a set of machines. For example, suppose you've got a bunch of java processes running on three nodes that you'd like to nuke. Here you would do the following:

Basic Usage - plurp - h ip file - L local directory remote file local file name

[puppet@root$:/work]$ pnuke -h pssh-hosts -l vagrant java

Warning: do not enter your password if anyone else has superuser

privileges or access to your account.

Password: 

[1] 01:31:06 [SUCCESS] devm.foohost.vm

[2] 01:31:07 [SUCCESS] devm.foohost.vm

Pslurp – The Pslurp command is usefull when we want to copy files in parallel from multiple remote hosts to the central host

Basic Usage - plurp - h ip file - L local directory remote file local file name

- L specify the directory to store files locally

[puppet@root$:/work]$  pslurp -h pssh-hosts -L /work /tmp/master.txt /tmp

[1] 01:24:03 [SUCCESS] root@devm.foohost.vm

[2] 01:24:03 [SUCCESS] root@devs.foohost.vm

The above Command will grab a file or directory (/tmp/master.txt) from the remote machine defined in the hosts file to the local /tmp location. The –r option which is a recursive will make a new directory with the name of the remote host and inside the directory a copy of the files will be stored.

Prsync – This command is much similar to the rsync command in linux which will sync the local locations to the remote Location

Basic Use - pslurp -h hosts -L ~/tmp /tmp/example.txt example.txt

The above will download /tmp/example.txt as example.txt, and will store the file in ~/tmp/[host]/, where [host] is the corresponding hostname to that found in your hosts file. 

[puppet@root$:/work]$  prsync -r -h pssh-hosts -l vagrant ./roles /tmp

[1] 09:31:15 [SUCCESS] devs.foohost.vm

[2] 09:31:16 [SUCCESS] devm.foohost.vm

Note – Make Sure the Rysnc command is available on the remote machines if this needs to work.

More to Come. Happy learning J

Read More