Thursday, August 25, 2016

Sonar Integration with maven

In the next series of sonar, we will see how we can integrate sonar with maven and use maven command in analyzing the source code of a project and display the results on the sonarqube console.

1. Make sure JDK,Maven and Sonar are available ( with sonar running ).
2. Make the changes to the global maven setting.xml file available at location ~/.m2
3. Add the pluginGroups as,

4. Add the Profile pointing the location of the sonar console as,


4. Once the changes are done, Move to you java application and run,

[puppet@root$:/work/testing/javaee7-simple-sample]$ mvn clean verify sonar:sonar
[INFO] Scanning for projects...
[INFO] ------------------------------------------------------------------------
[INFO] Building javaee7-simple-sample 1.11-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO] --- maven-war-plugin:2.3:war (default-war) @ javaee7-simple-sample ---
[INFO] Packaging webapp
[INFO] Assembling webapp [javaee7-simple-sample] in [/work/testing/javaee7-simple-sample/target/javaee7-simple-sample]
[INFO] Processing war project
[INFO] Copying webapp resources [/work/testing/javaee7-simple-sample/src/main/webapp]
[INFO] Webapp assembled in [63 msecs]
[INFO] Building war: /work/testing/javaee7-simple-sample/target/javaee7-simple-sample.war
[INFO] ------------------------------------------------------------------------
[INFO] Building javaee7-simple-sample 1.11-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO] --- sonar-maven-plugin:3.0.2:sonar (default-cli) @ javaee7-simple-sample ---
 [INFO] User cache: /root/.sonar/cache
[INFO] SonarQube version: 4.5.7
[INFO] Default locale: "en_US", source code encoding: "UTF-8"
[INFO] [02:21:17.060] Load global referentials...
[INFO] [02:21:17.393] Load global referentials done: 334 ms
[INFO] [02:21:17.411] User cache: /root/.sonar/cache
[INFO] [02:21:17.420] Install plugins
[INFO] [02:21:17.739] Install JDBC driver
[INFO] [02:21:17.743] Create JDBC datasource for jdbc:h2:tcp://localhost/sonar
[INFO] [02:21:18.596] Initializing Hibernate
[INFO] [02:21:19.930] Load project referentials...
[INFO] [02:21:20.315] Load project referentials done: 385 ms
[INFO] [02:21:20.315] Load project settings
[INFO] [02:21:21.131] Loading technical debt model...
[INFO] [02:21:21.151] Loading technical debt model done: 20 ms
[INFO] [02:21:21.153] Apply project exclusions
[INFO] [02:21:21.395] -------------  Scan javaee7-simple-sample
[INFO] [02:21:21.398] Load module settings
[INFO] [02:21:21.720] Loading rules...
[INFO] [02:21:22.089] Loading rules done: 369 ms
[INFO] [02:21:22.113] Configure Maven plugins
[INFO] [02:21:22.170] No quality gate is configured.
[INFO] [02:21:22.329] Initializer FindbugsMavenInitializer...
[INFO] [02:21:22.330] Initializer FindbugsMavenInitializer done: 1 ms
[INFO] [02:21:22.330] Base dir: /work/testing/javaee7-simple-sample
[INFO] [02:21:22.330] Working dir: /work/testing/javaee7-simple-sample/target/sonar
[INFO] [02:21:22.330] Source paths: src/main/webapp, pom.xml, src/main/java
[INFO] [02:21:22.330] Binary dirs: target/classes
[INFO] [02:21:22.331] Source encoding: UTF-8, default locale: en_US
[INFO] [02:21:22.331] Index files
[INFO] [02:21:22.363] 4 files indexed
[INFO] [02:21:22.414] Quality profile for java: Sonar way
[INFO] [02:21:22.427] Sensor JavaSquidSensor...
[INFO] [02:21:22.600] Java Main Files AST scan...
[INFO] [02:21:22.603] 4 source files to be analyzed
[INFO] [02:21:22.765] 4/4 source files analyzed
[INFO] [02:21:22.767] Java Main Files AST scan done: 167 ms
[INFO] [02:21:22.768] Java bytecode scan...
[INFO] [02:21:22.790] Java bytecode scan done: 22 ms
[INFO] [02:21:22.790] Java Test Files AST scan...
[INFO] [02:21:22.790] 0 source files to be analyzed
[INFO] [02:21:22.791] Java Test Files AST scan done: 1 ms
[INFO] [02:21:22.791] 0/0 source files analyzed
[INFO] [02:21:22.795] Package design analysis...
[INFO] [02:21:22.799] Package design analysis done: 4 ms
[INFO] [02:21:22.822] Sensor JavaSquidSensor done: 395 ms
[INFO] [02:21:22.822] Sensor QProfileSensor...
[INFO] [02:21:22.826] Sensor QProfileSensor done: 4 ms
[INFO] [02:21:22.826] Sensor InitialOpenIssuesSensor...
[INFO] [02:21:22.863] Sensor InitialOpenIssuesSensor done: 37 ms
[INFO] [02:21:22.863] Sensor ProjectLinksSensor...
[INFO] [02:21:22.866] Sensor ProjectLinksSensor done: 3 ms
[INFO] [02:21:22.867] Sensor VersionEventsSensor...
[INFO] [02:21:22.872] Sensor VersionEventsSensor done: 4 ms
[INFO] [02:21:22.872] Sensor FileHashSensor...
[INFO] [02:21:22.873] Sensor FileHashSensor done: 1 ms
[INFO] [02:21:22.873] Sensor SurefireSensor...
[INFO] [02:21:22.874] parsing /work/testing/javaee7-simple-sample/target/surefire-reports
[WARN] [02:21:22.874] Reports path not found: /work/testing/javaee7-simple-sample/target/surefire-reports
[INFO] [02:21:22.874] Sensor SurefireSensor done: 1 ms
[INFO] [02:21:22.874] Sensor Maven dependencies...
[INFO] [02:21:22.897] Sensor Maven dependencies done: 23 ms
[INFO] [02:21:22.897] Sensor CPD Sensor (wrapped)...
[INFO] [02:21:22.897] JavaCpdEngine is used for java
[INFO] [02:21:22.898] Cross-project analysis disabled
[INFO] [02:21:22.917] Sensor CPD Sensor (wrapped) done: 20 ms
[INFO] [02:21:23.018] Execute decorators...
[INFO] [02:21:23.121] Store results in database
[INFO] [02:21:23.208] ANALYSIS SUCCESSFUL, you can browse
[INFO] [02:21:23.255] Executing post-job class org.sonar.plugins.core.issue.notification.SendIssueNotificationsPostJob
[INFO] [02:21:23.255] Executing post-job class org.sonar.plugins.core.batch.IndexProjectPostJob
[INFO] [02:21:23.272] Executing post-job class org.sonar.plugins.dbcleaner.ProjectPurgePostJob
[INFO] [02:21:23.283] -> Keep one snapshot per day between 2016-07-27 and 2016-08-23
[INFO] [02:21:23.284] -> Keep one snapshot per week between 2015-08-26 and 2016-07-27
[INFO] [02:21:23.284] -> Keep one snapshot per month between 2011-08-31 and 2015-08-26
[INFO] [02:21:23.284] -> Delete data prior to: 2011-08-31
[INFO] [02:21:23.288] -> Clean javaee7-simple-sample [id=1]
[INFO] ------------------------------------------------------------------------
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 38.600 s
[INFO] Finished at: 2016-08-24T02:21:23-04:00
[INFO] Final Memory: 30M/319M
[INFO] ------------------------------------------------------------------------

Once the Build is success we can see the results in the console as,

More to Come, Happy learning J
Read More

SonarQube - Static Code Analysis

Testing is one of the important aspects of code to identify various issues that can occur while running the code. At the same time it is necessary to examine the code quality while moving the code to production.

Static analysis also called as static code analysis is a method of debugging that is done by examining the code without executing that. This allows developers a better understanding of the code structure and can help developers to ensure that the code adheres to industry standards. The main advantage of the static analysis is that it reveals errors that do not occur until months and years of application running. It is said that the static analysis is only a first step in a comprehensive software quality-control regime. Sonar is one such tool which provides us the static code analysis.

Sonar is an open source web-based application to manage code quality which covers seven axes of code quality as: Architecture and design, comments, duplications, unit tests, complexity, potential bugs and coding rules. Developed in Java and can cover projects in Java, FlexPHPPL/SQL,  Visual Basic 6. It's very efficient to navigate; offering visual reporting and you can follow metrics evolution of your project and combine them.

In this article we will see how we can install SONAR tool and use that.

1. Download SonarQube from here

2. Extract the tar file to /op/sonarqube.
Once extracted move to the /opt/sonarqube/bin/linux-x86-64.And run the “  start”.

That’s all you need to do in starting the sonarqube. Access the sonarqube console using the “localhost:9000” and we can see the web console as below,

 The default credentials for login are admin and admin. More to come using this tool. Stay learning J
Read More

Wednesday, August 24, 2016

PSSH – Parallel SSH

Many times we use SSH to login to the remote machines, Copy files, perform administrative tasks. But in all the cases we do these operations machine by machine i.e. we log to one machine perform the operations and then log out to go to the next machine. If we want to increase the productivity with SSH, we can use Tools that allow commands to be executed on multiple remote machines simultaneously. Parallel SSH is one such tool which allows running commands on multiple servers at the same time.

This article will give you a brief introduction about the Parallel SSH tool.

1. Install Parallel SSH
Parallel SSH tool is based on Python. PSSH is supported on Python 2.4 or more.In order to install use the PIP tool provided by Python. Install the parallel tool using

Pip install pssh

[puppet@root$:~/.pip]$  pip install pssh
Collecting pssh
  Downloading pssh-2.3.1.tar.gz
Building wheels for collected packages: pssh
  Running bdist_wheel for pssh ... done
Stored in directory: /root/.cache/pip/wheels/b6/98/92/eab367fee5ded0129e0d91feb3207e76fcb282ccc87507a6d0
Successfully built pssh
Installing collected packages: pssh
Successfully installed pssh-2.3.1

A version of pssh with 2.3.1 version is installed.

Note – pip is a tool by Python to install python packages. In order to install pip use the command “yum install python-pip”. In most linux version python will be already installed and hence use the “pip install pssh” command to install the pssh tool.

2. PSSH tools
Pssh includes parallel versions of OpenSSH and there are many other related tools like

  • pssh – is a program for running ssh in parallel on a multiple remote hosts.
  • pscp – is a program for copying files in parallel to a number of hosts.
  • prsync – is a program for efficiently copying files to multiple hosts in parallel.
  • pnuke – kills processes on multiple remote hosts in parallel.
  • pslurp – copies files from multiple remote hosts to a central host in parallel.
3. Starting PSSH
In order to see the help for the pssh, we can use

[puppet@root$:~/.pip]$  pssh --help
Usage: pssh [OPTIONS] command [...]

  --version             show program's version number and exit
  --help                show this help message and exit
  -h HOST_FILE, --hosts=HOST_FILE
                        hosts file (each line "[user@]host[:port]")
                        additional host entries ("[user@]host[:port]")
  -l USER, --user=USER  username (OPTIONAL)
  -p PAR, --par=PAR     max number of parallel threads (OPTIONAL)
  -o OUTDIR, --outdir=OUTDIR
                        output directory for stdout files (OPTIONAL)
  -e ERRDIR, --errdir=ERRDIR
                        output directory for stderr files (OPTIONAL)
  -t TIMEOUT, --timeout=TIMEOUT
                        timeout (secs) (0 = no timeout) per host (OPTIONAL)
  -O OPTION, --option=OPTION
                        SSH option (OPTIONAL)
  -v, --verbose         turn on warning and diagnostic messages (OPTIONAL)
  -A, --askpass         Ask for a password (OPTIONAL)
  -x ARGS, --extra-args=ARGS
                        Extra command-line arguments, with processing for
                        spaces, quotes, and backslashes
  -X ARG, --extra-arg=ARG
                        Extra command-line argument
  -i, --inline          inline aggregated output and error for each server
  --inline-stdout       inline standard output for each server
  -I, --send-input      read from standard input and send as input to ssh
  -P, --print           print output as we get it

In order to use pssh tool,we need to pass the host file as argument and this is done by using “-h <host File>” or “—hosts <host File>”

We can also pass the user name using “-l <user Name>” or “—user <user Name>”

Standard Error and Standard message on each host executions can be seen by passing the “-i or -–inlineoption” and we can use the “-o <path to Dir>” to save the standard Output.

If we want the pssh to ask for the password we can pass “-A” Option.

Since we will be doing a SSH to the remote machine we need to specify a Timeout to check how long a command takes. It defaults to 60 seconds. This means that if your command fails to complete within 60 seconds on a host, pssh will consider it an error and report it. By default, pssh uses at most 32 ssh processes in parallel to ssh to the various nodes. By default, it also uses a timeout of one minute to ssh to a node and obtain a result.

4. Using PSSH

Create a file containing the hosts that we need to ssh and run the commands. I created the file “pssh-hosts” and added the remote machines that I want to ssh

[puppet@root$:/work]$ cat  pssh-hosts 

During this point, we need to make sure both the above machines have the user vagrant available and both machines have the ssh keys configured for the root machine that we are running pssh commands. We can add the arguments which will ask for the password while executing the pssh command. Passing of the user vagrant is not mandatory as we can pass the user while running the command too.

Once the file is created lets run a sample command as,

[puppet@root$:/work]$  pssh -h pssh-hosts -l vagrant echo "hello World"
[1] 08:09:14 [SUCCESS] devs.foohost.vm
[2] 08:09:14 [SUCCESS] devm.foohost.vm

[puppet@root$:/work]$  pssh -h pssh-hosts -l vagrant "touch /tmp/hello"
[1] 08:10:47 [SUCCESS] devm.foohost.vm
[2] 08:10:47 [SUCCESS] devs.foohost.vm

If we want to see the output, we can use “-I” argument as,
[puppet@root$:/work]$  pssh -h pssh-hosts -l vagrant -i "df -hT"
[1] 08:11:54 [SUCCESS] devm.foohost.vm
Filesystem              Type      Size  Used Avail Use% Mounted on
/dev/mapper/centos-root xfs        50G  1.8G   49G   4% /
devtmpfs                devtmpfs  235M     0  235M   0% /dev
/dev/mapper/centos-home xfs        29G   49M   29G   1% /home
/dev/sda1               xfs       497M  148M  350M  30% /boot
vagrant                 vboxsf     49G   14G   36G  28% /vagrant
[2] 08:11:54 [SUCCESS] devs.foohost.vm
Filesystem              Type      Size  Used Avail Use% Mounted on
/dev/mapper/centos-root xfs        50G  1.1G   49G   3% /
devtmpfs                devtmpfs  235M     0  235M   0% /dev
/dev/mapper/centos-home xfs        29G   33M   29G   1% /home
vagrant                 vboxsf     49G   14G   36G  28% /vagrant

These are some of the basic usages of the Pssh command. As we already discussed we do have other tool available with the pssh command.

PSCP – This is much similar to the SCP command available in Linux but the only difference is that this will do the scp on multiple machines at same time. we can use the command as,

Basic Usage - pscp -h ip remote file on the local file directory

[puppet@root$:/work]$  pscp -h pssh-hosts -l root /work/ /tmp
[1] 01:13:22 [SUCCESS] devm.foohost.vm
[2] 01:13:23 [SUCCESS] devs.foohost.vm

Pnuke - The pnuke command is useful when you want to kill a bunch of processes on a set of machines. For example, suppose you've got a bunch of java processes running on three nodes that you'd like to nuke. Here you would do the following:

Basic Usage - plurp - h ip file - L local directory remote file local file name

[puppet@root$:/work]$ pnuke -h pssh-hosts -l vagrant java
Warning: do not enter your password if anyone else has superuser
privileges or access to your account.
[1] 01:31:06 [SUCCESS] devm.foohost.vm

[2] 01:31:07 [SUCCESS] devm.foohost.vm

Pslurp – The Pslurp command is usefull when we want to copy files in parallel from multiple remote hosts to the central host

Basic Usage - plurp - h ip file - L local directory remote file local file name
- L specify the directory to store files locally

[puppet@root$:/work]$  pslurp -h pssh-hosts -L /work /tmp/master.txt /tmp
[1] 01:24:03 [SUCCESS] root@devm.foohost.vm
[2] 01:24:03 [SUCCESS] root@devs.foohost.vm

The above Command will grab a file or directory (/tmp/master.txt) from the remote machine defined in the hosts file to the local /tmp location. The –r option which is a recursive will make a new directory with the name of the remote host and inside the directory a copy of the files will be stored.

Prsync – This command is much similar to the rsync command in linux which will sync the local locations to the remote Location

Basic Use - pslurp -h hosts -L ~/tmp /tmp/example.txt example.txt

The above will download /tmp/example.txt as example.txt, and will store the file in ~/tmp/[host]/, where [host] is the corresponding hostname to that found in your hosts file. 

[puppet@root$:/work]$  prsync -r -h pssh-hosts -l vagrant ./roles /tmp
[1] 09:31:15 [SUCCESS] devs.foohost.vm

[2] 09:31:16 [SUCCESS] devm.foohost.vm

Note – Make Sure the Rysnc command is available on the remote machines if this needs to work.

More to Come. Happy learning J
Read More

Saturday, August 20, 2016

Nexus-Maven Integration

In this article we will see how we can integrate Maven with nexus and push our artifacts to the nexus repository.

1. Install and configure maven.
2. Once maven is configured , a local repository is created. The maven local repository is a local folder that is used to store all your project’s dependencies (plugin jars and other files which are downloaded by Maven). In simple, when you build a Maven project, all dependency files will be stored in your Maven local repository.

By default, Maven local repository is default to .m2 folder:
  1. Unix/Mac OS X – ~/.m2
  2. Windows – C:\Documents and Settings\{your-username}\.m2 
The local repository contains a settings.xml file which contains the configuration dertails.

3. Make changes to the settings.xml file for making our artifacts go to the Nexus repository.
There will 2 users ID admin for admin operations and deployment for deploy operations.

Add the below content to the ~/.m2/setting.xml as

Add the nexus Mirror with the location where nexus is running as,

      <name>Human Readable Name for this Mirror.</name>

Add a server location which  Specifies the authentication information to use when connecting to a particular server, identified by


4. Once the changes are done. let’s try to build a artifact and push to the nexus. Create a maven application or Download the javaee7-simple-sample application from the GITHUB location

Add the distribution Management element to the pom.xml file as

            <name>Internal Releases</name>
            <name>Internal Snapshot Releases</name>

In the URL’s, add the URL where the nexus Repository is running.

5. Once the changes are done. Run the “mvn clean deploy” command 
Read More

Nexus – HTTPS Configuration

Since we use nexus as a part of warehouse for our organization that contains all our building blocks and software, it is very important to secure the access to nexus. Nexus is normally access over HTTP. If we do the communications over normal http, this traffic can be intercepted with any network sniffer and credentials can be read. Using Secure Socket Layer (SSL) communication with the repository manager is an important security feature and a recommended best practice. 

In this article we will see how we can configure nexus over HTTPS.

1. add the ssl port to the file

2. We need to create a Key store file for configuring the nexus with ssl. We need to have a password for creating the keystore. The same password needs to be used in the configuration file too. If we use the plain text password in configuration file it may lead to a security breach. So we Obfuscate password.

Password can be Obfuscated using,

[root@puppet lib]# java -cp NEXUS_HOME/lib/jetty-util-8.1.16.v20140903.jar changeit changeit

jetty-util-8.1.16.v20140903.jar - NEXUS_HOME/lib contains the jetty-util jar. Use thejar based on the version. I have the jetty-util-8.1.16.v20140903.jar in the nexus iam using.

3. Now once we have the password obfuscated we now create the keystore using the same password as above “changeit”.

keytool -genkey -noprompt -keystore NEXUS_HOME/conf/keystore.jks -alias nexus-alias -keyalg RSA -keypass changeit -storepass changeit -dname ", OU=ID, O=artl, L=jags, S=macnhala, C=GB" 

Now the keystore file is stored in NEXUS_HOME/conf location.

4. Now make the changes to the jetty-https.xml configuration for the addConnector section as,

<Set name="keyStore"> NEXUS_HOME/conf/keystore.jks</Set>
<Set name="trustStore"> NEXUS_HOME/conf/keystore.jks</Set>
<Set name="keyStorePassword">1vn21ugu1saj1v9i1v941sar1ugw1vo0</Set>
<Set name="keyManagerPassword">1vn21ugu1saj1v9i1v941sar1ugw1vo0</Set>
<Set name="trustStorePassword">1vn21ugu1saj1v9i1v941sar1ugw1vo0</Set>

For the Store password, we change the value with Obfuscated password.

5. Don’t make any changes to the jetty-http-redirect-to-https.xml file. This file is used to redirect http to https automatically. Whenever we access the nexus URL with http it automatically directs to https.

6. Make the last changes to the NEXUS_HOME/bin/jsw/conf/wrapper.conf file with below contents as, NEXUS_HOME/conf/jetty-https.xml NEXUS_HOME/conf/jetty-http-redirect-to-https.xml

7. Start the nexus application using NEXUS_HOME/bin/nexus restart

8. Access the application using http://localhost:9091/nexus which will automatically redirect to https://localhost:9443/nexus

More to Come, Happy learning J
Read More

Artifactory - Nexus

Before starting to understand what nexus is and how it can be used we need to understand what a software repository is. A Software repository is a storage location from which software packages may be retrieved and installed on the computer (according to WIKI).

So an artifactory repository is a binary repository manager. Much like source control that we use for our code (SVN, GIT) we use artifactory repository manager for storing and sharing our binary artifacts like jar and war. Artifactory is also a place where you can put a shared library so that it is easily accessible in other projects across the enterprise.

Many programming languages maintain their artifactory so that other users can download from that artifactory repository. An artifactory provide Operators on repositories like a package management system, tools intended to search for, install and otherwise manipulate software packages from the repositories

So why do we need a artifactory repository tool.

1. Re-building from source introduces points of failures. BY having the artifact in a binary repository manager we can down and use which will be a versioned copy of the tested artifact

2. Since the binaries are version it is easy to have multiple versions of libraries so that all projects do not have to be on the same version.

Repository types – There are different types of repositories available in nexus such as,

Virtual repositories - “virtual repositories” are proxies to all of the “well known” repositories on the internet. This give you access to most of the public shared libraries such as Apache Commons, Spring, Hibernate, etc..; through one conduit.

Proxy Repository - Is a repository with the type proxy, also known as a proxy repository, is a repository that is linked to a remote repository.Any request for a component is verified against the local content of the proxy repository. If no local component is found, the request is forwarded to the remote repository. The component is then retrieved and stored locally in the repository manager, which acts as a cache. Subsequent requests for the same component are then fulfilled from the local storage, therefore eliminating the network bandwidth and time overhead of retrieving the component from the remote repository again.

Hosted Repository - A repository with the type hosted, also known as a hosted repository, is a repository that stores components in the repository manager as the authoritative location for these components.

In this article we will see how we can configure nexus repository and use,

1.  Nexus comes with 2 flavors “Nexus Repository Manager Pro” and “Nexus Repository Manager OSS”.

Nexus Repository Manager OSS is a fully-featured repository manager which can be freely used, customized, and distributed under the Eclipse Public License.

Nexus Repository Manager Pro is a distribution with features that are relevant to large enterprises and organizations which require complex procurement and staging workflows in addition to more advanced LDAP integration, Atlassian Crowd support, and other development infrastructure. This is available for a 30 days trail.

2. Download the “Nexus Repository Manager OSS” from

3. Unzip or Tar based on the file you downloaded using
unzip OR tar xvzf nexus-2.13.0-01-bundle.tar.gz

4. Before starting nexus ,we need to make a few changes to the file in NEXUS_HOME/conf/

Change the nexus-work=<Location to store the nexus runtime Data>
Change the application-port to a port number where we can access the nexus Browser.

4. Once saved, start the nexus using NEXUS_HOME/bin/nexus start.

More to Come, Happy learning J
Read More