Complete Build Automation for Python Application Using Jenkins Pipeline

In this example pipeline, we will be deploying a python application using jenkins Pipeline code. The pipeline code includes checkout source code from Github, performs a quality scan using the pylint tool, does a unit test using the pytest tool and finally sends an email to the users.

1. Install the necessary tools python, pylint and pytest.

Installing Python  - Python will be by default available in all linux machines.

Installing Pylint - Pylint is a Python static code analysis tool which looks for programming errors, helps enforcing a coding standard, sniffs for code smells and offers simple refactoring suggestions.

Open a Command Prompt or Terminal. On Windows, navigate to the Python root directory (install location) and run the following command: python -m pip install pylint

Installing Pytest - Installing pytest is same as pylint. Open a Command Prompt or Terminal. On Windows, navigate to the Python root directory (install location) and run the following command: pip install -U pytest

Plugins to Install - There are couple of Jenkins plugin to be installed for the below pipeline to work. The plugins are warnings and publishHTML.

2. Create a Jenkins pipeline job, with scm pointing to the below github location. Download the source from here and create your own github repository.

3. Understanding the Jenkinsfile. The pipeline code has 3 different stages. The pipeline looks like below,

pipeline {
  agent any

 stages {
     stage('Checkout') {
            steps {
                git credentialsId: 'github-jenkins', url: 'https://github.com/jagadish12/funniest.git'
                echo 'CheckOut Success'
            }
        }

stage('lint'){
   steps {
        sh "virtualenv --python=/usr/bin/python venv"
        sh "export TERM='linux'"  
        sh 'pylint --rcfile=pylint.cfg funniest/ $(find . -maxdepth 1 -name "*.py" -print) --msg-template="{path}:{line}: [{msg_id}({symbol}), {obj}] {msg}" > pylint.log || echo "pylint exited with $?"'
        sh "rm -r venv/"
       
        echo "linting Success, Generating Report"
         
        warnings canComputeNew: false, canResolveRelativePaths: false, defaultEncoding: '', excludePattern: '', healthy: '', includePattern: '', messagesPattern: '', parserConfigurations: [[parserName: 'PyLint', pattern: '*']], unHealthy: ''
       
       }   
     }
      
stage('test'){
   steps {
         
        sh "pytest --cov ./ --cov-report html --verbose"
        publishHTML(target:
            [allowMissing: false,
              alwaysLinkToLastBuild: false,
            keepAll: false,
            reportDir: 'htmlcov',
            reportFiles: 'index.html',
            reportName: 'Test Report',
            reportTitles: ''])  
           
        echo "Testing Success"   
          }  
        }
       
stage('mail'){
        steps{
            emailext attachLog: true, body: 'Jenkins Build - Status Report', subject: 'Build Report', to: 'jagadesh.manchala@gmail.com'
        }
    }   

       }
}
The Pipeline contains 4 stages, checkout , lint , test and email.

Checkout - Create a Credential with the name “github-jenkins”. The same credential will be used to check out source code form the github.
   
stage('Checkout') {
            steps {
                git credentialsId: 'github-jenkins', url: 'https://github.com/jagadish12/funniest.git'
                echo 'CheckOut Success'
            }
        }

Lint - the seconds stage is the lint stage. In this stage we will run linting ( quality scan ) using the pylint tool. We will create a python virtualenv and perform the pylint testing inside the virtual environment.  Once the linting of the python code is done, the results will be displayed with the warning plugin in Jenkins.

stage('lint'){
       steps {
        sh "virtualenv --python=/usr/bin/python venv"
        sh "export TERM='linux'"  
        sh 'pylint --rcfile=pylint.cfg funniest/ $(find . -maxdepth 1 -name "*.py" -print) --msg-template="{path}:{line}: [{msg_id}({symbol}), {obj}] {msg}" > pylint.log || echo "pylint exited with $?"'
        sh "rm -r venv/"
       
        echo "linting Success, Generating Report"
         
        warnings canComputeNew: false, canResolveRelativePaths: false, defaultEncoding: '', excludePattern: '', healthy: '', includePattern: '', messagesPattern: '', parserConfigurations: [[parserName: 'PyLint', pattern: '*']], unHealthy: ''
       
       }   
     }

Test - The third stage is testing stage in which we will use the pytest tool to do the unit testing for the code that we have written.

stage('test'){
      steps {
         
        sh "pytest --cov ./ --cov-report html --verbose"
        publishHTML(target:
            [allowMissing: false,
              alwaysLinkToLastBuild: false,
            keepAll: false,
            reportDir: 'htmlcov',
            reportFiles: 'index.html',
            reportName: 'Test Report',
            reportTitles: ''])  
           
        echo "Testing Success"   
          }  
        }

Email - the Final stage is sending the email to the user defined in this stage. I have given mine , you can change that.

The Complete source code is available here. More to Come, Happy learning :-)