Pipeline Abstraction

I struggle quite often with how much code to actually abstract in a build pipeline. We use Jenkins Pipelines for our builds, with a Jenkinsfile in each project to control its build. We also have a few different versions of pipeline Libraries for helping out with the builds.

All in the Jenkinsfile

One obvious way to go is to put everything in the Jenkinsfile, and not share any code via a library. This approach is quite tempting, because you can actually treat each service independently (an ideal for microservices). However, in practice it can be quite wasteful and just a hassle. You will end up with duplications all over. And we all know that things change, so at some point you will need to go in and update all the projects with a similar change.

I don’t think we have any of these any more for examples. We have pulled out the most egregious duplications. This is a balancing act; how close to the ideal do you want to be, balanced with how effectively you can actually make changes.

Nothing in the Jenkinsfile

We also have some services that really do not have much in the Jenkinsfile. Most of our services follow the same basic build flow, with certain predictable and minor changes. So these ones will only declare the things that change between services.

#!groovy
import com.example.pipeline.NodeBuildManager

msvcPipeline {
  buildManager = new NodeBuildManager(this, 'node-10.16')
  serviceName = 'nodejs-template'
  group = 'my-services/template'
  automatedTest = { env, config ->
    String serviceUrl = getServiceUrl(env, config.serviceName)
    try {
      sh 'chmod u+x automated-test/gradlew'
      sh "automated-test/gradlew -b automated-test/build.gradle test -Duri=${serviceUrl}/${config.serviceName}/"
    }
    finally {
      junit(allowEmptyResults: true, testResults: 'automated-test/build/test-results/TEST-*.xml')
    }
  }
}

The most important/surprising pieces of this format is that you need to define a buildManager, which is a predefined set of things that know how to build code (Gradle for Java and NPM for Node.js), and the automatedTest closure. Each service runs these tests, which are basically functional integration tests, but each have their own little tweaks to them.

This approach is nice and quick to get started, but has some issues for longer-term maintenance. Because so much is abstracted away, the underlying library can change at any time, causing exciting new errors. Also, specifically for how Jenkins works, if you want to go in and muck with the build using the replay functionality, there isn’t much you can play with. It is all pulled into the library, which you can’t change in the replay.

Striking a Balance

The approach that seems to work the best is to strike a balance between the two approaches. Share some code, but not all of it.

#!groovy

properties([[$class: 'GitLabConnectionProperty', gitLabConnection: 'GitLab']])

serviceName = 'my-service'
branchName = env['BRANCH_NAME']
envName = buildUtilities.getEnvironmentName(branchName)

buildUtilities.initVars(branchName, envName)

serviceStartDelay = 180 // seconds
serviceRetryDelay = 30  // seconds
serviceRetryCount = 15

node {
  gitlabBuilds(builds: buildUtilities.buildsList) {
  stage('setup') {
    gitlabCommitStatus(name: 'setup') {
      buildUtilities.setup()
    }
  }

  stage('version check') {
    gitlabCommitStatus(name: 'version check') {
      buildUtilities.versionCheck(getCodeVersion())
    }
  }

  stage('verify') {
    gitlabCommitStatus(name: 'verify') {
      try {
        withEnv(["JAVA_HOME=${tool 'JAVA7'}", "PATH+JAVA=${tool 'JAVA7'}/bin"]) {
          sh './grailsw test-app unit: -coverage -xml --non-interactive'
        }
      } finally {
        junit(allowEmptyResults: true, testResults: 'target/test-reports/TEST-*.xml')
      }
      buildUtilities.sonarVerify(getProjectVersion())
    }
  }

  stage('build') {
    gitlabCommitStatus(name: 'build') {
      withEnv(["JAVA_HOME=${tool 'JAVA7'}", "PATH+JAVA=${tool 'JAVA7'}/bin"]) {
        sh "./grailsw prod war"
      }
      docker.withRegistry(buildUtilities.dockerRepoURI, 'docker-repo-login') {
        def dockerImage = docker.build(serviceName)
        def taggedImageName = dockerImage.tag(getProjectVersion())
        sh "docker push ${taggedImageName}"
        sh "docker rmi -f ${taggedImageName}"
      }
    }
  }

      if (branchName.startsWith('dv') || branchName == 'master') {
        stage('automated-test') {
          gitlabCommitStatus(name: 'automated-test') {
            buildUtilities.deployToEnvironment(envName, serviceName, branchName, getProjectVersion())
            if (buildUtilities.ensureDeploymentIsReady(envName, serviceName, getProjectVersion(), serviceStartDelay, serviceRetryDelay, serviceRetryCount)) {
              runAutomatedTests(envName)
            }
            else {
              sh "exit 1;"
            }
          }
        }
      }

    if(branchName == 'master')
    {
      stage('deploy') {
        gitlabCommitStatus(name: 'deploy') {
          buildUtilities.deployToAllDevEnvironments(serviceName, branchName, getProjectVersion())
          buildUtilities.tagGitRepo(getProjectVersion())
        }
      }
    }
  }
}

String getCodeVersion() {
  def versionProps = readProperties(text: readTrusted('application.properties'))
  return versionProps['app.version']
}

String getProjectVersion() {
  def projectVersion = getCodeVersion()
  if (env['BRANCH_NAME'] != 'master') {
    projectVersion += "-${buildUtilities.getGitCommit().take(12)}"
  }
  return projectVersion
}

def runAutomatedTests(String envName) {
  String nodePort = buildUtilities.getNodePort(envName, serviceName)
  String userPort = buildUtilities.getNodePort(envName, 'user')
  String personPort = buildUtilities.getNodePort(envName, 'person')
  try {
    sh "./gradlew -b automated-test/build.gradle test " +
     "-Duri=http://${buildUtilities.devClusterName}:${nodePort}/${serviceName}/ " +
     "-Duser.services=http://${buildUtilities.devClusterName}:${userPort}/user/ " +
     "-Dperson.services=http://${buildUtilities.devClusterName}:${personPort}/person/ "
  } finally {
    junit(allowEmptyResults: true, testResults: 'automated-test/build/test-results/**/TEST-*.xml')
  }
}

This one leaves much of the pipeline up to the “consumer”, but just abstracts away the kind of utility code. For example, we have that call buildUtilities.ensureDeploymentIsReady which hides the complexity involved with that, but allows us to pass in some parameters. This allows us to define how long to wait, and how many iterations to go through, but we don’t have to see the actual looping and parsing code that may be involved.