Executive Summary

DevOps has been the buzzword for the last couple of years and enterprises, big and small have slowly started adopting the practice. DevOps is the part of an IT organization’s team responsible for creating and maintaining the infrastructure. It can also be described as a practice that addresses a holistic view of the entire software delivery chain or the product life cycle and considers account shared services, facilitating continuous development, integration and delivery. This is achieved by using a set of tools that automate testing and streamline the development process thereby, enabling collaboration between the software development team and the operations team. DevOps tools enable teams to work more quickly and efficiently. But are these tools enough to foster a collaborative DevOps environment? The answer is no. DevOps is more than just a practice. It requires a change in the organizational culture. It requires a reassessment and re-alignment of how people think about the overall business, their customers as well as the teams.

Conway’s law states that organizations that design systems are constrained to produce designs which are copies of the communication structures of these organizations. This means the structure of your software is likely to reflect the structure of your software development organization.

Achieving Increased Collaboration through DevOps

On one hand the development team delivers new code as and when required, the operations team on the other hand maintains the stability by monitoring the changes and the frequency with which the code is deployed.

In order to achieve increased collaboration, there has to be a culture shift, not just within the team, but at an organization level as well.

Shared responsibility

If the development team works in silos, it is possible that they show some disinterest in the maintenance of a system, if it is solely handled by a different operations team. It would seem as if the development team’s job is to code only and eventually, present it on a platter for the operations team to handle it. The development team can share the responsibility of looking after a system, which will help them to identify ways to simplify deployment and maintenance. This can be achieved through automated deployments and monitoring. They will also get additional insights from monitoring the system in production.

Similarly, when the operations team works closely with the developers, they are able to understand the operational needs of a system in a better way and implement and adopt new automation tools and practices. It is recommended to slowly move on from the traditional process of handover and sign-offs as it discourages people from sharing responsibility. The developers and operations team both, are responsible for the success or failure of a system.

Autonomous teams

The concept of an autonomous team is that the developers as well as the members from the operations teams should be able to make decisions without following a convoluted decision making process. The teams are able to work in a favorable environment where there is no scope of discrepancies. Instead of waiting for a manual sign off for code to be deployed in testing, you can rely on a version control system which can easily be audited. Without the need of a manual sign-off, the team can automate the deployments and speed up the testing cycle. This would also, ensure that the team builds quality product in the development process. The concept of self-testing code also, allows regular and low-risk deployments.

Feedback

This aspect is extremely important in order to continuously improve the collaboration between development and operations and to improve the overall system itself. Monitoring how an application or a system behaves in production creates an environment of ownership and takes note of potential improvement areas.

Figure 1 : Dev & Ops Collaboration

Building a Successful DevOps Team Structure

Adopting practices that promote continuous integration and continuous delivery is a key enabler of DevOps within organizations. However, organizations have to first build a DevOps team structure that facilitates these practices in order to be successful.

In order to establish a successful DevOps culture, we need to follow the steps mentioned below:

01 Assemble the right resources for the DevOps team structure

Techbeacon has identified seven different roles that are important for any organization looking to implement DevOps practice.

The DevOps Evangelist

This person promotes the benefits of the DevOps approach by identifying the business benefits that an organization sets to gain by implementing it. He/she is also, responsible for being the change agent who identifies the key roles and gets the development, as well as the operations team on board and ensures that these resources receive the necessary training to make required changes.

Release Manager

This person is responsible for managing and coordinating the lifecycle of the product, from development through production. He/she is also, responsible for creating and managing the end-to-end application delivery tool chain.

Automation Architect

This person is responsible for designing, implementing and analyzing continuous deployment strategies. There is a need to ensure high availability of the application on production and pre-production systems.

Software Developer/Tester

The role of a developer now is quite different from what his/her role was as per the traditional approach. In a DevOps environment, the developer is not just expected to write code but also, responsible for unit testing, deployment and ongoing monitoring of the application.

Experience Assurance Professional

The traditional QA takes on this role in a DevOps environment wherein, the focus of the tester is functionality testing and user experience testing.

Security Engineer

In DevOps environment, the security engineer works in parallel with the developers to ensure that his/her recommendations are taken into consideration much earlier in the process rather than towards the end.

Utility Technology Player

This is a person who can operate effectively across different platforms, tools, networks, servers and databases. The person has expertise in multiple areas, adept at collaborating across disciplines and has the ability to apply knowledge in areas of expertise other than his/her own. These roles can vary depending on the kind of organization and the kind of product that is being built. It is important to have communication and collaboration among the team members in order to deliver a successful product

02 Business goals should be aligned

Even though a team has been setup, an organization should not make hasty decisions with regard to the implementation of DevOps practices. A company’s business goals must be aligned with the team’s setup. The vision of the company must align with the objectives of the DevOps team.

This will instill a sense of purpose in the minds of the team members wherein, they will not just be working on a particular project or task but will have the organization’s business objectives in mind while implementing their strategies.

03 Adopt suitable tools

Organizations that are looking to implement DevOps and trying to achieve continuous integration and delivery through automated testing and continuous deployment have a plethora of automation tools available in the market. It is extremely important to identify the set of tools that align with your setup and your team structure.

04 Measure the effectiveness of the approach

There should be key performance indicators in place to closely monitor the effectiveness of the team structure, roles and the tools selected. Some examples of performance indicators could be : measuring the number of deployments happening per day, whether the security or quality standards are being met or not, the volume of errors and bugs reported and how efficiently they are being handled.

Adopting DevOps

In earlier days, successful deployment of applications was a lengthy process. Once the code had been checked in to the code repository, the developers themselves used to build the deliverable in their local system and then the package was shared with the operations team. For operations team to deploy a package, a service request/ change request would have to be initiated in the change management system, and then requests for approvals were sent to the business, operations, etc. Once all approvals were in place, the operations team would validate the package and schedule deployment for the next day. The package was shared via a shared location such as FTP. During the deployment, someone from development team had to be available to assist in case any issues came up during deployment. The development/ QA team then had to verify the deployment and inform the stakeholders about the successful deployment. The entire process was time consuming and the chances of occurrence of errors were too high

Below is a summary of the limitations of the existing process:

1. Development team creating the build

• Development team tends to include last minute fixes also, to the build, thereby, breaking build in higher environments.
• Chances of deploying partially developed code.

2. Creating ticket for operations team

• As operations team has to deploy for every team, each team has to wait for 24 to 48 hours for deployment
• As operations team manually deploys the application, there are chances of missing steps resulting in error.

3. Assistance during deployment

• During deployment, the development team has to be available on call to assist operations team in case of any issues.

4. QA – limited control

• QA team does not have much control as the package creation and sharing is done by the development team. Therefore, QA cannot assure that the verified package is getting deployed to higher environments.

DevOps (Development and Operations) evolved to overcome the limitations of the above process and to speed up the overall delivery time. One of the key factors for the quick adoption of the DevOps strategy is the emergence of Agile development methodology which required deployment of production ready code in every 2 weeks.

The Change in Approach

The term DevOps means “Development and Operations” i.e. both development and operations happen simultaneously, instead of happening is silos. The key features of DevOps are Continuous Integration and Continuous Delivery (CI / CD).

Continuous Integration means that the completed and unit tested code is integrated to the repository continuously, while verifying the quality and completeness of the integrated code by performing build, test case execution, code quality analysis, etc. Continuous Delivery means that the merged code is continuously deployed to the environment automatically at pre-defined periods. The CI/CD pipelines ensure that quality code is available in the repository all the time and that the latest available code is deployed to the testing environment. The branching and merging strategy can also, be integrated with the DevOps strategy so that the merging of code to higher environments can be performed via the continuous integration tools itself.

Following are the key steps for implementing a DevOps process:

• Defining a branching strategy
• Defining the environments
• Setting up continuous integration tool
• Configuring the continuous integration
• Configuring continuous deployment
• Configuring branch management

Defining a Branching Strategy

For implementing the DevOps process, first the whole team (Development, QA, Product Owner and Scrum Master) discusses and finalizes a branching strategy. The branching strategy and the environments are closely coupled and can be mapped to each other respectively, in most cases. The basic branching strategy for a typical project consists of three branches: Develop, Release, and Master.

1.Develop

All development activities take place in this branch. Ideally, a separate branch will have to be created for each user story and it needs to be reviewed when merged to Develop branch.

2.Release

This is the branch where the testing/QA is performed. A versioning strategy is followed to name the tags in the Release branch. Semantic versioning is the commonly used versioning convention. Once the changes are merged to Release branch, a tagged version is created which will be used for further deployment and testing.

3.Master

Once the code is deployed to production, it is merged from the tagged release branch to the Master branch.

Figure 2 : Defining a Branching Strategy

Defining the Environments

The environments usually, complement the branches. Corresponding to each branch, there will be an environment where it will be deployed.

1. Development/integration environment

Applications are deployed from the Develop branch. In some scenarios, the developers’ local machine is considered as the development environment and a named server based environment is used as an integration environment, where all the different components can be integrated and tested. In both cases, the code from Develop branch is used.

2. QA/test environment

The versioned application from the Release branch is deployed here. The QA team performs testing in this environment.

3. Production environment

The code once merged to Master branch is the production code. The production environment always runs the application in the latest Master branch.

There can be other environments, depending on the need, like staging environment, performance test environment etc. that can have different branches or can connect to existing branches.

 

Figure 3 : Defining the Environments

Setting Up Continuous Integration Tools

Jenkins is the most popular and commonly used continuous integration tool across industry. It can integrate with multiple source repositories like Git, SVN, etc and also, has plugins for integrating with variety of systems/tools. Jenkins can be installed in a standalone machine or can be installed on cloud instance. Alternatively, there are Jenkins services available which can be directly used as containers where in Jenkins image can be deployed.

Once installed, the Jenkins UI can be accessed over the browser and all the configurations can be performed via the UI.

Configuring the Continuous Integration

There are many different ways to configure Jenkins to perform continuous integration. Below are some of the key methods:

• Creating job to checking out from source repository (SVN, GIT).
• Trigger the build job so as to trigger build when code is committed.
• Invoke source code analysis – for static code analysis (SonarQube).
• Invoke unit test and integration testing – for test verification.
• Invoke coverage tool for coverage report.
• Create the deployable deliverable (jar, war) and add to repository.

Below are the key steps for performing continuous integration.

• Creating separate job for deploying to an environment.
• Configuring periodic jobs for deploying to an environment.

Jenkins can be configured to deploy applications to servers directly such as weblogic or can also, be configured to deploy to cloud-based instances [Azure, AWS].

Configuring Branch Management

Branch management is an optional feature of DevOps. It helps to avoid issues like developer directly committing code to QA/ Release branches for fixing bugs identified in QA, thereby, introducing more bugs. Branch management via DevOps will allow much better process and help to avoid bugs related to commit, merge, etc.

A simple branch management process is described below:

Developer creates branch for his own user story.

• Once development is complete, the developer creates a pull request to merge the code to Develop branch.
• Team lead performs code review and then merges the code to Develop branch.
• Automated build is triggered that verifies that the committed code is as per standard and no features are broken.
• Once all features for that sprint is committed to Develop branch, QA triggers the Release branch job in Jenkins, which will merge the code in Develop branch to the Release branch and a tagged version will be created. (Ex – release_1.2.0).
• QA now triggers the deployment of the above tagged branch to the QA/test environment.
• If bugs are identified in QA, then the developer takes a copy of the Release branch and creates a new branch for the defect. The defect fixes are then committed. This will be merged to the Release branch and a new tagged version will be created (release_1.2.0_1). This is also, manually merged to Develop branch by developer.
• Once the QA testing is complete, the tagged version is deployed to production and the code merged to Master.

Following are the advantages of using this approach.

• Developers will have ‘write’ access only to Develop branch and ‘read only’ access to other branches, thereby, limiting issues related to late commits.
• QA will be responsible for merging code to QA/Release branch and for deploying code to QA environment, thereby avoiding non-tested code getting released to customers.
• Code commit always follow the Develop – Release – Master process thereby, avoiding conflicts in Release/Master branches.

Conclusion