This is a post by Rajesh Yarlagadda and David Vidal, our test engineers.

This post is the third post in our series covering how we approach continuous delivery, covering our approach to testing.

Testing is an essential component of each and every phase of the mygov.scot development process, with quality being “baked in” to the product at every stage. This blog post provides an overview of how we approach testing, and some of the tools we use. Feel free to get in touch if you’d like to know a bit more about a particular technology or our reasons behind using it.

After creating a new feature to meet a particular user need, our developers submit changes to our version control system – this can include source code, environment modifications and configuration files.

Following a commit to the version control system, the continuous integration server will do the following:

• Trigger a build
• Run automated unit tests
• Perform sonar code quality check
• Automatically deploy the successful build to a development environment

Continue reading →

This is a post by Gordon Clark, one of our Infrastructure Engineers, and Jono Ellis, our Social Media Manager.

This post is the second post in our series covering how we approach continuous delivery, covering the tools that we use.

We are not precious about our servers, they are just tools for a defined purpose and when they’ve reached the end of their usefulness we can get rid of them. They are disposable. This is something you can only do with cloud computing – we don’t own a physical server anywhere; instead we create and configure servers from our cloud providers. The benefits are the speed and agility that this setup offers us. As an example, in order to test a proof of concept we could spin up several servers to do a task, gather whatever stats we needed, and then we could destroy them. With no physical server needing to be purchased, set up, powered, etc. there are significant environmental benefits to this approach.

Tools

We use several tools in order to facilitate our server configuration management:

• Puppet – provisioning and configuration of hosts (we use Puppet in a masterless way);
• Packer – definition and creation of our base image;
• Base images – assignment of a role and an environment (based on the rules for the role a host is given, the host will self-provision. This means that the host gathers packages and configuration from a known artifact repository such as Debian);
• Vagrant – deploying Puppet runs locally for testing purposes.

Roles and Profiles

The role a server is provided with equates to a business need i.e. a web server, an application or a database server.

Each role (e.g. web server) has an accompanying module in Puppet. This Puppet module in turn contains one or more technology profiles, which are also described in Puppet. Each host can only be assigned a single role but each role can contain multiple profiles (which is turn can contain multiple modules). The modules are directly related to an individual technology, e.g. NGINX, Java or PostgreSQL. The diagram below shows how the hierarchy is arranged. It should be noted that the modules are generic, they can be used in any environment and do not contain any configuration related to an individual host. The configuration data is abstracted to Hiera.

Continue reading →

This is a post by Gordon Clark, one of our Infrastructure Engineers, and Jono Ellis, our Social Media Manager.

Over the next few weeks, we have a series of blog posts coming up which cover our approach to continuous delivery. This first post is a high-level overview of our work, mostly written for our readers who work in government, the individual posts will be more detailed and will be aimed at infrastructure engineers, developers and testers. We’d love to hear your comments and questions, so let us know either at the comments section below or via Twitter.

Over the coming weeks we aim to cover:

• Zero downtime releases;
• Masterless Puppet and Declarative provisioning;
• Virtual infrastructure;
• Definitive Media Library in the cloud;
• Monitoring;
• Build pipeline;
• Test automation.

The mygov.scot site is built from the ground up around the needs of the user. To find out if we’re building the right thing we want to be able to quickly get our work in front of our users – this ensures that we receive feedback early and often, and allows us to change direction accordingly. We also want to ensure that we are delivering consistently, and with quality. The model that we have in place to control how we work is called continuous delivery.

Continue reading →