I'd like to share how we used pipelines in my own company to manage the growing complexity of our development so you can apply some of the same lessons in your organization. With the increase in the number of our development branches, we run into problems maintaining the quality of the production environment. Some of the things I wanted to do were:
- Start writing Docker images so we can reuse the production environment outside of a DevOps team
- Make sure we're using the same packages across all development branches
- Use tools to quickly stop and start services to serve different projects with the same code
There are lots of other tools to use too:
- Logfiles and logs for developers
- Logs and server audit tools to check changes and deploy
- Monitoring (Audience Manager, Sysdig)
- Monitoring (SQLi Guard, Splunk, Splunk DB)
- Monitoring (Puppet for managing groups of machines)
- Monitoring (Redis)
Tools like DevOps Automation, OpsGenie, and Nagios (many more are out there) can help you automate the various stages of your DevOps process and make your life easier, as well as the rest of the DevOps team.
But we don't want to make ourselves feel better. We want to deliver value and make our users happy.
We're trying to become a Zero Day Release organization (which sounds funny, but if you think about it, it makes a lot of sense for a DevOps organization):
You don't get more bugs when a new code change is pushed. You'll get the latest version of your code with the proper bug fixes and security fixes. You can use automation to ensure that you won't ever have a bad release when you push code. Even if you use the best developers, you can't guarantee that they'll always write the correct code. But if you can always predict exactly which code will break, you can fix your release early. The only way you can do this is to build in regression testing, build in testing early, and automate regression testing.
If you want to become a Zero Day Release organization, you can't use DevOps just to deploy more and more software. You'll need to learn to use tools like continuous integration (CI), integration tests, instrumentation, leak testing, unit testing, and other tools to guarantee quality and have your software ready for use right when you want to use it.
Other essential parts of our DevOps culture are that we don't accept promises that we can't keep. We push back on the quality assurance people or build in a complex way to catch problems. That includes many things:
- To not release a software update that doesn't meet our standards
- To not open service for people that don't know how to use it
- To not release a new product version in a way that makes it easier to break it
- To not remove a product or service that does not meet our customer expectations
It's a simple philosophy: release code when it's ready, not when a release engineer or product manager decides. We believe it's the right way, and it gives us the best chance to meet the demands of our users. We also think that other teams benefit from this when the product grows beyond the scope of one person.
Continuous Delivery and Continuous Deployment
I don't like the word "continuous delivery" (CD). Why does the name CD have to be so continuous? People often ask, "Well, when I think of continuous delivery, what am I doing?"
We use a few different acronyms when we talk about continuous delivery:
- CD: Continuous Deployment
- CBT: Continuous Testing
- CI: Continuous Integration
- CDN: Continuous Delivery Network
- (Sometimes) CDP: Continuous Delivery
These acronyms have different roles, responsibilities, and definitions, so they're not interchangeable.
I also don't like the term "continuous deployment." It's just another way of saying "Continuous Delivery" and often carries with it a set of preconceptions. Common ones are that you're all ready for release after you've deployed, so it's a continuous delivery, not a continuous deployment. Or that it happens 24/7.
DevOps, Continuous Delivery, and Continuous Deployment are just excellent ways of saying the same thing: be ready for deployment when the product/service is ready, not when the release engineer decides. You should deploy in any schedule or mode that makes the product/service ready. You should be able to push your changes to the target cloud, push your changes to a development branch, or push your changes to a staging branch, and then go to production.
If something goes wrong, just fix it. If the service stops working, try to figure out why and fix it. If you have a problem with your CI, check it. If your test suite fails, fix it. If you don't like how your integration tests are running, try to fix them. If your machine fails to connect to the server, try to figure out why and fix it.
When you fail, try to fix it. Don't expect someone else to fix it. Don't think that someone else will fix it if it's important. When you fail, fix it.
I like the term "continuous deployment." It's a word that clarifies that deployment happens continuously, whether the service is online or offline. The user should not see a slow or offline service.
In a sense, you are always deployed when you deliver software. All you're trying to do is keep the right people up-to-date and ensure that new customers get access to the latest version. This objective requires different skills than a release (once a month). Your deployment needs to be as reliable as a release. And as tricky as a release. There are always problems.
You can deploy by the hour, by the week, or by the day. Sometimes you have lots of people around to help. Sometimes it's just you. You just need to ensure that you have the skills and resources to make the correct deployments. You should only deploy when the code and the environment are ready.
And even if your deployment is never done, that doesn't mean that you're not deploying. You could deploy every hour and have lots of people helping you. It just means that the deployment is not done yet. You have to go back and make sure that the service works. You have to check the environment. You have to test it. And then, after it's been deployed, you have to make sure it works.
It's just the time elapsed from your first test to when you deploy. You're testing. You're debugging. You're fixing.
You shouldn't see any difference between your traditional software release and continuous deployment.
Eliminate: Deliver and Measure
We should have no notion of releases and deployments and other kinds of temporary improvements. Most things we ship today don't change. They stay the same all the time.
Because, for the most part, we measure things. If we measure what works, we can fix things. If we don't measure what's working, we can't fix it.
You don't need to ship more of the same thing. The next release could be only a slight improvement. We don't need to ship a big release every month.
You're in this mode of continuous delivery because you care about the quality of the software that you ship. So you want to fix bugs early, so you can get customers using it. So you can get more developers using it.
Enroll in DevOps Engineer Masters Program and learn to work on tools like Docker, Git, Jenkins, Puppet & Nagios.
In summary, be prepared for failure. And be open to change and to different ways of doing things. Build-in tests early and use tools to find the things that break. And be brave. You will fail, and you won't always get it right the first time. But you'll be much more likely to get it right if you do things in the right way.
That's how you get to continuous delivery.
Simplilearn offers two multi-course programs in DevOps. Our DevOps Engineer Master’s Program provides you key skills in DevOps, including certification in Docker and Kubernetes. The Post Graduate Program in DevOps in collaboration with Caltech CTME provides more comprehensive training in advanced DevOps skills.