Academy on Example Academy

Characteristics of the Pre-DevOps Era

Mon, 01 Jan 0001 00:00:00 +0000

Key Characteristics of the Pre-DevOps Era 🔗

The pre-DevOps era in software development was a starkly different landscape than what we see today. It was a time of siloed teams, rigid methodologies, and a lot of manual work, which often led to slow and unreliable software delivery.

Waterfall Model The dominant methodology was the Waterfall model, a linear approach where each stage had to be completed before moving on to the next. This made it difficult to adapt to changes and respond quickly to new requirements.
Siloed Teams Developers, testers, and operations teams worked independently, often with little communication or collaboration. This created a “throw it over the wall” mentality, where each team blamed the other for problems.
Manual Processes Most tasks, from testing to deployment, were done manually. This was time-consuming and error-prone, leading to delays and inconsistencies.
Limited Automation There were few tools available to automate repetitive tasks, making it difficult to scale software development.
Slow and Unreliable Delivery Releases were infrequent and often buggy, causing frustration for both developers and users. Next, let’s take a look at some specific examples of the challenges faced in the pre-DevOps era.

CI/CD Overview

Mon, 01 Jan 0001 00:00:00 +0000

Continuous Delivery 🔗

Continuous Delivery (CD) is a software development practice that aims to automate the process of releasing software changes to production environments in a frequent and reliable manner. It builds upon the concept of Continuous Integration (CI) by extending the automated pipeline to include deployment and release processes. In continuous delivery, the goal is to have software in a state where it can be released to production at any given time.

DevOps and Site Reliability Engineering

Mon, 01 Jan 0001 00:00:00 +0000

DevOps (Development + Operations) is a set of practices, principles, and cultural philosophies that aim to enhance collaboration and communication between software development (Dev) and IT operations (Ops) teams. The primary goal is to automate and streamline the processes of software delivery and infrastructure changes, fostering a culture of continuous improvement and faster, more reliable releases.

Site Reliability Engineering, or SRE, is a discipline that incorporates aspects of software engineering and applies them to infrastructure and operations problems. It was developed at Google to create scalable and highly reliable software systems.

Emergence of DevOps

Mon, 01 Jan 0001 00:00:00 +0000

The DevOps Era 🔗

DevOps software development methodology that emphasizes collaboration and communication between development and operations teams, didn’t appear overnight. Its roots can be traced back to several contributing factors and key turning points.

Early Influences 🔗

Agile movement: The Agile movement, emerging in the 1990s and formally established in 2001, emphasized iterative development and collaboration, paving the way for DevOps’ focus on faster delivery and team integration.
Automation: The rise of automation tools throughout the software development lifecycle (SDLC) simplified manual tasks, allowing teams to focus on higher-level activities.
Cloud computing: The emergence of cloud platforms provided a scalable and flexible environment for development and deployment, further facilitating collaboration and faster delivery.

Formative Years (2007-2009) 🔗

Patrick Debois: In 2007, frustrated with the traditional silos between development and operations, Patrick Debois, a Belgian consultant and project manager, began advocating for closer collaboration. He coined the term “DevOps” in 2009, which resonated with the growing desire for a more integrated approach to software development. To learn more, you can read “The Origins of DevOps: What’s in a Name?” article, by Steve Mezak or watch “The (Short) History of DevOps” video.
Agile Infrastructure: Around the same time, the “Agile Infrastructure” movement emerged, focusing on applying Agile principles to infrastructure management, aligning it with development practices.

Gaining Momentum (2010s) 🔗

Continuous Integration and Continuous Delivery (CI/CD): The adoption of CI/CD practices became instrumental in accelerating software delivery and improving feedback loops between development and operations.
DevOps tools and platforms: A surge of specialized DevOps tools and platforms emerged, automating tasks and providing shared visibility throughout the SDLC.
Increased awareness and adoption: DevOps became a popular topic in conferences and publications, leading to increased awareness and adoption across various industries.

Continuous Evolution (Present Day) 🔗

DevOps continues to evolve: As technology and business needs change, new DevOps practices and tools emerge, focusing on areas like security, compliance, and automation in cloud environments.
Shift towards DevOps culture: The focus expands beyond tools and processes to create a collaborative culture that fosters communication, shared responsibility, and continuous improvement.

Infrastructure as Code

Mon, 01 Jan 0001 00:00:00 +0000

Overview 🔗

Infrastructure as Code (IaC) is a revolutionary approach in the space of software development and IT operations, transforming the way we build, manage, and scale infrastructure. At its core, IaC involves describing and provisioning infrastructure elements through machine-readable script files, treating infrastructure in a manner similar to software code.

In traditional setups, infrastructure deployment was a manual and often error-prone process. With IaC, this paradigm shifts towards automation, enabling developers to define infrastructure configurations using high-level programming languages. One of the key advantages is the ability to version control the infrastructure code, ensuring traceability, reproducibility, and easy collaboration across development teams.

Lab 1. Installing OpenTofu

Mon, 01 Jan 0001 00:00:00 +0000

Overview 🔗

In this hands-on activity, we will learn how to install OpenTofu and verify the installation.

Installing OpenTofu 🔗

Linux Environment Setup 🔗

While this section presents various setup options, we will install OpenTofu on an Ubuntu 22.04 host. Depending on the operating system and the installation method, there are several ways to install OpenTofu. The most convenient is to use the installation script provided by the official OpenTofu documentation.

Lab Overview

Mon, 01 Jan 0001 00:00:00 +0000

In this lab, you will gain hands-on experience installing the Jenkins Server using Docker containers. The Jenkins container image is a stripped down version of Jenkins and does not contain all the plugins. We will understand how to set up Jenkins in a matter of a few minutes and create a simple job.

Lab Overview

Mon, 01 Jan 0001 00:00:00 +0000

Overview 🔗

Prometheus and Grafana are two powerful open source tools widely used in the monitoring and visualization of metrics in cloud native environments. They serve complementary roles: Prometheus is primarily used for metrics collection and alerting, while Grafana is leveraged for visualizing the data collected by Prometheus. Let’s explore both tools, including how they work together to provide a comprehensive monitoring solution.

Prerequisites 🔗

Ensure you have an Ubuntu 20.04 host. You can follow the previous lab to provision a virtual machine for yourself.

Lab Overview

Mon, 01 Jan 0001 00:00:00 +0000

This lab provides practical experience installing Docker and Podman and setting up a Kubernetes cluster using kind. Once we have installed the container engine, we will understand the most commonly used container lifecycle operations and move on to build a container image using Dockerfile. We will uninstall Docker and install Podman to understand the Docker alternate tool. While there are many ways in which we can set up a Kubernetes cluster, we will use a simple method of setting up the cluster using kind.

Overview

Mon, 01 Jan 0001 00:00:00 +0000

Chapter Overview 🔗

Managing IT infrastructure can be complicated, but Infrastructure as Code (IaC) makes it easier to define infrastructure using code. In this chapter, you’ll learn what IaC is, why it’s important, and how it helps manage infrastructure faster, more reliably, and more easily scale. You’ll also explore two main ways to write IaC: declarative, where you describe the desired outcome, and imperative, where you list the steps to get there.

Overview

Mon, 01 Jan 0001 00:00:00 +0000

In this lab, you will gain hands-on experience installing Terraform - an infrastructure as code tool that enables you to safely and predictably provision and manage infrastructure in any cloud. We will provision a Google Compute Engine using Terraform and then move on to implementing OpenTofu, which is a fork of Terraform that is open source, community-driven, and managed by the Linux Foundation.

Pre-Requisites 🔗

Ensure you have an Ubuntu 20.04 host. If not, then provision a virtual machine from Google Cloud Platform as per the lab in chapter 3.

Overview

Mon, 01 Jan 0001 00:00:00 +0000

This lab provides practical experience of creating a Google Cloud Compute Engine VM. There are different public cloud providers like AWS, Azure, Digital Ocean, etc. In our lab we will use GCP. Creating and consuming the infrastructure is similar across different cloud providers with a few vendor-specific differences. In this exercise we will learn how to create a Google Cloud Compute Engine (Virtual Machine).

Pre-Requisites 🔗

Ensure you have a Google Cloud account. If not, create an account on Google Cloud; new customers get 300$ free credits to evaluate, run, test and deploy workloads.

What Are Containers?

Mon, 01 Jan 0001 00:00:00 +0000

Containers have revolutionized Application Packaging and Deployment. In today’s fast-paced world, the ability to quickly and efficiently deploy software applications is critical for businesses of all sizes. Containers have emerged as a revolutionary technology that addresses this need by providing a lightweight, portable, and isolated environment for applications to run.

Imagine a container as a standardized box that holds everything your application needs to run - its code, libraries, runtime environment, and system configuration. Unlike virtual machines, which virtualize the entire hardware stack, containers share the underlying operating system (OS) with other containers, making them much more lightweight and efficient.

What Is Observability?

Mon, 01 Jan 0001 00:00:00 +0000

Observability is the ability to understand the internal state of a system based on its external outputs. This means being able to infer what is happening inside the system based on the information it produces, such as logs, metrics, and traces.

Observability is becoming increasingly important in the modern world of software development, where systems are becoming ever more complex and distributed. With traditional monitoring approaches, it can be difficult to understand what is happening inside a system and why it is behaving in a certain way. Observability provides a way to overcome this challenge by giving us a more holistic view of the system.

What is Site Reliability Engineering (SRE)?

Mon, 01 Jan 0001 00:00:00 +0000

Overview 🔗

Site Reliability Engineering (SRE) is an approach to running large-scale, reliable services. Google is widely credited with formalizing and popularizing the term “SRE” and has since been adopted by many other tech companies. SRE blends aspects of software engineering with traditional IT and focuses on creating scalable and highly reliable software systems. The primary focus of SRE is to improve the reliability of services. Reliability is often measured through service level indicators (SLIs), service level objectives (SLOs), and service level agreements (SLAs). These metrics help define the performance and availability levels expected from a service.

What Is the Cloud?

Mon, 01 Jan 0001 00:00:00 +0000

Overview 🔗

Picture the sky as a boundless expanse where you can store your digital dreams, access them from anywhere, and scale your ambitions limitlessly. That, in essence, is the magic of the cloud.

The cloud is not an elusive fortress in the sky; it’s a virtual space, a digital playground where data finds its home. Instead of hoarding information on your personal devices, the cloud allows you to offload it to powerful servers that float, metaphorically, in this digital sky. These servers are not tied to a physical location; they are dispersed worldwide, forming a network that operates seamlessly.

Challenges: Example 1: Sony PlayStation Network Outage (2011)

Mon, 01 Jan 0001 00:00:00 +0000

PSN Outage 🔗

In April 2011, the Sony PlayStation Network (PSN) experienced a massive outage that lasted for 23 days and affected over 100 million users. This incident is considered a classic example of the challenges faced in the pre-DevOps era and serves as a cautionary tale for organizations transitioning to modern development practices.

Pre-DevOps Challenges Contributing to the PSN Outage 🔗

Silos and Communication Gaps: The development and operations teams at Sony worked in separate silos with limited communication and collaboration. This led to a lack of understanding of each other’s work and challenges, making it difficult to respond effectively to the evolving situation during the outage.
Manual and Slow Processes: Deployments and infrastructure changes were performed manually, requiring significant time and effort. This slowness hampered Sony’s ability to quickly assess the situation and implement necessary fixes.
Limited Scalability and Flexibility: The PSN’s infrastructure was not designed to handle the surging demand caused by the attack, leading to widespread outages and service disruptions.
Lack of Visibility and Tracking: Sony lacked effective monitoring tools to identify and diagnose the source of the outage promptly. This delayed the response time and made it difficult to determine the full scope of the attack.
Culture of Blame and Finger-Pointing: The siloed environment and lack of communication led to blame and finger-pointing between different teams, hindering collaboration and problem-solving efforts.

Consequences of the PSN Outage 🔗

Financial Losses: Sony estimated the outage cost the company approximately $170 million in lost revenue and legal settlements.
Reputation Damage: The incident severely damaged Sony’s reputation and eroded user trust in the PSN platform.
Customer Frustration: Millions of users were frustrated by the prolonged outage and lack of information from Sony.

Lessons Learned from the PSN Outage 🔗

Here are some of the lessons learned from the PSN outage:

CI/CD - Why Should You Bother?

Mon, 01 Jan 0001 00:00:00 +0000

Why CI/CD Matters? 🔗

In the ever-evolving landscape of software development, the adoption of Continuous Integration and Continuous Delivery (CI/CD) has become not just a trend but a fundamental necessity. This chapter aims to provide a comprehensive understanding of why developers and engineers must prioritize and embrace CI/CD practices.

The Need for Speed & Efficiency: One of the primary reasons to incorporate CI/CD is the need for rapid and efficient software delivery. In a world where time-to-market is a critical success factor, developers face immense pressure to deliver high-quality code quickly. CI/CD automates and streamlines the entire development pipeline, from code integration to deployment, ensuring a swift and efficient release process.

Infrastructure as Code (IaC)

Mon, 01 Jan 0001 00:00:00 +0000

Infrastructure Provisioning: The Manual Way 🔗

Applications fuel the growth and efficiency of modern businesses. They depend on infrastructure—such as compute, storage, and networking—to run effectively. Traditionally, system administrators and IT professionals managed this infrastructure manually, physically setting up servers, network devices, and other hardware components needed for business applications.

In the early days, every business application required dedicated physical hardware. This meant deploying and configuring individual servers for each application—a time-intensive and resource-heavy process. Virtualization revolutionized this approach by allowing multiple virtual machines (VMs) to run on a single physical server using hypervisors, making better use of underutilized hardware. Despite this innovation, infrastructure provisioning and configuration were still mostly done manually, which brought its own set of challenges.

Install Docker

Mon, 01 Jan 0001 00:00:00 +0000

In this exercise, we will install Docker and understand the most commonly used container lifecycle commands and then build an image using Dockerfile.

We can install Docker using the Docker’s apt repository, or install it manually, or use a convenient script provided by Docker. For simplicity’s sake, we will use the below method.

Install Docker:

curl -fsSL https://get.docker.com/ | sh

Enable Docker to start on boot:

sudo systemctl enable --now docker

Check that the Docker service is running:

sudo systemctl status docker

Add current user to the docker group:

sudo usermod -aG docker $USER

Refresh shell session (by exiting & logging in again).
Verify the Docker installation:

docker ps

Install Terraform

Mon, 01 Jan 0001 00:00:00 +0000

In this exercise, we will install Terraform on our Ubuntu host.

Download the public signing key for the package repositories. To get started, execute the following commands:

wget -O- https://apt.releases.hashicorp.com/gpg | sudo gpg --dearmor -o
/usr/share/keyrings/hashicorp-archive-keyring.gpg

Add the appropriate apt repository:

echo "deb [signed-by=/usr/share/keyrings/hashicorp-archive-keyring.gpg]
https://apt.releases.hashicorp.com $(lsb_release -cs) main" | sudo tee
/etc/apt/sources.list.d/hashicorp.list

Update the apt package index and install Terraform:

sudo apt update && sudo apt install terraform

Verify the Terraform installation.

terraform --version
Terraform v1.7.4
on linux_amd64

Key Factors Contributing to the Emergence of DevOps

Mon, 01 Jan 0001 00:00:00 +0000

Key Factors 🔗

Need for Faster Software Delivery: Increased competition and evolving customer expectations drove the need for faster and more frequent software releases.
Inefficiencies of Siloed Teams: Traditional separation between development and operations led to communication gaps, delays, and finger-pointing.
Rise of Automation Tools: Automation enabled teams to focus on higher-level activities and eliminate manual work, improving efficiency and accuracy.
Cloud Computing: Cloud platforms offered scalability, flexibility, and access to shared resources, facilitating collaboration and faster deployments.

Lab 2. Learning the Basics of OpenTofu

Mon, 01 Jan 0001 00:00:00 +0000

Overview 🔗

In this lab, you will gain hands-on experience with the basics of the OpenTofu command line and be introduced to the OpenTofu configuration language, which specifies the resources OpenTofu would create and the providers it would use.

Prerequisites 🔗

You will need OpenTofu installed as per Lab 1.

Initialize and Create a Local Resource 🔗

In this exercise, we will write a basic OpenTofu code to create a resource locally and understand the basic workflow to create any infrastructure in OpenTofu.

SRE versus DevOps

Mon, 01 Jan 0001 00:00:00 +0000

Characteristics of SRE 🔗

Focus on Reliability: SRE places a primary emphasis on ensuring the reliability and availability of services. SREs are specifically tasked with maintaining a high level of service reliability and meeting defined Service Level Objectives (SLOs).
Error Budgets: SRE introduces the concept of error budgets, which quantifies the allowable amount of downtime or errors within a specific timeframe. This allows for a balance between reliability and innovation, as long as the error budget is not exhausted.

The Difference Between Containers and Virtual Machines

Mon, 01 Jan 0001 00:00:00 +0000

Virtual Machines: Isolated and Versatile 🔗

Virtual machines provide a fully isolated virtual environment, complete with an operating system and all necessary dependencies. Each VM runs on a hypervisor, a layer that abstracts the physical hardware and allows multiple VMs to coexist on a single physical machine.

Here are some key characteristics of virtual machines:

Strong isolation

Each VM operates as an independent entity, with its own dedicated resources and a separate OS instance. This isolation makes VMs highly secure, as vulnerabilities within one VM do not impact others.

The Three Pillars of Observability

Mon, 01 Jan 0001 00:00:00 +0000

Logs 🔗

Logs are digital records of events that have occurred within a system. These records are sequential and time-stamped, offering a granular, time-stamped audit trail of activities. Logs can include a wide range of data, from the simple recording of user access to detailed error messages that software developers use to debug issues. In the context of observability, logs provide the narrative detail that complements the quantitative data from metrics and the flow information from traces. They enable developers and operations teams to drill down to the root cause of issues, understand system behavior over time, and maintain operational excellence.

Why Should You Bother?

Mon, 01 Jan 0001 00:00:00 +0000

Software Engineers 🔗

Efficiency and Consistency 🔗

Consider a scenario where a software engineer needs to set up a development environment with specific server configurations. With IaC, they can script these configurations, ensuring consistency across different environments, be it development, testing, or production.

Collaboration and Version Control 🔗

When multiple engineers collaborate on infrastructure changes, version control becomes crucial. IaC enables engineers to use tools like Git to manage changes, track history, and collaborate seamlessly, just like they do with application code.

Why Should You Bother?

Mon, 01 Jan 0001 00:00:00 +0000

Why Should You Bother? 🔗

Scaling Success with Flexibility Imagine you’re a software developer working on an e-commerce platform. During a major sale event, your application experiences a surge in traffic as customers rush to make purchases. Without cloud infrastructure, you would be limited by the server capacity you initially set up. If the traffic exceeds expectations, the website could crash, leading to lost sales and frustrated customers.

A Brief History of the Cloud

Mon, 01 Jan 0001 00:00:00 +0000

1. Precursors and Pioneers (1950s-1990s) 🔗

The roots of the cloud can be traced back to the 1950s, when mainframe computers laid the groundwork for centralized computing. However, it wasn’t until the 1990s that the term “cloud computing” began to take shape (To learn more, check out TechTarget’s “The history of Cloud Computing Explained”). Early pioneers like Salesforce and Amazon Web Services (AWS) began experimenting with the concept of delivering computing power and storage as a service.

Brief History of Containers

Mon, 01 Jan 0001 00:00:00 +0000

Chroot (1970s) 🔗

The concept of isolation dates back to the chroot system call in Unix operating systems, which was introduced in the 1970s. Chroot allows a process to have its own isolated view of the filesystem.

FreeBSD Jails (2000) 🔗

FreeBSD introduced the concept of jails, which provided lightweight virtualization by isolating processes, filesystems, and networking. This was an early form of containerization.

Categories of Infrastructure As Code Tools

Mon, 01 Jan 0001 00:00:00 +0000

Declarative IaC Tools 🔗

Declarative tools focus on describing the desired state of the infrastructure. Users specify what they want, and the tool ensures the system reaches that state.

Examples:

Terraform allows users to define infrastructure using a declarative configuration language. It supports various providers like AWS, Azure, and Google Cloud, enabling multi-cloud and hybrid cloud scenarios.
Pulumi provides Infrastructure as Code in multiple programming languages like Python, TypeScript, and Go. It allows integration with various cloud providers and offers flexibility.
SaltStack is an open source platform with YAML configuration files for IaC. It focuses on automation and offers centralized infrastructure management.

Configuration Management Tools 🔗

These tools focus on configuring and maintaining software applications on existing infrastructure. While not purely IaC, they play a crucial role in managing the software layer.

Challenges: Example 2: The Launch of healthcare.gov Website (2013)

Mon, 01 Jan 0001 00:00:00 +0000

The Launch of healthcare.gov Website 🔗

The launch of the United States Department of Health and Human Services’ healthcare.gov website in 2013 was plagued by technical problems, causing major delays and embarrassment for the government. This was due in part to the use of outdated technology and a lack of collaboration between development and operations teams.

The pre-DevOps era was a time of significant challenges for software development. However, it also laid the groundwork for the emergence of DevOps, which has revolutionized the way software is built and delivered.

DevOps and SRE: A Simplified Analogy

Mon, 01 Jan 0001 00:00:00 +0000

Overview 🔗

DevOps and SRE are two different disciplines of Software Engineering, but they are deeply interconnected and rely on each other for success. DevOps focuses on creating new features and improvements, while SRE focuses on ensuring the reliability and stability of the system. Both DevOps and SRE require strong collaboration and communication to achieve common goals. Anyone interested in a career in tech should consider exploring both DevOps and SRE to understand the full picture of software development and delivery.

Introducing OpenTofu

Mon, 01 Jan 0001 00:00:00 +0000

Introducing OpenTofu 🔗

According to its documentation,

OpenTofu is an infrastructure as code tool that lets you define both cloud and on-prem resources in human-readable configuration files that you can version, reuse, and share. You can then use a consistent workflow to provision and manage all of your infrastructure throughout its lifecycle. OpenTofu can manage low-level components like compute, storage, and networking resources, as well as high-level components like DNS entries and SaaS features.

Key Principles of DevOps

Mon, 01 Jan 0001 00:00:00 +0000

Key Principles of DevOps 🔗

Adhering to the key principles of DevOps will ensure a more successful development, delivery, and implementation of applications and services.

Collaboration: DevOps emphasizes breaking down silos between development and operations teams, encouraging shared responsibilities and improved communication.
Automation: Automation is a core tenet of DevOps, involving the use of tools to automate manual and repetitive tasks in the software development and delivery pipeline.

Lab 3. Creating a Virtual Machine using OpenTofu

Mon, 01 Jan 0001 00:00:00 +0000

Overview 🔗

In this lab, you will gain hands-on experience creating a virtual machine from code using OpenTofu. OpenTofu can also create infrastructure across various providers. In this exercise, we will use AWS and create an EC2 instance for demonstration purposes.

Prerequisites 🔗

OpenTofu installed as per Lab 1.
AWS account created - access and secret keys. Note: It is out of scope for this course to help you create an AWS account. Please refer to the official AWS docs (for example, How to create an account and How to configure user).

Creating an EC2 Virtual Machine with OpenTofu 🔗

Create a separate directory for storing your OpenTofu code:

mkdir newproject && cd newproject

Using the editor of your choice, create the file main.tf with the content below:

provider "aws" {
 region = "us-east-1"
 access_key = "xxxxxxxxxxxxxxxxxxxxxxx"
 secret_key = "xxxxxxxxxxxxxxxxxxxxxxxx"
}
resource "aws_instance" "firstvm" {
 ami = "ami-053b0d53c279acc90"
 instance_type = "t2.micro"
 subnet_id = "subnet-xxxxxxxxxx"
}

Note: Please replace the “access_key”, “secret_key” and “subnet_id” with your credentials and subnet ID.

Manage Container Lifecycle with Docker

Mon, 01 Jan 0001 00:00:00 +0000

Now that we have installed and verified Docker, let’s create our first container. This command pulls the hello-world image from Docker Hub and runs it as a container.

student@ubuntu:~$ sudo docker container run hello-world

Output:

Unable to find image 'hello-world:latest' locally
latest: Pulling from library/hello-world
c1ec31eb5944: Pull complete
Digest:
sha256:ac69084025c660510933cca701f615283cdbb3aa0963188770b54c31c8962493
Status: Downloaded newer image for hello-world:latest
Hello from Docker!
This message shows that your installation appears to be working correctly.
To generate this message, Docker took the following steps:
1. The Docker client contacted the Docker daemon.
2. The Docker daemon pulled the "hello-world" image from the Docker Hub.
(amd64)
3. The Docker daemon created a new container from that image which runs the
executable that produces the output you are currently reading.
4. The Docker daemon streamed that output to the Docker client, which sent
it
to your terminal.
To try something more ambitious, you can run an Ubuntu container with:
$ docker run -it ubuntu bash
Share images, automate workflows, and more with a free Docker ID:
https://hub.docker.com/
For more examples and ideas, visit:
https://docs.docker.com/get-started/

List the container running on our host, by executing the following command:

student@ubuntu:~$ sudo docker container ls

Output:

Measuring Reliability with SLIs

Mon, 01 Jan 0001 00:00:00 +0000

Service Level Indicators (SLIs) 🔗

SLIs are specific, quantitative metrics that define the performance and reliability of a service. They are typically expressed as a ratio, percentage, or a specific numerical value. SLIs represent the aspects of a service that are most critical to its users. Selecting the right SLIs is crucial. They should align with user expectations and business goals. For example, if users value a fast response time, latency might be a critical SLI.

Set up the GCP Service Account and Configure Google Cloud SDK

Mon, 01 Jan 0001 00:00:00 +0000

In this exercise, we will install Google Cloud SDK, configure Google Cloud account to create a service account and use the credentials to create infrastructure using Terraform.

Before you install the gcloud CLI, update the packages:

sudo apt-get update

Install the curl and apt-transport-https packages:

sudo apt-get install apt-transport-https ca-certificates gnupg curl

Import the Google Cloud public key:

curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo gpg --dearmor
-o /usr/share/keyrings/cloud.google.gpg

Add the gcloud CLI distribution URI as a package source:

echo "deb [signed-by=/usr/share/keyrings/cloud.google.gpg]
https://packages.cloud.google.com/apt cloud-sdk main" | sudo tee -a
/etc/apt/sources.list.d/google-cloud-sdk.list

Update and install the gcloud CLI:

sudo apt-get update && sudo apt-get install google-cloud-cli

After installing the SDK, initialize it by running the following command and follow the prompts:

gcloud init
Welcome! This command will take you through the configuration of gcloud.
Your current configuration has been set to: [default]
You can skip diagnostics next time by using the following flag:
gcloud init --skip-diagnostics
Network diagnostic detects and fixes local network connection issues.
Checking network connection...done.
Reachability Check passed.
Network diagnostic passed (1/1 checks passed).
Choose the account you would like to use to perform operations for this
configuration:
Follow the prompts and enter your project details accordingly.

Create a Service Account:

You’ll need to create a service account in your Google Cloud project. This service account will be used by applications and scripts to authenticate with Google Cloud APIs.

What Is Continuous Integration?

Mon, 01 Jan 0001 00:00:00 +0000

Continuous Integration (CI) 🔗

Continuous Integration (CI) is a pivotal methodology in software development, reshaping the way teams collaborate and deliver high-quality code. At its core, CI involves the regular merging of code changes from multiple contributors into a shared repository. This iterative process serves as a proactive measure to identify and rectify integration issues at an early stage, fostering a development environment characterized by stability and efficiency.

Why Is Observability Needed?

Mon, 01 Jan 0001 00:00:00 +0000

Why? 🔗

Observability is a crucial DevOps concept, playing a pivotal role in ensuring the reliability, performance, and overall success of software applications and services. It goes beyond traditional monitoring to offer a comprehensive, real-time insight into the health and behavior of systems. Here are several key reasons why observability is needed in DevOps.

Complexity of Modern Systems: With the rise of microservices architecture, cloud computing, and distributed systems, applications have become more complex. Observability is crucial for understanding and managing this complexity.

Benefits of DevOps

Mon, 01 Jan 0001 00:00:00 +0000

Benefits of DevOps Includes 🔗

Speed: DevOps enables faster software delivery by automating processes and eliminating manual handoffs between development and operations teams. By breaking down silos between teams, DevOps promotes a more agile and efficient approach to software development.
Rapid Delivery: DevOps allows organizations to release new features and updates quickly and reliably. Continuous integration, delivery, and deployment ensure that software is always in a deployable state and ready to be released.

Components of an Observability Stack

Mon, 01 Jan 0001 00:00:00 +0000

Key Components of an Observability Stack 🔗

An observability stack is a collection of tools and technologies that work together to provide insights into the health, performance, and behavior of your software systems. These insights help you identify and resolve issues quickly, optimize performance, and ensure a smooth user experience. A well-integrated observability stack is essential for maintaining high-performing and reliable systems in today’s complex digital environments. It empowers teams with the insights needed to make informed decisions and maintain optimal operational efficiency.

Create a Google Cloud Compute Engine Instance Using Terraform

Mon, 01 Jan 0001 00:00:00 +0000

Create a directory to store our Terraform code:

mkdir -p ~/code

Create a Terraform configuration file and save the file as main.tf with the following configuration:

provider "google" {
 project = "<ADD YOUR PROJECT ID HERE>"
 region = "us-central1"
 zone = "us-central1-a"
}

resource "google_compute_instance" "my_instance" {
 name = "my-instance"
 machine_type = "n1-standard-1"
 zone = "us-central1-a"

 boot_disk {
 initialize_params {
 image = "debian-cloud/debian-10"
 }
 }

 network_interface {
 network = "default"
 access_config {
 // Ephemeral IP
 }
 }
}

Note: Replace " ADD YOUR PROJECT ID HERE " with your Google Cloud project ID. You can also customize other parameters like the instance name, machine type, zone, and image.

Docker: What’s under the hood?

Mon, 01 Jan 0001 00:00:00 +0000

Docker 🔗

Docker is a platform that uses containerization technology to simplify the development, deployment, and scaling of applications. Under the hood, Docker relies on several key components and technologies to achieve its functionality.

Docker Engine 🔗

Docker Engine, often simply referred to as Docker, is the heart of Docker technology, providing the necessary functionality to create and manage container lifecycle. Its architecture allows for seamless development, shipment, and running of applications in containers. Docker Engine is modular in nature and consists of many modules which help in container lifecycle management:

Embracing Risks with SLOs and Error Budgets

Mon, 01 Jan 0001 00:00:00 +0000

SLOs and Error Budgets 🔗

In SRE, embracing risks is an inherent part of the approach, and it’s managed through the concepts of Service Level Objectives (SLOs) and error budgets. Let’s explore how SLOs and error budgets enable teams to balance innovation and reliability while embracing a certain level of risk.

Service Level Objectives (SLOs) 🔗

SLOs are specific, measurable targets that define the acceptable level of reliability for a service. They are expressed as a percentage or ratio and represent the agreed-upon performance level that a service should achieve.

Five Fundamental Features of an IaC Tool

Mon, 01 Jan 0001 00:00:00 +0000

Features of IaC Tools 🔗

In the dynamic landscape of modern software development, Infrastructure as Code (IaC) tools have become indispensable for automating and managing infrastructure. Let’s dive into five fundamental features that make these tools essential for developers, using straightforward scenarios and relatable examples.

Declarative Configuration 🔗

What is it?

Declarative configuration allows users to specify the desired state of their infrastructure without detailing the steps to reach that state. It’s like ordering a meal in a restaurant without instructing the chef on how to cook it.

Image Management with Docker

Mon, 01 Jan 0001 00:00:00 +0000

List the available images on our host. Notice when the images are not available on the host, they are by default downloaded from DockerHub.

student@ubuntu:~$ sudo docker image ls
REPOSITORY TAG IMAGE ID CREATED SIZE
ubuntu latest 174c8c134b2a 3 weeks ago 77.9MB
nginx latest d453dd892d93 2 months ago 187MB
hello-world latest d2c94e258dcb 8 months ago 13.3kB

If the image is already existing on the host, then creating a container is very quick. Let’s see how to pull an image from the registry:

student@ubuntu:~$ sudo docker image pull alpine
Using default tag: latest
latest: Pulling from library/alpine
661ff4d9561e: Pull complete
Digest:
sha256:51b67269f354137895d43f3b3d810bfacd3945438e94dc5ac55fdac340352f48
Status: Downloaded newer image for alpine:latest
docker.io/library/alpine:latest

By default, the Daemon is configured to fetch the image from the Docker Hub registry and from the user account library, where Docker stores all the publicly available official images and fetches the image with tag as latest. We can override the default behavior by specifying what we want.

student@ubuntu:~$ sudo docker image pull gcr.io/google-samples/hello-app:2.0
2.0: Pulling from google-samples/hello-app
07a64a71e011: Pull complete
fe5ca62666f0: Pull complete
b02a7525f878: Pull complete
1933f300df8c: Pull complete
36cc1f0ec872: Pull complete
Digest:
sha256:7104356ed4e3476a96a23b96f8d7c04dfa7a1881aa97d66a76217f6bc8a370d0
Status: Downloaded newer image for gcr.io/google-samples/hello-app:2.0
gcr.io/google-samples/hello-app:2.0

Let us build a simple image using Dockerfile. We will use the index.html file we created in an earlier step and copy it to our new image.

student@ubuntu:~$ cat > Dockerfile <<EOF
> FROM nginx
> COPY . /usr/share/nginx/html
> EXPOSE 80/tcp
> CMD ["nginx", "-g daemon off;"]
> EOF
student@ubuntu:~$ sudo docker image build . -t nginx:version1
[+] Building 0.4s (7/7) FINISHED
docker:default
=> [internal] load .dockerignore
0.1s
=> => transferring context: 2B
0.0s
=> [internal] load build definition from Dockerfile
0.1s
=> => transferring dockerfile: 123B
0.0s
=> [internal] load metadata for docker.io/library/nginx:latest
0.0s
=> [internal] load build context
0.1s
=> => transferring context: 5.74kB
0.0s
=> [1/2] FROM docker.io/library/nginx
0.1s
=> [2/2] COPY . /usr/share/nginx/html
0.1s
=> exporting to image
0.0s
=> => exporting layers
0.0s
=> => writing image
sha256:69be93e604b157a2dda142b3444100e5e24013f2c29db16bd25c0c8de84c28b2
0.0s
=> => naming to docker.io/library/nginx:version1

Create a container using the image we built and access the application.

student@ubuntu:~$ sudo docker container run -dit --name webapp -p 8081:80
nginx:version1
8fd1b47bafda05c96faba7d46486ca90f4a828c221cb5fd689e819592b839c84
student@ubuntu:~$ curl localhost:8081
Welcome to Docker Training!!

Clean up the unused images on the host by executing the following command:

student@ubuntu:~$ sudo docker image prune -a
WARNING! This will remove all images without at least one container
associated to them.
Are you sure you want to continue? [y/N] y
Deleted Images:
untagged: alpine:latest
untagged:
alpine@sha256:51b67269f354137895d43f3b3d810bfacd3945438e94dc5ac55fdac340352f
48
deleted:
sha256:f8c20f8bbcb684055b4fea470fdd169c86e87786940b3262335b12ec3adef418
deleted:
sha256:5af4f8f59b764c64c6def53f52ada809fe38d528441d08d01c206dfb3fc3b691
untagged: gcr.io/google-samples/hello-app:2.0
<Output truncated>

There are additional image options; explore the sudo docker image –help command and try additional options.

OpenTofu: Installation, Workflow, and Building Blocks

Mon, 01 Jan 0001 00:00:00 +0000

Installing OpenTofu 🔗

OpenTofu is very easy to install and works on various Linux, Windows, and MacOS distributions. The installation method differs depending on your choice of operating system. The easiest is to use the installer script made available in the official documentation.

Download the Installer script:

curl -fsSL https://get.opentofu.org/install-opentofu.sh -o install-opentofu.sh

Grant execute permissions and review the script:

chmod +x install-opentofu.sh && less install-opentofu.sh

Install using the script:

./install-opentofu.sh --install-method standalone

How Does OpenTofu Work? 🔗

OpenTofu leverages application programming interfaces (APIs) to create and manage resources on cloud platforms and other services. Through providers, OpenTofu can interact with virtually any platform or service that has an accessible API.

Types of Clouds

Mon, 01 Jan 0001 00:00:00 +0000

Infrastructure as a Service (IaaS) 🔗

What it is: Imagine you’re building a house. IaaS is like renting the land and the basic utilities (electricity, water, etc.). You have the freedom to design and build your house the way you want.

For developers: With IaaS in the cloud, developers get the basic tools (servers, storage, and networking) to create and manage their own digital ‘houses’ (applications). It’s like having your own digital construction site.

What Is Continuous Delivery (CD)?

Mon, 01 Jan 0001 00:00:00 +0000

Continuous Delivery (CD) 🔗

Continuous Delivery is a software development practice that extends the principles of Continuous Integration to ensure that the codebase is always in a deployable state. The primary objective is to make software releases reliable, predictable, and sustainable. In a Continuous Delivery pipeline, automated testing, and deployment processes are integral components.

Key Characteristics 🔗

Automated Testing: Similar to Continuous Integration, Continuous Delivery places a strong emphasis on automated testing. This includes unit tests, integration tests, and other forms of testing that validate the correctness and functionality of the codebase.

Cloud Deployment Models

Mon, 01 Jan 0001 00:00:00 +0000

Next, let’s delve into the different cloud deployment models:

Public Cloud

In the realm of public cloud, computing resources are provided by a third-party cloud service provider and are made accessible to the general public over the internet. Most prominent examples are the top three providers - Amazon Web Services (AWS), Google Cloud Platform (GCP) and Microsoft Azure, who offer a comprehensive public cloud platform offering a myriad of services such as virtual machines, databases, and AI capabilities. Other notable players are Alibaba Cloud, IBM Cloud, Oracle Cloud Infrastructure (OCI), and DigitalOcean who offer similar features.

Core Concepts of OpenTofu

Mon, 01 Jan 0001 00:00:00 +0000

Providers 🔗

OpenTofu uses providers, which are plugins enabling it to communicate with platforms like cloud services, SaaS applications, and APIs. To use OpenTofu, you must specify the providers your infrastructure needs. These providers are then downloaded and configured to work within your environment. Some may require additional details, such as the region of a cloud service or specific API endpoints, to function properly.

Providers allow OpenTofu to manage specific types of resources and data. OpenTofu cannot interact with infrastructure platforms without providers, as a provider implements every resource type. Most providers are designed to handle a particular platform, whether a cloud service or an on-premises system. Some offer utility functions like generating unique identifiers or handling small automation tasks.

Infrastructure as Code (IaC) with Meshery

Mon, 01 Jan 0001 00:00:00 +0000

Meshery as Infrastructure as Code (IaC) 🔗

Meshery is a cloud native management plane that enables Infrastructure as Code (IaC) through declarative configuration management. Using Meshery Designs, users can define, import, manage, and deploy infrastructure configurations in a structured and repeatable manner.

Meshery Designs: The Foundation of IaC 🔗

Meshery Designs are YAML-based configurations that serve as the core construct for managing infrastructure as code. They provide a declarative approach similar to Kubernetes manifests.

Observability in DevOps and SRE

Mon, 01 Jan 0001 00:00:00 +0000

Observability in DevOps 🔗

Continuous Monitoring:

Objective: DevOps emphasizes continuous integration, continuous delivery (CI/CD), and continuous monitoring. Observability is integrated into the monitoring phase to provide real-time insights into the health and performance of applications.
Role: DevOps teams use observability tools to monitor application metrics, logs, and traces, enabling them to detect and address issues quickly.

Feedback Loops:

Objective: DevOps practices involve shortening feedback loops to facilitate rapid iteration and improvement. Observability contributes to these feedback loops by providing actionable insights into the impact of changes on the system.
Role: Developers receive feedback on how code changes affect system behavior, helping them make informed decisions and improvements.

Collaboration Across Teams:

Podman: An Alternative to Docker

Mon, 01 Jan 0001 00:00:00 +0000

Podman Overview 🔗

Podman (Pod Manager) is an open source tool for managing OCI (Open Container Initiative) containers and pods. Its architecture and approach differ in several key aspects from Docker, particularly in how it operates without a daemon. Podman is a powerful container management tool that can be considered a potential alternative to Docker Engine.

Key Features of Podman 🔗

Daemonless Architecture

Service Level Agreements (SLAs)

Mon, 01 Jan 0001 00:00:00 +0000

SLA 🔗

An SLA is a formal contract between a service provider and its customers, outlining the expected level of service. It defines the agreed-upon quality of service, including performance metrics, availability, and support expectations. In Site Reliability Engineering (SRE), a Service Level Agreement (SLA) plays a crucial role in managing expectations and ensuring a service meets user needs.

Benefits of SLAs 🔗

Clear Communication: SLAs establish a shared understanding between SREs, developers, and stakeholders regarding the expected performance of a service.
Improved Reliability: Having defined SLAs motivates SREs to proactively monitor and improve the system’s reliability to meet the agreed-upon targets.
Risk Management: Clear SLAs help manage customer expectations and define potential consequences for outages. This allows for proactive planning and mitigation strategies.

Real-world Example 🔗

Let’s take a look at an example of an SLA in SRE applied to an e-commerce platform.

Setting up OpenTofu

Mon, 01 Jan 0001 00:00:00 +0000

OpenTofu is a fork of Terraform that is open source, community-driven, and managed by the Linux Foundation. OpenTofu is an infrastructure as code tool that lets you define both cloud and on-prem resources in human-readable configuration files that you can version, reuse, and share. You can then use a consistent workflow to provision and manage all of your infrastructure throughout its lifecycle. OpenTofu can manage low-level components like compute, storage, and networking resources, as well as high-level components like DNS entries and SaaS features.

Tools in DevOps

Mon, 01 Jan 0001 00:00:00 +0000

DevOps Tools 🔗

There are numerous DevOps tools available that cater to different stages of the software development lifecycle. Let’s take a look at some popular tools across various categories.

Version Control 🔗

Git is widely used for version control, and helps track changes in source code during development.
Subversion (SVN) is a centralized version control system. It allows users to keep track of all changes made to files and directories in a repository. SVN is known for its simplicity and is often used in projects where a centralized, linear workflow is preferred.
Bitbucket is a cloud-based platform offering Git-based version control, code review, and CI/CD tools. It offers features like code hosting, issue tracking, and pull requests, making it popular for open source projects and collaboration.
GitHub is a popular cloud-based platform for version control, social coding, and project management.

Continuous Integration/Continuous Deployment (CI/CD) 🔗

Jenkins is an open source automation server that facilitates building, testing, and deploying code changes.
Travis CI is a CI/CD service that integrates with GitHub repositories for automated testing and deployment.
CircleCI is a cloud-based CI/CD platform supporting automation and parallel testing.

Configuration Management 🔗

Ansible is an open source automation tool that simplifies configuration management, application deployment, and task automation.
Chef enables infrastructure automation and configuration management using reusable scripts called recipes.
Puppet is a configuration management tool for automating the provisioning and management of infrastructure.

Containerization and Orchestration 🔗

Docker is a very popular platform for developing, shipping, and running applications in containers.
Podman is an open source container engine that offers a Docker-compatible command-line interface and runtime. It features image creation, management, and deployment, but lacks some advanced features like built-in orchestration.
LXC (Linux Containers) is a lightweight containerization technology built into the Linux kernel, offering efficient resource utilization and isolation. It is popular for its simplicity and portability, but lacks some advanced features of other tools.
Kubernetes is an open source container orchestration platform for automating the deployment, scaling, and management of containerized applications. It is considered the de facto orchestration platform for containers due to its popularity.
OpenShift is a container platform developed by Red Hat. It extends Kubernetes and provides additional features for enterprise applications, including source-to-image builds and developer-focused tools.
Apache Mesos is an open source cluster management and orchestration platform designed to simplify the management of distributed applications and resources in data centers or cloud environments.

Infrastructure as Code (IaC) 🔗

Meshery is a cloud native management plane that provides lifecycle, performance, and configuration management for cloud native infrastructure, including service meshes and Kubernetes.
Terraform is a tool for building, changing, and versioning infrastructure efficiently and safely.
AWS CloudFormation is Amazon’s IaC service for provisioning and managing AWS resources using templates.
OpenTofu is a fork of Terraform that is open source, community-driven, and managed by the Linux Foundation.

Continuous Monitoring 🔗

Prometheus is an open source monitoring and alerting toolkit designed for reliability and scalability.
Zabbix is a comprehensive monitoring solution for servers, networks, applications, and other IT infrastructure. It offers monitoring capabilities, alerting, and reporting features. It supports various monitoring protocols and provides extensive customization options.
Nagios is a mature and popular open source monitoring tool with a focus on server and network monitoring. It offers a flexible plugin system for extending its monitoring capabilities and features robust alerting and notification options.
OpenNMS is an open source network management system with built-in monitoring capabilities. It provides comprehensive network discovery, mapping, and monitoring features, and offers a web-based interface for managing and visualizing network performance.
Grafana is a visualization platform that integrates with various data sources, including Prometheus, for creating interactive and shareable dashboards.
Meshery is a cloud native management plane that provides performance analysis and monitoring for cloud native applications.

Collaboration and Communication 🔗

Slack is a team collaboration tool for communication and sharing updates.
Microsoft Teams is a communication and collaboration platform that integrates with other Microsoft tools.

Logging 🔗

ELK Stack (Elasticsearch, Logstash, Kibana) is a set of tools for log management, enabling search, analysis, and visualization of log data.
Fluentd is a lightweight and high-performance log collector and forwarder. It offers flexible configuration and supports various input plugins for collecting logs from different sources. It integrates seamlessly with other logging tools and platforms.
Splunk is a comprehensive logging platform offering log collection, indexing, search, analysis, and visualization. It provides powerful real-time analytics and AI-driven insights for troubleshooting and security.
Datadog is a comprehensive monitoring platform offering log management, metrics collection, and application performance monitoring functionalities.

Source Code Management (SCM) 🔗

Bitbucket is a Git repository management solution with features for code collaboration, continuous delivery, and more.
GitLab is a web-based Git repository manager with CI/CD, code review, and collaboration features.
GitHub is a very popular web-based platform that allows developers to store, track changes, and collaborate on code.

These tools, along with many others, form an ecosystem that supports the principles of DevOps, facilitating collaboration, automation, and efficiency throughout the software development lifecycle. It’s essential to choose tools based on your specific needs, the technologies you use, and your team’s preferences.

Uninstall Docker

Mon, 01 Jan 0001 00:00:00 +0000

Uninstall the Docker Engine, CLI, containerd, and Docker Compose packages:

student@ubuntu:~$ sudo apt-get purge docker-ce docker-ce-cli containerd.io
docker-buildx-plugin docker-compose-plugin docker-ce-rootless-extras
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following packages were automatically installed and are no longer
required:
libatasmart4 libblockdev-fs2 libblockdev-loop2 libblockdev-part-err2
libblockdev-part2 libblockdev-swap2 libblockdev-utils2 libblockdev2
libmbim-glib4 libmbim-proxy libmm-glib0 libnspr4
libnss3 libnuma1 libparted-fs-resize0 libqmi-glib5 libqmi-proxy
libudisks2-0 libxmlb2 pigz slirp4netns usb-modeswitch usb-modeswitch-data
Use 'sudo apt autoremove' to remove them.
The following packages will be REMOVED:
containerd.io* docker-buildx-plugin* docker-ce* docker-ce-cli*
docker-ce-rootless-extras* docker-compose-plugin*
0 upgraded, 0 newly installed, 6 to remove and 10 not upgraded.
After this operation, 410 MB disk space will be freed.
Do you want to continue? [Y/n] y
(Reading database ... 62643 files and directories currently installed.)
Removing docker-ce-rootless-extras (5:24.0.7-1~ubuntu.20.04~focal) ...
Removing docker-ce (5:24.0.7-1~ubuntu.20.04~focal) ...
<Output Trucncated>

Images, containers, volumes, or custom configuration files on your host aren’t automatically removed. To delete all images, containers, and volumes, run:

student@ubuntu:~$ sudo rm -rf /var/lib/docker
student@ubuntu:~$ sudo rm -rf /var/lib/containerd

What Is Continuous Deployment (CD)?

Mon, 01 Jan 0001 00:00:00 +0000

Continuous Deployment (CD) 🔗

Continuous Deployment takes the principles of Continuous Delivery a step further by automatically deploying every code change that passes automated testing directly to the production environment. The goal is to maximize efficiency and deliver new features or bug fixes to end-users as quickly as possible.

Key Characteristics 🔗

Automated Production Deployment: The hallmark of Continuous Deployment is the automatic deployment of code changes to the production environment once they pass automated tests. This minimizes manual intervention in the release process.

Challenges in Observability

Mon, 01 Jan 0001 00:00:00 +0000

What observability challenges can you expect? 🔗

Data Volume and Overhead 🔗

Challenge: Modern systems generate vast amounts of data, including logs, metrics, and traces. Managing and analyzing this volume of data can be overwhelming, leading to challenges in storage, processing, and the associated costs.

Solution: Implementing intelligent data sampling and aggregation techniques can help reduce the data volume without sacrificing essential insights.

Continuous Deployment and Continuous Delivery

Mon, 01 Jan 0001 00:00:00 +0000

Continuous Deployment (CD) and Continuous Delivery (CD) are practices closely related to Continuous Integration (CI) in the realm of software development. While they share some similarities, each has distinct characteristics that contribute to the overall goal of delivering high-quality software efficiently.

In summary, Continuous Delivery and Continuous Deployment are practices that extend the benefits of Continuous Integration. Continuous Delivery emphasizes a reliable and automated process up to the production environment, with human intervention required for the final release decision. On the other hand, Continuous Deployment takes automation further by automatically deploying code changes to production, providing immediate access to new features for end-users. The choice between Continuous Delivery and Continuous Deployment depends on factors such as the organization’s risk tolerance, regulatory requirements, and the need for human oversight in the release process.

Installing Podman

Mon, 01 Jan 0001 00:00:00 +0000

Let’s download the latest Podman binaries and extract them to set up Podman on our host. Note: If you are running Ubuntu 20.10 or higher, you can simply use the package manager to install Podman.

student@ubuntu:~$ curl -fsSL -o podman-linux-amd64.tar.gz \
>
https://github.com/mgoltzsche/podman-static/releases/latest/download/podmanlinux-amd64.tar.gz
student@ubuntu:~$ tar -xf podman-linux-amd64.tar.gz
student@ubuntu:~$ sudo cp -r podman-linux-amd64/usr podman-linux-amd64/etc /
student@ubuntu:~$ sudo ln -s /usr/bin/podman /usr/local/bin/podman

Verify the Podman installation:

student@ubuntu:~$ podman --version
podman version 4.8.2

To explore all the available options with Podman, execute:

student@ubuntu:~$ podman --help
Usage:
podman [options] [command]
Available Commands:
attach Attach to a running container
auto-update Auto update containers according to their auto-update policy
build Build an image using instructions from Containerfiles
commit Create new image based on the changed container
compose Run compose workloads via an external provider such as
docker-compose or podman-compose
container Manage containers
cp Copy files/folders between a container and the local
filesystem
create Create but do not start a container
diff Display the changes to the object's file system
events Show podman system events
exec Run a process in a running container
export Export container's filesystem contents as a tar archive
generate Generate structured data based on containers, pods or volumes
healthcheck Manage health checks on containers
help Help about any command
<Output Truncated>

List the existing container, by executing the following command:

student@ubuntu:~$ sudo podman container ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS
NAMES

Create a container, by executing the following command:

student@ubuntu:~$ sudo podman container run hello-world
✔ docker.io/library/hello-world:latest
Trying to pull docker.io/library/hello-world:latest...
Getting image source signatures
Copying blob c1ec31eb5944 done |
Copying config d2c94e258d done |
Writing manifest to image destination
Hello from Docker!
This message shows that your installation appears to be working correctly.
To generate this message, Docker took the following steps:
1. The Docker client contacted the Docker daemon.
2. The Docker daemon pulled the "hello-world" image from the Docker Hub.
(amd64)
3. The Docker daemon created a new container from that image which runs the
executable that produces the output you are currently reading.
4. The Docker daemon streamed that output to the Docker client, which sent
it
to your terminal.
To try something more ambitious, you can run an Ubuntu container with:
$ docker run -it ubuntu bash
Share images, automate workflows, and more with a free Docker ID:
https://hub.docker.com/

We can list the available images on our host by executing the following command:

student@ubuntu:~$ sudo podman image ls
REPOSITORY TAG IMAGE ID CREATED SIZE
docker.io/library/hello-world latest d2c94e258dcb 8 months ago 26.3
kB

An Image can be built using Podman as well. We will use the same Dockerfile as in the earlier lab exercise.

student@ubuntu:~$ sudo podman image build . -t nginx:podman
STEP 1/4: FROM nginx
✔ docker.io/library/nginx:latest
Trying to pull docker.io/library/nginx:latest...
Getting image source signatures
Copying blob af107e978371 done |
Copying blob 5e99d351b073 done |
Copying blob 336ba1f05c3e done |
Copying blob 8c37d2ff6efa done |
Copying blob 51d6357098de done |
Copying blob 782f1ecce57d done |
Copying blob 7b73345df136 done |
Copying config d453dd892d done |
Writing manifest to image destination
STEP 2/4: COPY . /usr/share/nginx/html
--> 95fa83a8887f
STEP 3/4: EXPOSE 80/tcp
--> aecf247e1625
STEP 4/4: CMD ["nginx", "-g daemon off;"]
COMMIT nginx:podman
--> 7958e43aed3c
Successfully tagged localhost/nginx:podman
7958e43aed3c9591ca7ab5cae328dfdc8344a8a5420b07ac473bed169319869d

Podman provides a command line interface (CLI) familiar to anyone who has used the Docker Container Engine. Most users can simply alias Docker to Podman (alias docker=podman) without any problems. Explore further options and commands using Podman.

Introduction to Terraform

Mon, 01 Jan 0001 00:00:00 +0000

What is Terraform? 🔗

Imagine this: you’re a software developer, coding away on an amazing new app. But then comes the dreaded task – setting up the infrastructure for your application. Manually provisioning servers, configuring networks, and managing databases sounds like a nightmare, right?

Well, fear no more! Infrastructure as Code (IaC) tools like Terraform are here to save the day. Think of Terraform as your personal infrastructure chef, taking care of all the tedious infrastructure setup and management tasks with ease.

Major Cloud Services

Mon, 01 Jan 0001 00:00:00 +0000

Cloud Services 🔗

Compute Services

Compute services are like the engine of the cloud, providing the power to run applications and process data. Imagine them as the virtual brains behind your software. Virtual Machines (VMs) act as versatile computing environments, while containers, like those managed by Google Kubernetes Engine (GKE), Azure Kubernetes Service (AKS) or Rancher, offer a lightweight and scalable way to package and deploy applications.

The 7 Principles of SRE

Mon, 01 Jan 0001 00:00:00 +0000

Principles of SRE 🔗

Site Reliability Engineering (SRE) follows a set of principles that guide the approach to building and maintaining reliable, scalable, and efficient software systems. These principles are key to the SRE philosophy.

Embracing Risk

SREs aim to identify the acceptable level of risk and manage it appropriately. No system is ever truly perfect. SRE acknowledges that there will be failures and focuses on minimizing their impact and ensuring fast recovery.

The Future of DevOps: Exciting Trends and Predictions

Mon, 01 Jan 0001 00:00:00 +0000

The Future of DevOps 🔗

DevOps has revolutionized software development by fostering collaboration, automation, and rapid feedback loops. As technology continues to evolve at an unprecedented rate, it’s natural to wonder what the future holds for this vital practice.

Emerging Trends 🔗

AI-powered automation

Artificial Intelligence (AI) and Machine Learning (ML) are poised to significantly impact DevOps. AI-powered tools will automate tedious tasks, predict failures, and optimize software delivery processes, freeing up human resources for higher-level thinking and strategic decisions.

The Need for Container Orchestration

Mon, 01 Jan 0001 00:00:00 +0000

Why is Container Orchestration Essential? 🔗

As containerization has become increasingly popular for deploying applications, the need for container orchestration has also grown. While individual containers offer many advantages, managing a large number of them across different environments can quickly become complex and cumbersome. Container orchestration platforms address this challenge by automating the deployment, scaling, and management of containerized applications.

Automating container management 🔗

Manually managing a large number of containers is error-prone and time-consuming. Container orchestration platforms automate many tasks, including:

Building SRE Practices in Your Organization

Mon, 01 Jan 0001 00:00:00 +0000

Building SRE Practices 🔗

Building a robust Site Reliability Engineering (SRE) practice within your organization requires a multi-pronged approach.

Gain buy-in and support 🔗

Demonstrate the value of SRE: Highlight the benefits of improved reliability, performance, and efficiency to key stakeholders.

Secure funding and resources: Allocate budget and personnel to build and maintain a dedicated SRE team.

Foster a culture of collaboration: Encourage collaboration between SRE, development, and operations teams.

Challenges and Opportunities

Mon, 01 Jan 0001 00:00:00 +0000

Challenges 🔗

Despite the exciting possibilities, the future of DevOps also presents challenges. These include:

Managing the complexity of new technologies

Integrating AI, ML, cloud platforms, and other advanced technologies can be complex and require significant expertise.
Ensuring security in a rapidly evolving environment

The increasing reliance on automation and self-service platforms raises security concerns that need to be addressed effectively.
Fostering a culture of collaboration and shared responsibility

Continuous Integration (CI) Principles and Practices

Mon, 01 Jan 0001 00:00:00 +0000

Continuous Integration (CI) Principles 🔗

Frequent Code Integration: Developers integrate their code changes into a shared repository frequently, ensuring that the codebase is continuously evolving in a collaborative manner.
Automated Builds: Every integration triggers an automated build process, including compiling the code, running tests, and creating executable artifacts. Automation ensures consistency and reliability.
Automated Testing: A comprehensive suite of automated tests, including unit tests and integration tests, is run with each integration to detect and address issues early in the development process.

Introduction to OpenTofu

Mon, 01 Jan 0001 00:00:00 +0000

What is OpenTofu? 🔗

OpenTofu is an open source IaC tool forked from Terraform version 1.5. It inherits the familiar syntax and functionality of Terraform, making it easy for existing Terraform users to pick up and use. However, OpenTofu goes beyond simply replicating Terraform. It introduces several key enhancements and features that cater to the needs of the open source community and address the limitations of the BSL.

Kubernetes: The Orchestrator of Containerized Applications

Mon, 01 Jan 0001 00:00:00 +0000

Overview 🔗

Kubernetes, often abbreviated as k8s, is an open source platform for automating the deployment, scaling, and management of containerized applications. It has become the de facto standard for container orchestration, powering a vast number of applications across various industries.

Imagine you have a large orchestra with many different instruments. Each instrument plays its own unique part, but they all need to work together in harmony to create beautiful music. Kubernetes is like the conductor of this orchestra, coordinating the different containerized applications (instruments) to ensure your application runs smoothly and efficiently.

Popular Observability Tools

Mon, 01 Jan 0001 00:00:00 +0000

Tools 🔗

Monitoring and Metrics 🔗

Prometheus is an open source monitoring and alerting toolkit designed for reliability and scalability. It collects metrics from configured targets, stores them, and makes them available for querying.

Grafana is a popular open source platform for visualizing and analyzing metrics. Grafana integrates with various data sources, including Prometheus, InfluxDB, and Elasticsearch.

InfluxDB is a high-performance, distributed, and scalable time-series database. It is commonly used for storing and querying metrics data.

Setting up a Kubernetes Cluster with kind

Mon, 01 Jan 0001 00:00:00 +0000

kind is a tool for running local Kubernetes clusters using Docker container nodes. kind was primarily designed for testing Kubernetes itself, but may be used for local development or CI. We have uninstalled Docker in exercise 4; kind needs Docker as a prerequisite; let’s follow Exercise 4.1 steps and install Docker. Note: Docker should be installed on the node before proceeding further. To interact with our Kubernetes cluster, kubectl is the primary go to command line tool. Let us install kubectl on our node by executing these commands:

Continuous Delivery (CD) and Continuous Deployment (CD) Principles and Practices

Mon, 01 Jan 0001 00:00:00 +0000

Principles 🔗

Consistent Deployment Process: Ensure that the deployment process is consistent and repeatable, reducing the risk of errors and ensuring that each release is reliable.
Automated Deployment: Automate the deployment process to create a streamlined and efficient method for releasing software. This includes deploying to staging environments for additional testing.
Environment Parity: Strive for parity between different environments, such as development, staging, and production, to minimize issues related to environment-specific differences.

Deploy a Sample Application

Mon, 01 Jan 0001 00:00:00 +0000

In this exercise, we will deploy a sample application in Kubernetes and access it.

Create webapp.yaml file with contents below. This file defines a Kubernetes deployment for your Sample Application.

apiVersion: apps/v1
kind: Deployment
metadata:
 name: webapp
spec:
 replicas: 2
 selector:
 matchLabels:
 app: webapp
 template:
 metadata:
 labels:
 app: webapp
 spec:
 containers:
 - image: nginx
 name: nginx

Deploy the application:

kubectl apply -f webapp.yaml

Verify the deployment:

kubectl get deployments
NAME READY UP-TO-DATE AVAILABLE AGE
webapp 2/2 2 2 89s

Set up port forwarding to access the application. In this step, we are establishing a port forwarding rule that redirects network traffic from a specific port on your local machine to the corresponding port on the Kubernetes pod hosting the server. By doing so, you enable direct access to the server via http://localhost:8080 from your local computer. This action bridges the network gap between your local environment and the isolated Kubernetes pod, allowing you to test and interact with the deployed application as if it were running locally. It’s important to run this in a new terminal tab to keep the port forwarding active throughout your testing session.

kubectl port-forward deploy/webapp 8080:80
Forwarding from 127.0.0.1:8080 -> 80
Forwarding from [::1]:8080 -> 80

Test the sample application by sending an HTTP request.

curl http://localhost:8080/
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
html { color-scheme: light dark; }
<Output Truncated>

Cleanup by deleting the deployment:

kubectl delete deploy webapp

Future Trends and Evolutions in SRE

Mon, 01 Jan 0001 00:00:00 +0000

Trends and Evolutions 🔗

The world of Site Reliability Engineering (SRE) is constantly evolving, driven by the ever-changing landscape of technology and the increasing demands for reliable, scalable, and secure systems.

AI and Machine Learning Integration: SRE teams will increasingly use AI and machine learning to automate routine tasks, predict system failures, and optimize performance. This trend includes the development of self-healing systems that can automatically detect and rectify issues without human intervention.

Introduction to AWS CloudFormation

Mon, 01 Jan 0001 00:00:00 +0000

Overview of AWS CloudFormation 🔗

AWS CloudFormation is a powerful Infrastructure as Code (IaC) service offered by Amazon Web Services (AWS) that allows you to define, provision, and manage your cloud infrastructure in a declarative way. With CloudFormation, you can describe your infrastructure using templates, which are written in JSON or YAML and contain a set of instructions for AWS to create and configure the resources you need. This approach provides several key benefits, including:

Key Components of Kubernetes

Mon, 01 Jan 0001 00:00:00 +0000

Control Plane Node 🔗

Control plane nodes in Kubernetes play a critical role in managing the cluster’s state and configuration. They are responsible for making global decisions about the cluster (like scheduling), as well as detecting and responding to cluster events (like starting up a new pod when a deployment’s replicas field is unsatisfied). Here are the key components of control plane nodes:

API Server

Understanding Release Strategies

Mon, 01 Jan 0001 00:00:00 +0000

Release Strategies 🔗

Release strategies play a crucial role in the software development lifecycle, determining how and when new features, enhancements, and bug fixes are delivered to end-users. Different release strategies offer varying degrees of control, risk management, and flexibility.

Rolling Release 🔗

In a rolling release strategy, new features and updates are continuously and incrementally released as soon as they are ready. There are no distinct version numbers or major releases. The software is always in a state of evolution.

What Kubernetes is Not

Mon, 01 Jan 0001 00:00:00 +0000

Kubernetes is NOT a 🔗

Kubernetes is not a Traditional Virtualization Platform 🔗

Kubernetes operates at the container orchestration level, not at the virtual machine level. It doesn’t provide virtualization in the same way as hypervisors like VMware or KVM. Instead, it manages and orchestrates containerized applications.

Kubernetes is not a Containerization Platform 🔗

While Kubernetes works with containers, it is not a containerization platform itself. It relies on container runtimes like containerd or Docker to manage and run containers. Kubernetes focuses on orchestrating and automating container deployment, scaling, and management.

Developing Container Technologies

Mon, 01 Jan 0001 00:00:00 +0000

Containerd 🔗

While not exactly emerging (it has been around for a few years), containerd is worth mentioning. It’s a core container runtime that was originally part of Docker but has become a standalone project and a key component of various container platforms.

Podman 🔗

Podman is an open source container management tool that aims to provide a daemonless, rootless container experience. It allows users to manage containers without requiring a central daemon process, making it a lightweight alternative to Docker.

GitOps as a Deployment Tool

Mon, 01 Jan 0001 00:00:00 +0000

GitOps Overview 🔗

GitOps is a deployment methodology that leverages Git as a single source of truth for infrastructure and application configuration. This approach enables developers to manage their applications and infrastructure in a declarative manner, using familiar Git workflows.

Key principles of GitOps 🔗

Declarative configuration: The desired state of the infrastructure and application is declared in Git repositories as YAML manifests, Helm charts, or other declarative formats.

Mon, 01 Jan 0001 00:00:00 +0000

Course Test

Mon, 01 Jan 0001 00:00:00 +0000

Course Test

Mon, 01 Jan 0001 00:00:00 +0000

Quiz

Mon, 01 Jan 0001 00:00:00 +0000

Quiz

Mon, 01 Jan 0001 00:00:00 +0000

Quiz

Mon, 01 Jan 0001 00:00:00 +0000

Quiz

Mon, 01 Jan 0001 00:00:00 +0000

Quiz

Mon, 01 Jan 0001 00:00:00 +0000

Quiz

Mon, 01 Jan 0001 00:00:00 +0000

Quiz

Mon, 01 Jan 0001 00:00:00 +0000

Quiz

Mon, 01 Jan 0001 00:00:00 +0000

Quiz

Mon, 01 Jan 0001 00:00:00 +0000

Quiz

Mon, 01 Jan 0001 00:00:00 +0000

Quiz

Mon, 01 Jan 0001 00:00:00 +0000

Quiz

Mon, 01 Jan 0001 00:00:00 +0000

Quiz

Mon, 01 Jan 0001 00:00:00 +0000

Quiz

Mon, 01 Jan 0001 00:00:00 +0000

Quiz

Mon, 01 Jan 0001 00:00:00 +0000

Quiz

Mon, 01 Jan 0001 00:00:00 +0000

Quiz

Mon, 01 Jan 0001 00:00:00 +0000

Quiz

Mon, 01 Jan 0001 00:00:00 +0000

Quiz

Mon, 01 Jan 0001 00:00:00 +0000

Quiz

Mon, 01 Jan 0001 00:00:00 +0000

Quiz

Mon, 01 Jan 0001 00:00:00 +0000

Quiz

Mon, 01 Jan 0001 00:00:00 +0000

Quiz

Mon, 01 Jan 0001 00:00:00 +0000

Quiz

Mon, 01 Jan 0001 00:00:00 +0000

Quiz

Mon, 01 Jan 0001 00:00:00 +0000

Quiz

Mon, 01 Jan 0001 00:00:00 +0000

Quiz

Mon, 01 Jan 0001 00:00:00 +0000

Quiz

Mon, 01 Jan 0001 00:00:00 +0000

Quiz

Mon, 01 Jan 0001 00:00:00 +0000

Quiz

Mon, 01 Jan 0001 00:00:00 +0000

Quiz

Mon, 01 Jan 0001 00:00:00 +0000

Quiz

Mon, 01 Jan 0001 00:00:00 +0000

Quiz

Mon, 01 Jan 0001 00:00:00 +0000

Quiz

Mon, 01 Jan 0001 00:00:00 +0000

Quiz

Mon, 01 Jan 0001 00:00:00 +0000

Quiz

Mon, 01 Jan 0001 00:00:00 +0000

Quiz

Mon, 01 Jan 0001 00:00:00 +0000

Quiz

Mon, 01 Jan 0001 00:00:00 +0000

Quiz

Mon, 01 Jan 0001 00:00:00 +0000

Quiz

Mon, 01 Jan 0001 00:00:00 +0000

Quiz

Mon, 01 Jan 0001 00:00:00 +0000

Quiz

Mon, 01 Jan 0001 00:00:00 +0000

Quiz

Mon, 01 Jan 0001 00:00:00 +0000

Quiz

Mon, 01 Jan 0001 00:00:00 +0000

Quiz

Mon, 01 Jan 0001 00:00:00 +0000

Quiz

Mon, 01 Jan 0001 00:00:00 +0000

Quiz

Mon, 01 Jan 0001 00:00:00 +0000

Quiz

Mon, 01 Jan 0001 00:00:00 +0000

Test

Mon, 01 Jan 0001 00:00:00 +0000

Test

Mon, 01 Jan 0001 00:00:00 +0000

Test

Mon, 01 Jan 0001 00:00:00 +0000

Test

Mon, 01 Jan 0001 00:00:00 +0000

Test

Mon, 01 Jan 0001 00:00:00 +0000

Test

Mon, 01 Jan 0001 00:00:00 +0000

Test

Mon, 01 Jan 0001 00:00:00 +0000

Test

Mon, 01 Jan 0001 00:00:00 +0000

Test

Mon, 01 Jan 0001 00:00:00 +0000

Test

Mon, 01 Jan 0001 00:00:00 +0000

Test

Mon, 01 Jan 0001 00:00:00 +0000

Test

Mon, 01 Jan 0001 00:00:00 +0000

Test

Mon, 01 Jan 0001 00:00:00 +0000

Test

Mon, 01 Jan 0001 00:00:00 +0000

Test

Mon, 01 Jan 0001 00:00:00 +0000

Test

Mon, 01 Jan 0001 00:00:00 +0000

Test

Mon, 01 Jan 0001 00:00:00 +0000

Test

Mon, 01 Jan 0001 00:00:00 +0000

Test

Mon, 01 Jan 0001 00:00:00 +0000

Test

Mon, 01 Jan 0001 00:00:00 +0000

Test

Mon, 01 Jan 0001 00:00:00 +0000

Test

Mon, 01 Jan 0001 00:00:00 +0000

Test

Mon, 01 Jan 0001 00:00:00 +0000

Test

Mon, 01 Jan 0001 00:00:00 +0000

Test

Mon, 01 Jan 0001 00:00:00 +0000

Test

Mon, 01 Jan 0001 00:00:00 +0000

Test

Mon, 01 Jan 0001 00:00:00 +0000