DevOps Project: The Plan

In this project you build a delivery pipeline for a small web API:

• a CRUD API in Python, in its own GitHub repository;
• a GitHub Actions pipeline:
  • a push to your own branch runs the fast tests;
  • a pull request runs more: integration tests, security and code inspection tools;
  • a pull request waits for an approver before it can merge;
  • a merge to main builds a container image and deploys it;
• a deployment on AWS: an EKS cluster, or a set of EC2 instances.

See the finished pipeline at work, one event at a time, in The Pipeline, Step by Step.

The tracks

Choose one of each pair when you start, and follow its steps only.

The API framework

• Flask: the basic track.
  • Each request runs a plain function, start to finish, in a worker of its own.
  • Plain Python is all it takes.
• FastAPI: the advanced track.
  • It checks each request against declared types, and generates an OpenAPI page from them.
  • It is built on asyncio: a route may be an async def, run on the server's one event loop.
    • An await hands the loop to another request while this one waits.
    • A blocking call inside an async def (a time.sleep, a synchronous database driver) stops every request.
  • Choose it if you know async/await and the event loop, or want to learn them on the way.

The deployment target

• EKS: the main track.
  • A Kubernetes cluster: the deployment is a Deployment and a Service, the way most teams ship containers.
  • It costs money from the moment it exists, whether the app runs or not.
  • Created at the start of each working session and destroyed at its end, it costs about $13–$37 in all for 20–30 hours a month over 2–4 months: the computation is in Task 5.
  • Left running, it costs about $200 a month.
• EC2: a set of instances running the container, for students who find EKS too hard.
  • Fewer parts to learn, and only the instances and the load balancer to pay for.
  • The deploy is your own script, where EKS gives you rollouts and health checks.

The app runs in a container on both targets.

Working alone or in a pair

The project is done alone or by two students sharing one repository. Choose when you start.

• In both, the way changes reach main changes at Task 3:
  • Tasks 1–2: you push straight to main. Nothing protects it yet, and the empty pipeline is easiest to watch that way.
  • From Task 3 on: main is protected. Every change goes on a branch of its own, through a pull request, and is merged only once its checks pass and it is approved.
• The difference is who approves.
• Either way, every branch follows one naming scheme: see Topic branches.

Topic branches

Every change starts as a GitHub issue, and its branch is named <type>/<issue>-<what>.

1$ git switch -c feature/12-search-books
2$ git push -u origin feature/12-search-books

• The number links everything: the branch names the issue, the pull request's description says Closes #12, and the merge closes the issue.
• The type groups the branches: GitHub's branch list and a workflow's branches: filter can match fix/**.
• The words say what it is without opening the issue.
• One issue, one branch, one pull request; the branch is deleted after the merge.

The finished pipeline

The Pipeline, Step by Step plays all of it, one event at a time: a push to a topic branch, a pull request through its checks and its review, the merge and the delivery, and a refused push to main.

The order of the tasks

The tasks build the whole pipeline first, then fill it in.

• Tasks 1–3: the pipeline, empty.
  • Every job exists from Task 2 on, as a placeholder that prints what it will do.
  • You see each event run its jobs, and the approver gate the merge, before any job does real work.
• Tasks 4–6: the first complete solution.
  • The placeholders for the image and the deployment become real.
  • A merge to main puts the app on AWS.
• Tasks 7–10: the details.
  • Each task replaces more placeholders, or makes a working job stricter.

Before you start

• A GitHub account.
• An AWS account, from Task 5 on. The infrastructure costs money while it runs; Task 11 removes it.
• A Docker Hub account, from Task 4 on.
• Python 3.12 or newer, Docker and git on your machine.

The site's other material, used along the way:

• GitHub Actions Workflows: an Introduction
• Pipeline Identities: the credentials the pipeline uses
• Git Collaboration: pull requests, review and branch protection