Labs: Practice Environments You Run Yourself

These are labs you run on your own machine or your own cloud account. We do not host an environment, there is nothing to log into, and nothing here costs money beyond what you choose to spend with a cloud provider.

That is a deliberate choice rather than a limitation. A lab in somebody else's sandbox teaches you the sandbox. A lab you set up yourself teaches you the setup, which is most of the actual skill — and it leaves you with an environment you keep.

What you need#

Almost everything below runs on a laptop with a container runtime installed. Two labs benefit from a cloud account; use the free tier and set a budget alert before you start, not after.

Set up a scratch directory, use version control from the first lab, and keep your notes in the repo. Six months later the notes are worth more than the code.

Lab 1 — Break a container on purpose#

Teaches: why containers fail in production, which is rarely the reason people expect.

Build a small image for any application you like. Then, one at a time, cause each of these and observe what you actually see:

• Set a memory limit below what it needs. Note that the process is killed with no application-level error — this is the failure everyone misreads.
• Run it as root, then create a non-root user and switch. Try to write outside the permitted path.
• Add a file in one layer, delete it in the next, then find it in the image history. This is how secrets leak.
• Send it a termination signal while a request is in flight, without handling the signal. Then handle it and repeat.

Work with Docker open and check against the container review.

Lab 2 — A retrieval system over your own documents#

Teaches: that retrieval quality, not model quality, is the ceiling.

Take a folder of real documents you know well. Index them, ask twenty questions whose answers you already know, and record the hit rate before tuning anything. Then change one variable at a time — chunking on structure instead of fixed size, adding keyword search alongside semantic, prepending headings to chunks — and re-measure after each.

The lesson arrives when you find a question that fails no matter how you word the prompt, because the answer is not in any retrieved chunk. Follow RAG and the RAG checklist.

Lab 3 — Deploy, break, roll back#

Teaches: that rollback is a skill, not a button.

Deploy anything to a small server. Write a deployment runbook. Then deploy a version that is broken in a way that is not immediately obvious — a slow query, a wrong configuration value, an endpoint that returns success while doing nothing.

Time how long it takes you to notice, and how long to roll back. Both numbers will be worse than you expect, and both are the point. See DevOps.

Lab 4 — Fill a disk and diagnose it#

Teaches: the most common preventable outage on any Linux system.

On a disposable virtual machine, fill the disk in three different ways: with large files, with millions of tiny files until inodes run out, and with a log file held open by a running process after deletion.

Diagnose each using only command-line tools. The third is the one that defeats most people, because the space is used and the file cannot be found. See Linux.

Lab 5 — A pipeline that fails closed#

Teaches: why "the job succeeded" and "the data arrived" are different claims.

Build a small pipeline that fetches data from any public source and writes a file. Then simulate the source being unavailable.

Most first attempts write an empty file and exit zero — which is exactly how a real outage silently destroys good data. Fix it: below a minimum row count, write nothing, keep the last good output, exit non-zero. Then re-run the outage and confirm the old data survived. See data engineering and its pipeline specification.

Lab 6 — Kubernetes probes, done wrong first#

Teaches: the difference between liveness and readiness, permanently.

In a local single-node cluster, deploy an application with a dependency. Point the liveness probe at a health check that includes the dependency. Make the dependency slow.

Watch every pod restart at once. Then separate the probes correctly and repeat. Ten minutes, and you will never make this mistake in production. See Kubernetes.

How to get the most from a lab#

Predict before you run. Write down what you expect to happen. The gap between prediction and result is the learning; without the prediction there is no gap to notice.

Break it before you fix it. Working systems teach very little. The behaviour under failure is what you will need at 3am.

Do not clean up immediately. Sit with the broken state and investigate it properly. The instinct to reset and retry skips the diagnosis, which is the transferable skill.

Write down what surprised you. One line per lab. That file becomes genuinely valuable.

FAQ#

Do you host these environments?#

No. Everything runs on your own machine or your own cloud account. That is deliberate — setting up the environment is a substantial part of what you are learning, and you keep it afterwards.

Will these cost money?#

Labs 1, 2, 4, 5 and 6 run locally and cost nothing. Anything you choose to run in a cloud account costs whatever that provider charges — stay within a free tier and set a budget alert first.

How long does each take?#

Between one and four hours if you already know the surrounding technology, considerably longer if you are learning it alongside. The value is not correlated with speed.

What if I get stuck?#

Read the linked guide, then the actual error message properly, then the documentation. Getting stuck and working out is the exercise. A lab you completed by following instructions exactly has taught you to follow instructions.

Can I use these for team training?#

Yes, and they work well in pairs — one person predicts, the other runs. Discussion of the prediction gap is where most of the learning happens.

Are these the same as AI Labs?#

No. AI Labs are our own experiments, shown honestly as demonstrations rather than products. These are practice exercises for you to run.

What order should I do them in?#

Whichever matches what you are about to work on. If nothing is pressing, lab 1 then lab 3 gives the broadest return for the time.

Related Articles#

Pair these with Exercises for shorter practice, Projects for something substantial, and Courses for the reading order.

What else is coming for Labs