Learning in Public: DataTalks.Club AI Dev Tools Zoomcamp Week 4

Earlier this week I posted a meme on Instagram. The first line said, “Them: Build it on AWS”. The second line said, “Me:”. Below that was a screenshot of an AI generated TikTok about showing brown-skinned slaves with sweaty backs driving a massive stone block on wooden rollers toward the most famous of the Pyramids of Giza.

Get it?

Because, building an app on AWS is as challenging as building a great pyramid.

Yes, you are right. It was very clever. And we all laughed, and laughed.

But seriously, week 4 of the DataTalks.Club AI Dev Tools Zoomcamp has proved to be more of a hard knocks school education than I bargained for.

Assignment

This week's AI Dev Tools Zoomcamp assignment requires us to: build a mechanism to observe user impact; alert the and provide alert contact at the same time; provide mechanisms to investigate from evidence issues that impact the user; implement functionality whereby the coding agent can make a response within bounds or escalate to administration; be able to verify application recovery, and implement a workflow that audits the application code as well as the response trail.

"The concrete stack is OpenTelemetry into Prometheus, Loki, and Tempo, with Grafana on top, and a responder that runs Codex or Claude Code in headless mode. Those products are replaceable. "

Unintended Learnings:

• Absolute reliance on tmux
• Agents use text files to communicate across barriers

Intended Learnings:

• AWS
• Docker
• Grafana
• OpenTelemetry
• GitHub CI/CD (Automated merge, etc.)

Real world outcome:

• A mini-kanban web app running on AWS infrastructure at https://cards.terzotech.net
• Guys, I might totally be in to AWS
• The “one practical idea” expanded below

Unintended Learnings

If I had not made it clear, this was a gruelling assignment. It was worked out over about five days of work. In this respect, it was essential to have one continuius coding session, rather than four or five separate sessions logged into the development server on the cloud. Tmux was essential for this. It allowed for exactly that: One persistent session, that I could sign into and drop out of as my time budget allowed, but from Claude's perspective was one continuous session.

The other major unintended learning from this weeks assignment was that when Claude and I had issues passing information back and forth, it proposed and I followed the practice of creating text files on the server in a neutral space, that we could both read and write to without restriction.

Specifically, my CMD terminal from my Windows machine, would not allow me to scroll back up through the terminal session -- for some bizarre and strange reason. So when Claude gave me a long-ish configuration file to implement on the AWS admin console, I knew it was there, but I literally could not scroll back to see the whole thing. At the same time, I could neither copy from nor paste to the CMD window that was signed into the AWS server where Claude was running. I have grown quite accustomed to copying from and pasting to sessions on my Ubuntu cloud server, and the inability to do it this time was highly frustrating. (There were other technical reasons for this, but I will not bore with the details here.)

One Practical Idea

When architecting complex CI / CD workflows that leverage AI agents, it is advisable to supply the incident agent with evidence and limits, and to not give it carte blanche and unrestricted trust.

In the CI / CD pipeline, when an incremental development event is triggered, and the responder feeds metrics, logs, and traces all associated with one shared trace ID are routed to the principal coordinating agent as inputs. Furthermore, the principal coordinating agent is architected according to a narrowly scoped allowlist restricted to edit, test, rebuild, and curl functions that operate via localhost only.

The agent never performs repository commits under its own determination. It is a human who reviews the repository diff. In our working exercise, this approach fixed the month-end bug in 96 seconds. The one functional miss that occurred (a curl form outside the allowlist) failed safely, instead of doing something expected.

Conclusion

With respect to the AI Dev Tools Zoomcamp, this week's assignment was a high water mark for complexity, structured coordination, and technical stretch. I forced myself into the realm of AWS, and with the help of Claude Code doing the heavy lifting, was able to implement our app on a Docker container in the secure cloud.

With respect to my application development work going forward, while I will continue to utilise a coding agent and CI / CD pipeline, I do not feel that the added complexity of containerizing my own app with Docker has shown benefits that justify the extra complexity.