AI Dev Tools - Project Peer Review 3

For the AI Dev Tools Zoomcamp, a certificate of completion will not be issued unless, in addition to completing and submitting the final project, the learner also completes three peer reviews.

Peer Review Number 3

I have reached the last peer review, number 3. coo1y has developed a Fast, anonymous, multimodal AI assistant.

I scored the project against the criteria found at AI Dev Tools Zoomcamp - Final Project

Remarks for the Developer

1. Problem description: clear and specific.

2. Product Specification: product-spec.md is thorough and has 24 acceptance criteria (section 23). However, the 2-point level also requires a public task tracker that the agents demonstrably used, and there is none. docs/ai-workflow.md says outright that “one Claude Code session executed the plan”, with phases tracked inside that document rather than as issues. Nothing in the repository links to GitHub Issues or a project board, and the only pull request (#1) is a CI fix.

3. Agent team: The human/Claude delegation model, three specialist subagents and the gated phase plan are all defined. One caveat: the document admits the subagents are for “future work” and were not used to build the project.

1. Context Management: AGENTS.md does not link docs/ files by name, but it does point to modular documents: the product spec, the review checklist in agent-capabilities/review-checklist.md, and the process skills (for example contract-first-change). Its repository map also directs agents to docs/, ops/ and security/. CLAUDE.md imports AGENTS.md and points to docs/agent-extension-pack.md. I read this as meeting the intent of “points to modular docs (process, teams, etc.)”. If you keep 1, the reasonable justification is that there is no explicit link to a team or process document.

2. Application and 7. Containerization: docker compose up --build -d --wait starts PostgreSQL, the backend and the frontend, and the container runs alembic upgrade head on start. Without a .env file it uses a mock model and mock search, so it works offline.

1. Testing: separate unit and integration suites, run together by make check, and confirmed passing here.

2. Deployment: Full marks because the README has screenshots from the live app and the CI deploy job smoke-tests the live URL. The likely cause of the failure is in the developer’s own docs/deployment.md: Render’s free PostgreSQL instances expire after 30 days. The deployment proof is dated 3 October, so the database probably expired around 2 November.

3. CI/CD: The pipeline is excellent: lint, type checks, unit, integration (on PostgreSQL), contract and end-to-end tests, then an automatic Render deployment with a smoke test of the live URL. But the 3-point level requires deploying to a dev environment plus a manual deployment to production. Here every push to main goes straight to a single production environment, and there is no dev environment or manual promotion step. That is exactly the 2-point description.

1. Observability:

• What exists: structured JSON logs carrying a per-request request_id (backend/app/core/logging.py), and a content-free chat_requests telemetry table. That table is the “telemetry under database” you noticed. A token-protected /metrics endpoint aggregates it into error rate and time-to-first-token percentiles.
• What is missing: there are no traces or tracing library, and no UI where the signals are collected and viewed. The ops/diagnose.sh script produces a Markdown report on demand, which is not a dashboard.
• Correlation is also incomplete. The ID in the logs comes from the HTTP middleware. The chat service creates a separate UUID for each telemetry row and for the request_id sent to the browser, so a chat request’s logs and its telemetry row cannot be joined on one ID.

1. Alerting: No alert rule exists. ops/analyze.py flags SLO breaches only when someone runs the diagnosis script by hand, which is a report rather than an alert.

2. On-call agent: The ops-diagnostician subagent and the incident-diagnosis skill are well written, but they are invoked manually, have read-only tools, and diagnose rather than fix. Nothing is triggered by an alert.