How Non-Engineers Learned to Ship with AI

1

The CMS Wall

▼

The turning point came when Claude Code successfully updated pages stored in GitHub — but "hit a wall" at CMS-protected content. The AI agent could instantly modify half the site, but was completely blocked by the other half.

The primary driver for migration wasn't cost savings — it was velocity and consistency. An agent that can fix 147 pages in 2 PRs is useless if half the site is locked behind a separate system.

Key Insight: The value of AI agents compounds only when they can reach everything. A split content system creates an asymmetry that neutralizes the automation advantage.

Interactive — CMS vs Git: AI Agent Access Map

Click each content type to see whether AI agents can reach and modify it. Notice how CMS-managed content creates a blocked zone.

Select a content type to inspect AI agent access.

What was Sentry's primary reason for migrating away from their CMS?

2

The Migration Plan

▼

Migrating 2,500 pages isn't a weekend project — but Sentry completed it in two months with 2.5 developers, with Claude Code generating most of the code while engineers focused on planning and direction.

The phased approach was deliberate: start with the easiest content, build confidence and tooling, then tackle the complex cases last. The legacy CMS stayed on the free tier throughout — zero downtime, zero risk.

Pages

~2,500

Team

2.5 devs

Duration

2 months

Code gen

Claude Code

Migration Timeline — Click Each Phase to Inspect

Click Play to walk through the phases, or click any step to inspect it.

Why did Sentry migrate docs and marketing pages before blog content?

3

Technical Wins

▼

The migration wasn't just about content access — it unlocked a cascade of technical improvements. Switching from Gatsby to Astro cut build times by 71%, while consolidating 200 custom pages into 3 reusable templates made the codebase dramatically more maintainable.

Build Time Race — Gatsby vs Astro

Web Vitals Before

89

Web Vitals After

97

Staging Failures

↓ 95%

The Marketo form cache became one of the highest-impact changes — not just for speed, but for eliminating recurring staging failure modes that nobody initially predicted would matter.

Custom Pages Before

~200

Reusable Templates

3

200 one-off pages reduced to 3 templates means every future page follows the same pattern — and AI agents know exactly what to expect when reading or editing them.

What was Sentry's build time after migrating from Gatsby to Astro?

4

AI Workflows at Scale

▼

Once the migration was complete, the growth team shipped substantially more — and faster. Tasks that would have taken days became hours; hours became minutes. The key wasn't replacing humans — it was giving humans levers.

A team member who understands the goal can now direct an AI agent to execute it across hundreds of pages simultaneously.

What the Team Shipped Post-Migration — Click a Bar to Learn How

Click any bar to learn how it was accomplished.

Running 11 concurrent A/B tests wasn't possible before. The Git-based workflow makes branching, reviewing, and merging test variants a routine operation rather than an engineering project.

Sentry's team fixed linking issues across 147 pages. How many PRs did this take?

5

Skills as AI Templates

▼

"Skills" are AI instruction files that encode page-type requirements — structure, SEO metadata, brand patterns, and component libraries. They act as a bridge between what a non-technical team member wants and what the codebase needs.

When a growth marketer requests a new landing page, the AI doesn't guess. It follows the skill: interview the user for required fields, then produce properly structured Markdown with a PR ready for review.

The Pattern: Skills encode institutional knowledge. Instead of training every team member on Markdown frontmatter and brand standards, you encode that knowledge once and let the AI enforce it for everyone, every time.

Simulate — AI Skill Interview Flow

Watch how a "landing-page" skill guides a non-technical user through creating a correctly structured page.

What problem do "Skills" (AI instruction files) primarily solve?

6

Lessons & What's Next

▼

The migration surfaced surprises that data migration alone couldn't predict. Design fidelity proved harder than content migration, human error introduced regression, and the most impactful infrastructure change came from an unexpected direction.

Design Fidelity: Migrating data proved far simpler than achieving pixel-perfect design parity. Rendering differences required significant manual work that wasn't budgeted.

Human Error: An email validation rule added during migration tanked key conversion metrics for days before being identified and reversed.

Infrastructure Surprise: The Marketo form cache became one of the highest-impact changes — not just for build speed, but for eliminating recurring staging failure modes nobody expected.

Learning Curve: Real, but smaller than feared. Non-technical team members needed about one month of slower-than-comfortable ramp-up before opening PRs independently.

Launched alongside the migration, the Sentry Cookbook collects developer recipes like "Debug undefined properties in React Native with Sentry Logs."

The insight driving it: LLMs and AI Overviews cite structured, executable content far more reliably than traditional blog posts. Cookbooks are the SEO strategy of the AI citation era.

Format

Step-by-step recipes

Target

AI citation optimization

Audience

Developers

The team is pursuing autonomous workflows — agents running on schedules to handle well-scoped fixes without human initiation. They're also building AI code review to maintain quality across high-volume PR batches.

The trajectory: humans set direction and review outcomes. AI handles execution, scale, and routine corrections — autonomously.

What was unexpectedly impactful about the Marketo form cache change?

How Non-Engineers Learned to Ship

Module Complete