A team of 3 engineers can build an AI MVP in weeks. A team of 30 engineers can take months to ship the same feature.
Scaling is not about adding people. It is about maintaining velocity while growing. This post shares what we have learned about scaling AI engineering teams effectively.
| Size | Structure | Communication |
|---|
| 1-3 | Single team | Direct |
| 4-8 | Single team + roles | Direct |
| 9-15 | Pod structure | Team leads |
| 16-30 | Multiple pods | Cross-pod sync |
| 30+ | Organization | Formal process |
| Archetype | Focus | Skills |
|---|
| Research | Model development | ML, research |
| Platform | Infrastructure | Systems, MLOps |
| Product | User-facing features | Full-stack, UX |
| Data | Data pipelines | Data engineering |
| Evaluation | Quality, testing | Evaluation, QA |
| Pod | Size | Responsibilities |
|---|
| Core AI | 4-5 | Model integration, prompts |
| Platform | 3-4 | Infrastructure, APIs |
| Product | 3-4 | Features, UX |
| Shared | 1-2 | Data, evaluation |
| Pod | Size | Responsibilities |
|---|
| Foundation | 5-7 | Core AI capabilities |
| Platform | 4-6 | Infrastructure, tooling |
| Product A | 4-5 | Feature area 1 |
| Product B | 4-5 | Feature area 2 |
| Data/Eval | 3-4 | Data, quality |
| Role | Focus | When to Hire |
|---|
| AI Engineer | Model integration | Day 1 |
| ML Engineer | Model training/fine-tuning | 5+ engineers |
| MLOps | Infrastructure | 8+ engineers |
| Data Engineer | Pipelines | 10+ engineers |
| AI Product Manager | Strategy | 10+ engineers |
| Research Scientist | Novel approaches | 15+ engineers |
| Stage | Engineer Does | Specialist Does |
|---|
| Early | Everything | N/A |
| Growth | Core features | Infra, data |
| Scale | Feature area | Deep specialty |
| Mature | Domain expert | Technical leadership |
| Size | Leadership | Ratio |
|---|
| 1-5 | Tech Lead (IC) | 1:5 |
| 6-12 | Engineering Manager | 1:6-8 |
| 13-20 | EM + Tech Lead | 1:6-8 |
| 20+ | Director + EMs | 1:15-20 |
| Stage | Priority Skills | Why |
|---|
| 0-3 | Generalists | Do everything |
| 4-8 | T-shaped | Depth + breadth |
| 9-15 | Specialists | Scale specific areas |
| 15+ | Mix | Balance team |
| Area | What to Assess | How |
|---|
| Fundamentals | ML/AI basics | Technical questions |
| Systems thinking | Architecture design | System design |
| Pragmatism | Ship vs perfect | Past projects |
| Learning | Adaptability | New domain question |
| Communication | Explain AI to non-AI | Explanation exercise |
| Team Size | Hire Per Quarter | Growth Rate |
|---|
| 3-5 | 1-2 | 20-40% |
| 6-10 | 2-3 | 20-30% |
| 11-20 | 3-5 | 15-25% |
| 20+ | 4-6 | 15-20% |
| Mistake | Problem | Solution |
|---|
| Hiring too fast | Culture dilution | Pace to 25% per quarter max |
| Only specialists | Silos | Mix generalists |
| Only generalists | No depth | Add specialists |
| No AI experience | Slow ramp | At least 50% with AI exp |
| Document Type | Owner | Update Frequency |
|---|
| Architecture decisions | Tech Lead | On change |
| Runbooks | On-call | Monthly |
| API documentation | Team | On change |
| Onboarding guide | Manager | Quarterly |
| Best practices | Senior ICs | Quarterly |
| Ritual | Frequency | Duration | Purpose |
|---|
| Tech talks | Bi-weekly | 30 min | Deep dives |
| Demo day | Weekly | 30 min | Show work |
| Architecture review | Monthly | 60 min | Big decisions |
| Retrospective | Bi-weekly | 45 min | Process improvement |
| Reading group | Weekly | 30 min | Stay current |
| Week | Focus | Outcome |
|---|
| 1 | Setup, overview | Dev environment working |
| 2 | Codebase, architecture | First small PR |
| 3 | Feature work (paired) | Feature shipped with buddy |
| 4 | Independent feature | Solo PR |
| 5-8 | Ramp to full velocity | Full contributor |
| Metric | Target | Measure |
|---|
| Time to first PR | Under 5 days | Calendar days |
| Time to solo feature | Under 3 weeks | Calendar days |
| Time to full velocity | Under 8 weeks | Sprint velocity |
| 90-day retention | Over 95% | Still employed |
Maintaining Velocity
| Problem | Symptom | Solution |
|---|
| Too many meetings | Under 4 hours focus time | Meeting-free days |
| Unclear ownership | Diffusion of responsibility | RACI matrix |
| Tech debt | Slow feature delivery | 20% debt allocation |
| Poor tooling | Manual processes | Platform investment |
| Context switching | Many small tasks | Sprint focus |
| Metric | Measure | Target |
|---|
| Lead time | Idea to production | Under 2 weeks |
| Deployment frequency | Deploys per day | Over 1 |
| Change failure rate | Failed deploys | Under 5% |
| Recovery time | Incident to resolution | Under 1 hour |
| Practice | Impact | Implementation |
|---|
| Clear interfaces | Reduces coordination | API contracts |
| Autonomous pods | Parallel work | Pod ownership |
| Shared platform | Reduces duplication | Platform team |
| Async communication | Less meetings | Written culture |
| Meeting | Attendees | Frequency | Duration |
|---|
| Standup | Pod | Daily | 15 min |
| Sprint planning | Pod | Bi-weekly | 60 min |
| Pod sync | Pod leads | Weekly | 30 min |
| All hands | Everyone | Monthly | 45 min |
| 1:1s | Manager + IC | Weekly | 30 min |
| Channel | Use For | Response Time |
|---|
| Slack | Quick questions | 4 hours |
| GitHub | Code review | 24 hours |
| Docs | Permanent knowledge | N/A |
| Email | External, formal | 48 hours |
| Decision Type | Who Decides | Process |
|---|
| Day-to-day | IC | Just do it |
| Feature | Tech Lead | Discussion, decide |
| Architecture | Pod + stakeholders | RFC, review |
| Strategic | Leadership | Proposal, approval |
| Element | Small Team | Large Team |
|---|
| Values | Implicit | Explicit, documented |
| Norms | Organic | Defined, enforced |
| Rituals | Informal | Scheduled |
| Feedback | Direct | Structured |
| Practice | Purpose | Implementation |
|---|
| Values interview | Cultural fit | Behavioral questions |
| Onboarding buddy | Culture transfer | Pairing |
| Culture rituals | Reinforce values | Regular events |
| Recognition | Reward alignment | Public kudos |
| Sign | Indicates | Action |
|---|
| Siloed information | Poor knowledge sharing | Increase rituals |
| Blame culture | Low psychological safety | Leadership modeling |
| Meeting overload | Process over progress | Meeting audit |
| High attrition | Multiple issues | Stay interviews |
- 1Structure evolves - What works at 5 engineers will not work at 20. Plan for transitions.
- 2Hire at sustainable pace - Over 25% growth per quarter dilutes culture and overwhelms onboarding.
- 3Invest in knowledge sharing - Documentation and rituals become critical as you scale.
- 4Autonomous pods scale - Clear ownership and interfaces enable parallel progress.
- 5Measure velocity - What gets measured gets maintained. Track lead time and deployment frequency.
- 6Culture requires intent - Implicit culture becomes explicit or disappears. Document and reinforce.
Scaling is not about adding people. It is about adding people while maintaining the speed, quality, and culture that made you successful in the first place.
