The short version: SMBs with no dedicated technical bench should hire. Mid-market companies with a developer + ops person + product manager who can commit real weekly hours to AI should build. Most successful programs are actually hybrid, agency ships the first few playbooks while training an internal owner.
When in-house wins
- You already have a developer + an ops person + a product manager who can dedicate real time every week
- Your AI use case is unusual enough that no agency has shipped your exact thing before
- You're building IP that's a competitive differentiator (proprietary models, novel architectures)
- You're an enterprise with multi-year AI roadmap and a CTO who wants AI native
When an agency wins
- You're SMB-sized and don't have the bench to commit the weekly hours the build needs
- Your AI use case is something agencies have shipped many times (receptionist, missed-call text-back, lead qualifier, scheduling, review automation)
- You want speed-to-live on a single playbook without paying the in-house learning curve
- You want someone whose full-time job is staying current on AI platforms (the space changes weekly)
- You'd rather pay a quoted monthly fee for a working system than hire a full-time ML engineer
The cost math
| Agency (Mastodon) | In-house build | |
|---|---|---|
| Cost | Custom pricing, quoted per business. Request a quote. | Internal engineering, ops, and product time |
| Ongoing cost | Platform and tuning, included in the quote | Platform fees plus internal-owner time |
| First hire to enable in-house | None | A dedicated ML or full-stack engineer |
| Risk of pilot stalling | Low (agency has shipped these before) | High (first-build learning curve) |
| Risk of post-launch drift | Low (agency owns tuning) | High (internal owner gets pulled to other work) |
| IP ownership | You own it; agency owns the playbook | You own everything |
| Speed to next use case | Weeks (next playbook) | Months (re-scope, re-learn) |
Hybrid as a third option
Most successful SMB AI programs are hybrid: agency does the first few playbooks while training an internal owner. Once the owner is trained, the internal owner runs the routine work and the agency handles new platforms, complex integrations, and quarterly tuning. This is what we recommend most often.
Questions to ask either way
- Who specifically owns this system after launch?
- What's our written escalation path when the AI gets it wrong?
- How do we measure whether this worked, in dollars or hours?
- What does a vendor (or internal team) handoff look like if priorities change?
- Where does the data live? Who controls it?