How to Evaluate an AI Project in 30 Minutes
A simple framework for executives who need to decide what's worth investing in.
You’re in a meeting. Someone’s proposing an AI initiative. They’ve got a vendor deck, a timeline, and a budget number. You need to decide whether this is worth pursuing or whether it’s a solution looking for a problem. Here’s how I’d evaluate it in 30 minutes.
Five questions, in order
These aren’t comprehensive. They’re triage. The goal is to quickly separate the projects worth a deeper look from the ones that sound impressive but won’t deliver.
1. What’s the current workflow, in plain language?
Not the aspirational workflow. The actual one. Who does what, how long does it take, and where does it break? If the proposal can’t describe the existing process in two sentences, they don’t understand the problem well enough to solve it.
I’ve watched teams propose AI-powered “intelligent routing” for support tickets when the real problem was that three people had different spreadsheets defining the routing rules. You don’t need a model for that. You need a meeting.
2. What percentage of this workflow is preparation versus judgment?
This is the single most predictive question I’ve found. Preparation work — research, data gathering, drafting, summarization — is where AI delivers 50-80% efficiency gains reliably. Judgment work — decisions, trade-offs, stakeholder management — is where AI adds marginal value at best.
If the proposed workflow is 70%+ preparation, keep listening. If it’s mostly judgment work, push hard on what the measurable improvement actually looks like.
3. What does the team do with the time they save?
This is where most proposals fall apart. “It saves 10 hours a week per analyst” sounds great until you realize those 10 hours get absorbed into meetings and Slack, not redirected to higher-value work. The best proposals have a specific answer: “Analysts currently spend 10 hours on report preparation. With AI handling the first draft, they redirect that time to client-facing analysis, which we can bill at $X/hour.”
If the time savings don’t convert to revenue, capacity, or quality improvements you can measure, the ROI case is theoretical.
4. What happens when the AI is wrong?
Every AI system produces errors. The question is whether your workflow catches them before they cause damage. A good proposal has a human review step at every point where an error could be costly. A bad proposal assumes 95% accuracy is good enough without mapping what the 5% failure looks like.
Ask specifically: “If the AI makes a mistake on this task, who catches it, how, and what does it cost?” If the answer is vague, the project isn’t ready.
5. Can we test this in two weeks with five people?
The best AI projects start small and prove themselves. If the proposal requires six months of integration work before anyone sees a result, the risk is high and the feedback loop is too slow. Push for a pilot scope: a small team, a narrow use case, a clear success metric, and a two-to-four week timeline.
If the vendor or internal team resists a small pilot, that tells you something about their confidence in the outcome.
The 30-minute scorecard
After working through these five questions, you’ll have a rough score. Projects that have clear answers to all five are worth a deeper investment of time. Projects that stumble on questions 2 and 3 — the preparation ratio and the time conversion — are the ones most likely to disappoint.
This isn’t about being skeptical of AI. It’s about being disciplined with where you invest. The companies pulling ahead aren’t the ones doing the most AI projects. They’re the ones doing the right ones.