AI Vendor Evaluation

AI Vendor Evaluation

Use When

• Use this skill when the requested outcome is specifically a evaluation scorecard and the supplied brief falls within ai vendor evaluation.

Do Not Use When

• Use ai-readiness-diagnostic when its narrower output is the real deliverable; do not use this skill as a generic substitute.
• Do not use it to publish, send, spend, alter a live account, or make unsupported legal, platform, performance, or certification claims.

Required Inputs

Capability and Permission Boundaries

Default to read-only: inspect supplied material and report findings. Editing, publishing, contacting people, spending, or changing live systems requires separate explicit authority. Minimum capabilities are read access to supplied files and search across the authorised evidence set. Use only the files, tools, accounts and evidence made available for the engagement, expose every unassessed check, and obtain explicit authority before any mutation.

Degraded Mode

Fallback: if files, network access, platform data, language review or production tools are unavailable, return the narrowest useful qualified evaluation scorecard; mark unavailable checks not assessed and never convert them into a pass.

Decision Rules

Workflow

1. Confirm the exact evaluation scorecard, consumer, market, channel and approval boundary; route to ai-readiness-diagnostic if it is the closer match.
2. Inventory supplied facts, source provenance, constraints and missing inputs; stop if the objective, audience or authority is unknowable.
3. Select the domain method and record the material decision behind it before drafting.
4. Produce the smallest complete evaluation scorecard; keep facts traceable and placeholders visibly unresolved.
5. Test the result against the decision table, domain quality criteria and anti-slop gate; recover by narrowing or qualifying unsupported portions.
6. Deliver the artefact with evidence, assumptions, unassessed checks and the next approval or verification step.

Outputs

Evidence Produced

Quality Standards

• Preserve the domain guidance and East African market context below; replace it only when the requester names another market.
• Use British English unless the target language or market requires otherwise, and verify names, figures, quotations and platform rules before use.
• Make the key choice visible, cover failure and edge cases, and keep the result ready for its named consumer.
• Run the repository's anti-ai-slop ship gate; a blocking factual, cultural, safety or permission defect stops release.

Anti-Patterns

• Writing before the objective and audience are known. Fix: stop and obtain the missing brief fields.
• Reusing a neighbouring skill's template because the headings look similar. Fix: route by the requested evaluation scorecard, not vocabulary overlap.
• Adding a price, result, quotation, platform limit or cultural claim without a traceable source. Fix: verify it or qualify/remove it.
• Treating missing access, evidence or native-language review as approval. Fix: mark the check not assessed and narrow the result.
• Publishing, sending, spending or changing a live account from drafting authority alone. Fix: obtain explicit action-specific authority and retain the approval record.

References

• ai-readiness-diagnostic is the nearest routing comparison for this skill.
• Repository agent guide defines the engine-wide market, safety and anti-slop gates.

Framework: Venkatesan, R. and Lecinski, J. (2026) The AI Marketing Canvas, 2nd ed. Stanford Business Books.

Position in the Canvas: Step 2 — Experimentation. Use this skill to select which tools enter the experiment stage. Use meta-ai-tools-audit as the reference catalogue when the client does not yet have a shortlist. Use playbook-ai-automation-workflow once a tool has been selected and an automation build is underway.

Required Input

Ask for all of the following before generating any output:

1. Client business name — exact trading name
2. Industry — sector and sub-sector (e.g. retail > fashion, NGO > health)
3. Country / city — default is Uganda; note if outside EA
4. Current Canvas step — confirm from ai-readiness-diagnostic output
5. Specific marketing problem to solve — a named task, not "we want AI" (e.g. "write 20 Instagram captions per month", "qualify leads from our Facebook ads", "send WhatsApp order confirmations automatically")
6. Tools being evaluated — names of up to 4 tools the client is considering; if the client has no shortlist, prompt them to run meta-ai-tools-audit first
7. Current tech stack — list what the client already uses: CRM, email platform, social scheduler, payment platform, website CMS, WhatsApp setup
8. Monthly tool budget in UGX — if unknown, ask for a range
9. Team size and technical level — number of people who will use the tool and their comfort level (non-technical / basic digital skills / comfortable with no-code tools / has developer support)

Do not proceed until all nine inputs are confirmed.

Evaluation Framework — 8 Factors

Apply all 8 factors to every tool. Do not produce partial scorecards. Score each factor 1–5 using the criteria below. Sum to a total out of 40.

Factor 1 — Use Case Fit

Does the tool address the specific, named marketing problem the client stated?

Red flag: any vendor positioning the tool as an all-in-one AI platform without a primary specialisation. Record this explicitly in the scorecard.

Factor 2 — Data Requirements

What data does the tool need to function, and does the client have it?

Flag any data requirement that may conflict with the Uganda Data Protection and Privacy Act 2019 (PDPA 2019). In particular: personal data collection, third-party data sharing, cross-border data transfer, and automated profiling of individuals. Record the flag in the scorecard even if the score is high.

Factor 3 — Integration Compatibility

Does the tool connect to what the client already uses?

For any use case involving WhatsApp or SMS, check for Africa's Talking integration and note the result explicitly. Africa's Talking is the default recommendation for EA WhatsApp/SMS automation.

Factor 4 — EA Market Accessibility

Can the client actually buy, trial, and use this tool from Uganda or East Africa?

Always convert the pricing to UGX using the current approximate rate and state it explicitly. Note EAT (UTC+3) customer support availability if known — not a scoring criterion but record it as context.

Factor 5 — Team Capability Match

Can the client's team use this without specialist skills or paid training?

Factor 6 — Output Quality

Based on trial, demo, or available samples: is the AI output usable for the client's stated marketing task?

Apply the ai-content-humaniser standard when evaluating any tool that generates written content. If a trial is not possible before scoring, note this as a limitation and flag that output quality must be verified before the Step 2 experiment launches.

Factor 7 — Vendor Stability

Is this a vendor the client can rely on for at least 12 months?

Assess honestly. Do not recommend a tool with a score of 1 or 2 on this factor unless the client has technical capacity to migrate quickly and the tool cost is zero.

Factor 8 — Total Cost of Ownership

What is the real monthly cost once the trial period ends?

Always itemise: base plan cost, per-seat fees if any, usage limits and overage rates, annual vs monthly billing difference, and the total estimated monthly cost in UGX. Use the client's stated budget as the benchmark.

Scoring and Decision Rules

Sum the 8 factor scores for a total out of 40. Apply the following decision thresholds:

Every deferred tool must include: (a) the primary reason for deferral stated in one sentence, and (b) a named alternative tool to evaluate in its place.

Output Structure

Produce all five sections below. Do not omit any section.

Section 1 — Tool Evaluation Scorecards

One scorecard per tool. Use this format for each:

## [Tool Name]
**Use case being evaluated:** [restate the client's named marketing problem]

| Factor | Score (/5) | Notes |
|--------|-----------|-------|
| 1. Use Case Fit | | |
| 2. Data Requirements | | |
| 3. Integration Compatibility | | |
| 4. EA Market Accessibility | | |
| 5. Team Capability Match | | |
| 6. Output Quality | | |
| 7. Vendor Stability | | |
| 8. Total Cost of Ownership | | |
| **Total** | **/40** | |

**Decision:** Recommended / Conditional / Deferred

**Key strengths:**
- [Point 1]
- [Point 2]

**Key concerns:**
- [Point 1]
- [Point 2]

**PDPA 2019 flag:** [Yes — describe the specific data concern / No]

Section 2 — Recommended Shortlist

List all tools scoring 24 or above. For conditional tools, state explicitly which factor(s) must be addressed and how before the experiment launches.

Section 3 — Deferred Tools

List all tools scoring below 24. For each: one-sentence reason for deferral and one named alternative tool.

Section 4 — 30-Day Experiment Briefs

Produce one experiment brief for every recommended tool (score 24+). Use this format:

## 30-Day Experiment Brief — [Tool Name]
**Hypothesis:** If we use [tool name] for [specific task], we expect
[measurable result] within 30 days.

**Baseline metric:** [What is the current state before the tool? Give a number
or describe how to establish one in Week 1.]

**Success metric:** [The specific, measurable target at Day 30. Apply SMART
criteria: Specific, Measurable, Achievable, Relevant, Time-bound.]

**Week 1 — Setup and baseline**
- Actions: [what to do]
- Review: [what to check]

**Week 2 — First outputs**
- Actions: [what to do]
- Review: [what to check]

**Week 3 — Iteration**
- Actions: [what to do]
- Review: [what to check]

**Week 4 — Evaluate**
- Actions: [what to do]
- Review: [what to check]

**Day 30 Go/No-Go decision criteria:**
- Go: [specific condition that justifies continuing and paying for the tool]
- No-Go: [specific condition that means the experiment failed; state what happens next]

Section 5 — Budget Summary

Produce a single budget table covering all evaluated tools:

| Tool | Free Tier Adequate? | Monthly Cost (USD) | Monthly Cost (UGX) | Decision |
|------|--------------------|--------------------|---------------------|----------|
| | | | | |

State the USD/UGX conversion rate used. State the client's declared budget and whether the recommended shortlist is within budget. If the shortlist exceeds budget, recommend which single tool to start with and why.

EA-Specific Evaluation Notes

Apply these rules throughout the evaluation:

• Prioritise tools with free tiers. EA clients at Step 2 should not pay for a tool until a 30-day experiment has demonstrated measurable value.
• For any WhatsApp or SMS automation use case, include Africa's Talking in the evaluation automatically, even if the client did not name it. Note it as a recommended addition.
• Flag any tool that requires a developer, API key setup, or command-line access unless the client confirmed they have technical support.
• Flag any tool that requires high or consistent bandwidth for field team usage. Prefer offline-capable or low-bandwidth modes where field staff are expected to use the tool.
• Convert all USD pricing to UGX. Use the approximate rate at time of evaluation and state it explicitly.
• Do not recommend any tool that fails the Factor 4 (EA Market Accessibility) assessment unless the client has explicitly confirmed they can access it and pay for it.

Cross-References

Tool Categories Reference

Use this section when the client has no shortlist and needs guidance on which category of AI tool to evaluate. Cross-reference with meta-ai-tools-audit for the full catalogue.

Category 1: RAG (Retrieval-Augmented Generation) Tools

Tools that connect LLMs to client-specific knowledge bases for accurate, on-brand output:

Evaluation criteria: How easily can client documents be uploaded? Does the tool maintain source attribution? Can multiple team members access the same knowledge base?

Category 2: Synthetic Research and Persona Tools

Tools for generating AI-simulated audience research when primary fieldwork is unavailable or too costly:

EA note: For most Ugandan SME clients, prompted persona generation via Claude or ChatGPT is the most accessible option. Supernatural AI and Glimpse are better suited to multinational clients with larger research budgets.

Category 3: Agentic AI and Automation Tools

Tools for building autonomous or semi-autonomous marketing agents:

EA recommendation: n8n (self-hosted on a local server) combined with the Claude API is the most cost-effective agentic stack for EA-based consultancies. Zapier is the most accessible for clients with no technical resources.

Quality Criteria

• All 8 factors scored for every tool with written notes — no blank cells, no partial scorecards.
• EA accessibility assessed explicitly: name the payment method, name the pricing tier, state the UGX equivalent.
• Every recommended tool (score 24+) is paired with a complete 30-day experiment brief including a measurable hypothesis and Day 30 Go/No-Go criteria.
• Every deferred tool includes a one-sentence reason and a named alternative.
• The Uganda Data Protection and Privacy Act 2019 is flagged wherever the tool collects, processes, or transfers personal data — even when the overall score is high.
• The budget summary is produced in UGX, references the client's declared budget, and resolves conflicts where the shortlist exceeds budget.
• Output is a decision document a non-technical business owner can act on without further interpretation.
• Vendor stability is assessed honestly: do not recommend a tool scoring 1 or 2 on Factor 7 without explicitly noting the risk and the mitigation.

References

• Venkatesan, R. and Lecinski, J. (2026) The AI Marketing Canvas, 2nd ed. Stanford Business Books.
• Sweenor, D. and Mulkers, T. (2024) AI-Powered Business Intelligence. O'Reilly Media.
• Nayebi, H. (2025) Generative AI for Product and Marketing Teams. Packt Publishing.
• Bodnar, K. and Cohen, J. (2012) The B2B Social Media Book. Wiley.
• Chaffey, D. (2024) Digital Marketing: Strategy, Implementation and Practice. Pearson.