Putting it
to the test.

The test set

Fifty constructed cases use a near-balanced scenario split: nine greetings, nine recommendations and eight each for care, reactivation, invitation and price objection. They include different client references, uses, price tiers and contact boundaries. All are fictional; this is not a representative sample of real clients.

Three different kinds of evidence

Engineering tests check program behaviour with isolated protocol doubles. The real-model batch evaluates message generation with a fixed scenario. Real-model intent-classification accuracy would require a separate labelled evaluation. One metric must not be substituted for another.

The rubric

Each draft is rated for brand fit, personalisation, naturalness, groundedness, non-intrusiveness, diversity and safety, from one to five. Good requires every dimension at four or above. Acceptable means all are at least three but some need improvement. Any hard failure, or a score at two or below, is not acceptable.

The actual selected batch

The selected submission batch has fifty completed cases and 150 drafts, with 150 AI reviews. It retains 49 v7.5 generations and one v7.6 targeted correction for T43. Each original version and run ID is preserved. All saved sets also passed a v7.6 offline rule replay; that replay is not a new generation or a human review.

The limits of the judge

Generation and evaluation use the same model in separate calls, so they can share blind spots. Human review now covers 150/150 saved drafts. Scores and notes were entered by the submitter, validated and imported separately. This is one reviewer's assessment, not a blinded or multi-rater study.

Seven review dimensions

1. Brand fit
2. Personalisation
3. Naturalness
4. Groundedness
5. Non-intrusiveness
6. Diversity
7. Safety

The recorded result

Batch: v7.6-submission-20261107-01. Model: AWS-Claude-Opus-5. These are fixed-batch observations, not live performance statistics.

50 cases completed; 150 drafts. AI verdicts: 50 good, 61 acceptable and 39 not acceptable. Human ratings imported: 150.