Core promise
Synthetic checks that behave like customers
Run your prompts hourly or daily against any public HTTPS JSON endpoint, not just live traffic.
Synthetic monitoring for AI output quality
You already know if your server is down. Know when your AI got worse too.
PromptCanary schedules synthetic checks against your LLM endpoint, verifies objective assertions, and alerts you only when quality actually regresses. No SDK. No proxy. No observability platform sprawl.
Core promise
Objective scoring only in v1
JSON validity, schema matching, contains, regex, word count, and latency keep alert noise low.
Core promise
Failure diffs that explain the regression
Compare the latest failure against the last known good run so you know what changed before users complain.
Regression caught
Support bot JSON response
2 of 3 attempts failed JSON schema after a silent model change 47 minutes ago.
last passing
{
"intent": "refund_request",
"priority": "high"
}
latest failing
I'm sorry you're frustrated.
Here are a few options...
Pricing posture
Built for solo builders first
- Free: 2 monitors, daily checks, 7-day history
- Pro: 25 monitors, hourly checks, Slack alerts, 90-day history
- Team later: shared access and faster schedules only after demand shows up