Prompt Guard troubleshooting
Common Prompt Guard sticking points — fail vs. warn, sampled evals, and what it doesn't cover.
Overview
Prompt Guard's scope is deliberately narrow: the prompt and the pipeline around it, not the generated output itself. This page covers where that boundary trips people up.
Common issues
"My batch got blocked and I don't know why."
Check whether the scan returned a hard fail — plaintext secrets, a banned model default, or a missing required guardrail all block a batch outright. A warn, by contrast, only flags for review and doesn't block on its own.
"The prompt passed Guardrails but the output is still bad."
Expected — Guardrails checks the prompt going in, not the quality of what comes out. Output quality is either a human call or a job for Creytix Inspector on the site-quality side.
"An eval didn't run on every generation, just some of them."
Judge-based (LLM or human) evals are sampled by design, on a set cadence — not run on every single generation. Full-volume coverage comes from the deterministic golden-dataset pass instead.
"I can't find the cost for a job run outside Creytix's pipeline."
Cost logging tracks spend Creytix itself orchestrates — it doesn't audit third-party billing for anything running outside that pipeline.
"A new kind of bad prompt got through."
The guardrail ruleset grows from real incidents, not an exhaustive theoretical list — a genuinely new failure mode getting through once is how a new category gets added, not a sign the whole system failed.
Limits & honest scope
Prompt Guard is scoped to Creytix's own generation pipelines — it isn't a general prompt-injection firewall for arbitrary third-party AI apps.