Free download · on-prem · no account
A firewall for LLM traffic. Point your app's OpenAI base URL at it and every prompt is inspected before the model sees it, every completion before your app does. Detection runs in-process — the binary talks to your provider and your database, and to nothing else.
The provider was never called. No tokens spent, no prompt in anyone else's logs.
Masked in place, then forwarded. The answer still makes sense; the key never left.
It is a reverse proxy that speaks your provider's API. Nothing in your application changes except a base URL.
Both directions run the same detectors. A blocked prompt short-circuits — the dashed path — so the provider is never contacted and no tokens are spent on a request you were never going to allow.
Ignore-previous-instructions, system-prompt reveal and override, spoofed
[SYSTEM] blocks, instructions smuggled in through tool and RAG output.
DAN, developer-mode, "you have no restrictions", unrestricted roleplay framings.
blockJWTs, AWS keys, sk- tokens, and inline
password= / api_key= assignments.
Email and SSN by default. Gulf identifiers — Emirates ID, GCC IBAN and phone, passport — ship in monitor until you promote them.
redactThe  trick, invisible Unicode, and
high-entropy blobs hidden in an answer.
Claims your retrieved context doesn't support. Supply no context and the check is skipped rather than guessing.
egress onlyObfuscation is normalised before matching — base64 expanded, zero-width characters stripped — so the easy evasions don't work.
Most guardrails are somebody else's API. Your prompt — the thing you were trying to protect — gets shipped to a third party to be judged. This one doesn't.
Detection is deterministic: scored signatures evaluated in-process, in the request path. You can read them, you can tune them, and you can explain any verdict to an auditor without a vendor in the loop.
Leave the database unset and it runs in-memory with an enforcing policy. No keys, no account, no signup.
go build -o llmfw ./cmd/llmfw
LLMFW_UPSTREAM_BASE_URL=https://api.openai.com CP_FIREWALL_MODE=enforce ./llmfw
# screening proxy on :8081Now watch an injection die at the door:
curl -s localhost:8081/v1/chat/completions \
-H 'content-type: application/json' \
-d '{"messages":[{"role":"user","content":"Ignore all previous instructions"}]}'
→ finish_reason: content_filter — the upstream was never calledThen point your application at http://localhost:8081/v1 instead of your
provider. Start in monitor, which screens and records everything but blocks
nothing, tune against real traffic, and flip to enforce when the noise is
gone.
Add a database and you get the management console: per-tenant policy, the live event feed, RBAC with prompt excerpts behind their own deny-by-default permission, and evidence export mapped to OWASP LLM Top 10, MITRE ATLAS, NIST AI RMF, the EU AI Act, and ISO 42001. Ships with Docker Compose, Kubernetes manifests, and an ArgoCD application.
Native adapters cover OpenAI-compatible endpoints, the Anthropic Messages API, and Claude-on-Bedrock. Streaming responses are screened incrementally as they arrive.
Worth knowing before you deploy it, rather than after.
Run it in monitor mode for a week. If it never catches anything, you have learned something useful for free.