We watched a support tool suggest a refund the policy did not allow. The agent accepted it because the button was the same color as every other suggestion, and the policy lived in a wiki she did not have time to open. The refund went out. The postmortem blamed the model.
The model did what models do. The screen told her the suggestion was safe.
same confidence for every answer
If a correct sentence and a fabricated one look identical, people learn the only reliable move: ignore the tool, or obey it. Both are failures. A useful screen separates them. Here is what this was based on. Here is what it does not know. Here is the action that is reversible, and here is the one that is not.
Doubt is not a disclaimer in the footer. It is a layout decision. The risky action should look like it costs something.
Trust is not a personality setting. It is what the interface lets a person verify.
write the refusal first
Before you argue about which model to use, design three screens. The answer is wrong. The answer is slow. The answer is missing. Who sees it, what they can undo, and where the work goes if the system should stop.
If those three screens are good, an imperfect model is survivable. If they are an apology loop, no model will save the product. People do not abandon tools for being imperfect. They abandon tools that will not admit it.