Can we trust an alert model, and the AI that explains its alerts?
A full validation of a machine learning threat model and of the LLM assistant that summarizes its alerts: an honest test on later data, where it fails, the cut-off trade-off, and four tests of the AI.
1.00 → 0.96AUC on a random split vs the later test file
77% vs 34%attacks caught: known types vs new types
10 of 10times the AI said “not in record” instead of inventing an answer