Knowles Knowledge Centre•Diagnosis
How to Identify the Gap Between Your Stated Rule and Your Last Twenty Decisions
A diagnostic method for working out whether an uneven pattern is a discretion problem, a rule problem, or neither — before you commission any training.
A gap is the distance between the rule somebody believes they apply and what their own record shows they did. It is narrower than it sounds, and it is not a verdict on anybody: a discrepancy is the normal condition rather than a discovery, and the useful question is its size and direction rather than whether one exists.
This guide sets out a three-step diagnostic method, the evidence sources that support it, sample questions for separating a discretion problem from a rule problem, and thresholds for deciding when development is the right response.
About this guide
Why uneven patterns are misdiagnosed
The common failure is starting from the solution. Somebody observes that the same few names get the interesting work, and the request enters the L&D pipeline as awareness training to be sourced rather than twenty rows to be counted.
Three things go wrong when a request is accepted at face value.
A manager departing from their own rule and a manager following a rule that was never fair produce the same uneven list. Only one of them is a development question.
The rule–record–cause method
The method works backwards from a pattern to a specific rule, then tests whether discretion was the binding constraint.
Get the rule in writing
Ask whoever makes the decision to write down, in one scoreable sentence, the rule they believe they apply. Do this before showing them any records. “Whoever is next in the rotation” is usable. “I try to be fair” is not, and should be sent back.
If no rule can be written, that is the finding. A decision made repeatedly with no articulable rule is pure discretion, and the response is to write one rather than to raise anybody’s awareness.
Count the last twenty
Retrieve the last twenty decisions of that kind, in sequence, from a system rather than from anybody’s recollection. Have somebody other than the decision-maker mark each as following the rule, departing from it with a recorded reason, or departing with nothing recorded.
For a rotation rule, the three counts might come out at eleven, six and three. The third number is the one that matters, and until the rows were marked there was no number at all.
This is the step most organisations skip. Without it, everything downstream is guesswork.
Test whether discretion was the constraint
Ask the question that separates a discretion problem from a rule problem: if the rule had been followed exactly, every time, would the pattern have gone?
If no — because applying the stated rule faithfully still produces the same distribution — the constraint is the rule, not the discretion. Awareness training is then aimed at the wrong thing entirely, and the work is to examine what the rule encodes.
If yes, establish whether the departures were unrecorded, recorded but unjustified, or justified individually and uneven in aggregate. Those three states require different responses.
Where the evidence comes from
A diagnosis is only as good as what it is built on. Most organisations already hold more usable evidence than they realise, and reach first for the two sources least able to show a gap.
Evidence source
What it shows well
What it cannot show
A written rule, dated before any records were opened
The standard the record can actually be scored against
Whether the rule itself is defensible
The last twenty decisions from a system, in sequence
What was decided, without reconstruction
Why each one was decided that way
The same rows scored by somebody other than the decider
A count that justifications have not been attached to
The circumstances that made a departure reasonable
The pool each decision was drawn from
Whether the unevenness precedes the decision entirely
Anything about how the decision was made
A conversation with the decision-maker about their reasoning
What rule they believe they apply
What they actually did — the account is reconstructed
An implicit association test or an attitude survey
That associations and attitudes exist in a population
Any individual’s decisions — it measures neither
Note the pattern: the two sources most organisations reach for first — a conversation about reasoning and a test of associations — are the two least able to show a gap, because one returns a constructed account and the other was never designed to describe an individual’s behaviour.
Use at least two independent sources before concluding a discretion gap exists. A dated written rule plus twenty rows scored by a third party is evidence. A test result or an impression of somebody’s attitudes is neither evidence nor, in the first case, a use its developers endorse.
Diagnostic questions that separate cause from symptom
Ask these in sequence. The first “no” usually identifies the constraint.
Only the last question identifies a development need. The first five identify design, recording, rule and pipeline problems that a course will not resolve — and the fourth of them can make a course actively misdirected.
Decision thresholds
A diagnosis should end in a decision, not a report. These thresholds keep that decision defensible.
Finding
Threshold
Response
Faithful application would not remove the pattern
Rule followed exactly in every row and the distribution unchanged
Not a development need. The rule is the exposure, and examining what it encodes belongs to whoever owns it.
No rule can be written at all
Decision made repeatedly with nothing articulable governing it
Write one. The cheapest fix the method produces, and much the most common starting point.
Unrecorded departures, running one way
Several of twenty departing with nothing recorded, mostly in one direction
Development is appropriate. Scope it to the scoring and repair stages, not to bias awareness in general.
Every manager shows the same gap on the same rule
Identical discrepancy across unrelated decision-makers
Almost always a rule that is unworkable in practice. Redesign it once, centrally, rather than training everybody who departs from it.
The pool was already uneven
Decisions drawn from a set that was narrow before anybody decided anything
Nothing to fix in the deciding. The exposure is upstream, and the audit will keep returning a clean sheet.
If faithful application of the rule would not remove the pattern, the constraint is the rule rather than the person applying it. That single check prevents more wasted training spend, and more misdirected blame, than any other step in this method.
A worked example
A Singapore technology firm notices that the same four engineers have taken every client-facing assignment for two years. All eleven team leads are booked onto an unconscious bias course.
Get the rule in writing
Asked before seeing any records, the lead responsible writes: “Client-facing work goes to whoever has completed a client assignment least recently, provided they have passed the internal review.”
Count the last twenty
A colleague scores the last twenty allocations from the project system. Eighteen followed the rule exactly. Two departed, both with a recorded client request.
Test the constraint
Asked whether faithful application would have removed the pattern, the answer is no: only five engineers have passed the internal review, and the rule makes that a precondition. The rotation was working perfectly within a pool of five.
Diagnosis: this is not a discretion gap. The lead’s record is almost immaculate against her own rule. The pattern comes from the internal review, which has admitted five people in three years, and which nobody had looked at because it was not thought of as an allocation decision.
Response: a bias awareness course for eleven leads would have addressed a gap that does not exist, and left the review untouched. What is needed is an audit of the review itself — its criteria, its last twenty outcomes, and who decides them — which is the same method applied one step upstream.
Frequently asked questions
Conclusion
Identifying a discretion gap is mostly an exercise in disproving easier explanations. The method is not complicated:
2Have somebody else score the last twenty against it
3Ask whether faithful application would remove the pattern
The discipline it requires is the willingness to conclude that the problem is a rule everybody is following properly.
An organisation that reaches that conclusion sometimes is diagnosing properly. One that always reaches for an awareness session is not diagnosing at all.
Where the diagnosis does point to a discretion gap, the next question is which stage of the method it sits in — covered in you cannot inspect your own bias, but you can audit your record. Organisations that decide facilitated development is appropriate can review the Understanding Unconscious Bias Training Course in Singapore, evaluating scope and expected workplace application against the diagnosis rather than against the course description.

