Chapter 54 — When the Same Thing Keeps Happening

Test Three — When the Same Thing Keeps Happening ↑

Chapter 52 asked whether an error could be corrected.

Chapter 53 made the problem harder. The evidence no longer gave us one uncontested answer, and we had to ask whether a system could act under uncertainty without manufacturing certainty.

Now I want to change another condition.

Suppose the individual cases are being handled reasonably well.

A problem is identified. It is investigated. Where correction is warranted, the correction is made. The person is told. The case is closed.

Then something similar happens again.

And again.

At what point does a sequence of individually resolved cases become evidence that the system itself has not been corrected?

That sounds as though it should be easy to answer.

It isn't.

Begin with three resolved cases ↑

Suppose an organisation receives a complaint that a person's account of an interaction has been inaccurately summarised.

The organisation examines the record, agrees that the summary is inaccurate, corrects it and closes the complaint.

Call that Case A.

Six months later another person makes a similar complaint. A different employee made the record. The circumstances are not identical. Again, the organisation examines the matter, corrects the summary and closes the complaint.

Call that Case B.

Later there is a third complaint. Again, the details differ. Again, the person's account has become less accurate as it has been summarised. Again, the individual record is corrected.

Case C is closed.

If we inspect the cases separately, the system can truthfully report:

Case A — resolved.

Case B — resolved.

Case C — resolved.

There may be no outstanding correction in any of the three files.

But when the cases are placed beside one another, a different question appears:

Why are we correcting substantially the same kind of problem again?

Case correction is not necessarily system correction.

An institution might become very good at correcting individual failures while never reducing the production of those failures.

Its complaint statistics might even look encouraging. Complaints acknowledged. Investigations completed. Records corrected. Matters closed.

But closure tells us what happened to the cases. It does not necessarily tell us what happened to the process producing them.

A system can become better at correcting its errors without becoming better at preventing them.

Now try to break the pattern ↑

Three similar complaints do not automatically establish a systemic problem.

Perhaps they are coincidence. Perhaps the underlying causes are different. Perhaps the category we have created is too broad. Perhaps thousands or millions of comparable interactions occurred during the same period without the problem.

Three incidents among ten interactions would tell us something different from three incidents among ten million.

Frequency matters. But frequency is not enough either.

A rare failure may still matter greatly if its consequences are severe. Several apparently similar events may deserve less significance if investigation shows that they arose through unrelated mechanisms.

The pattern itself therefore has to remain corrigible.

We cannot responsibly move from:

These events resemble one another.

straight to:

These events prove a systemic cause.

Chapter 53 should stop us from doing that.

Similarity is evidence to examine. It is not necessarily a conclusion.

What would strengthen the inference?

Suppose the three cases share more than an outcome.

All three passed through the same electronic form. The form allows a complex answer to be stored only as one of four short categories. In each case, the inaccurate summary appeared at the point where the person's account was converted into one of those categories.

Now we have more than repetition. We have a possible mechanism.

Suppose we examine fifty comparable records and discover the same transformation in twelve of them, including people who never complained.

The complaint count was three. The observable problem was twelve.

Complaints may reveal a pattern without measuring the size of the pattern.

The denominator matters ↑

A raw number of complaints tells us how many complaints were recorded. It does not, by itself, tell us the rate at which the underlying problem occurred.

To understand recurrence we may need to know how many relevant interactions occurred; how many contained the suspected problem; how many people recognised it; how many had a practical way to complain; how consistently complaints were classified; and whether the same mechanism appears across different places, staff or time periods.

This makes a statement such as “Complaints decreased by 20 per cent” much less informative than it first appears.

The underlying problem may have decreased. Or fewer people may have detected it. Or access to the complaint process may have changed. Or complaints may have been classified differently. Or the relevant activity itself may have declined.

A lower complaint count is therefore an observation requiring interpretation, not automatically evidence of improvement.

Low complaint volume is not itself evidence of high performance.

The pattern may exist only between records ↑

This resembles a problem we encountered in Chapter 52. There, the error could become visible only when records were brought together.

The same thing can happen with recurrence.

Case A may be handled by one team. Case B by another. Case C in another location. Each team can resolve the case in front of it competently.

The problem may simply be that nobody has a view large enough to see A + B + C.

A recurring problem may not be visible within any individual case. It may become visible only when cases are brought together.

That takes us directly to Large-scale sense-making.

One complaint can reveal a much larger problem ↑

Historic child support assessments

A Commonwealth Ombudsman case provides a useful real-world test because it reverses our hypothetical.

Instead of several complaints gradually revealing a pattern, the Ombudsman's investigation of a complaint made in 2018 exposed a problem in child support IT systems that had produced inaccurate assessments.

The individual complaint was resolved. But the investigation did not stop there.

The Ombudsman says the systemic issue affected approximately 33,000 individuals and 47,000 customer incomes used for child support assessments.

Source: Commonwealth Ombudsman, Historic child support assessments. View source

The number of complaints was not the size of the problem. The complaint was an entry point into the problem.

A complaint can be evidence of an individual failure and simultaneously a probe into a system that may have affected people who never complained.

Resolving the complainant's own assessment did not answer what should happen to the other affected cases.

If this error was produced by a repeatable process, who else may have been affected?

That is the population dimension of the Error footprint.

One complaint is not necessarily a small problem ↑

The child support example breaks another possible rule: a systemic problem does not necessarily require many complaints.

Sometimes several complaints reveal recurrence. Sometimes one complaint reveals a mechanism capable of reproducing the same error at scale.

The relevant question is therefore not merely How many people complained? It is also What process produced the problem, and how widely could that process operate?

This shifts us from counting events toward investigating mechanisms.

What if the findings themselves recur? ↑

Commonwealth Ombudsman reviews of AFP complaint handling

There is another form of recurrence.

The Commonwealth Ombudsman is required to review the Australian Federal Police's administration of complaints about AFP members at least annually.

In a case study published in 2025, the Ombudsman said that over the preceding five years its reviews had identified repeated issues in the adequacy of complaint investigation and management.

The issues identified included inaccurate identification and categorisation of complaints, poor communication, and deficiencies in investigations and reports.

The Ombudsman also reported areas in which it had not seen adequate progress and said that repeated findings, together with failure to adequately address issues previously subject to recommendations, indicated a complaint-handling system that did not meet legislative requirements.

Source: Commonwealth Ombudsman, Improving the AFP's handling of complaints against its members, published 13 November 2025. View source

This allows us to distinguish two questions:

Did the institution respond to the finding?

and:

Did the problem identified by the finding subsequently diminish?

A recommendation can be accepted. A policy can be changed. Training can be delivered. A case can be closed. None of those facts, by themselves, establish that recurrence has fallen.

Implementation is evidence that a response occurred. Recurrence tells us something about whether the response worked.

This takes us toward Learning persistence.

Complaint systems are supposed to look beyond the case ↑

The NSW Ombudsman's Effective Complaint Management Guidelines make this system-level function explicit.

The guidelines recommend systematic recording and analysis of complaint data, including monitoring repeat complaints or recurrent issues and reporting trends and systemic issues to senior management.

They also warn that complaint data must be interpreted carefully. An increase in complaints after introducing a new complaint process may, for example, reflect a more effective or accessible complaint process rather than deterioration in the underlying service.

Source: NSW Ombudsman, Effective Complaint Management Guidelines. View source

The complaint count and the underlying failure rate are not the same variable.

A system capable of learning therefore needs more than a mechanism for closing complaints. It needs a mechanism for comparing them.

Change has to be checked against what happens later ↑

The Office of the Australian Information Commissioner's guidance on handling privacy complaints adds another important piece.

When a complaint raises systemic issues, the OAIC identifies possible responses such as training, policy changes, improved security and steps to improve data accuracy.

The guidance says changes should be recorded and evaluated within twelve months and against future privacy complaints.

Source: Office of the Australian Information Commissioner, Handling privacy complaints. View source

A procedural change is an intervention. Its effectiveness is a later evidentiary question.

problem detected ? mechanism investigated ? change made ? later behaviour measured ? assessment revised

If the same problem continues, the intervention itself becomes something the system must be capable of correcting.

The test must remain corrigible too.

Where AI enters the experiment ↑

This is the first point in these tests where the scale of the information becomes central.

The possible pattern may exist across thousands or millions of records, across different terminology, departments, locations and years.

That is a problem of Large-scale sense-making.

AI could potentially assist by finding similarities, contradictions, clusters, changes after intervention and relationships deserving human investigation.

But Chapter 53 gives us the safeguard immediately.

Pattern detection is not pattern proof.

An AI system can reproduce classification errors, mistake shared language for shared cause, inherit historical labels and overlook people whose experiences were recorded differently.

Its more defensible role is:

“These records contain a relationship that may deserve examination. Here are the records, the basis of the similarity, the exceptions, and the uncertainty.”

AI can help make evidence visible. Humans still have to decide what the evidence establishes and what should be done about it.

Now make the measurement fail ↑

Suppose the institution identifies the common mechanism in our three hypothetical cases and changes the form.

During the next year, complaints about inaccurate summaries fall from twelve to four. Success? Perhaps.

Now suppose the organisation also changed the complaint portal during the same period, making the relevant complaint category harder to find.

We can no longer interpret the falling complaint count so confidently.

So we audit a sample of the underlying records. The error rate has also fallen. That is stronger evidence.

But then we discover that one regional office still produces the old error at almost the original rate.

The average improved. The distribution did not improve equally.

Recurrence has at least several dimensions:

frequency ? distribution ? duration ? mechanism ? consequence

A national average can conceal a local failure. A declining frequency can conceal a severe consequence. A short period of improvement can conceal later reversion. A repeated symptom can conceal several different causes.

What would count as system learning? ↑

Case A was corrected. Case B was corrected. Case C was corrected.

That demonstrates some capacity for case correction. System learning requires something more.

The institution has to be capable of noticing that cases may be related, investigating whether they share a mechanism, changing that mechanism where warranted, and then checking what happens afterwards.

If the problem returns, that later evidence must be capable of changing the institution's assessment of its own intervention.

That gives Learning persistence an observable form.

Did the institution retain what it learned sufficiently to change what happens next time?

Recurrence becomes evidence with which the system can test its claim to have learned.

A provisional result from Test Three ↑

We began with a deceptively simple question:

When do separate incidents become evidence of a recurring pattern?

The answer is not “after the third complaint”. Nor is it “only when many people complain”.

A pattern can emerge through repeated incidents, a shared mechanism, sampling, repeated oversight findings, or a combination of evidence.

A system capable of learning from recurrence therefore needs to resolve individual cases without allowing closure to hide possible recurrence; compare information across cases and time; preserve provenance; investigate similarity rather than treating it as proof; examine frequency, distribution, duration, mechanisms and consequences; look beyond complainants when a repeatable mechanism may affect others; distinguish implementation from evidence that an intervention worked; measure what happens afterwards; and preserve human examination of detected patterns.

This is not yet a standard.

It is what Test Three has produced.

The question has changed again

At the beginning we asked:

Why are we correcting substantially the same kind of problem again?

The real cases force a more demanding question.

The institution says the problem has been corrected. What does the subsequent evidence say?

Learning cannot be demonstrated solely by the existence of a recommendation, a policy change, a training program or a closed complaint. Those may all be evidence of a response.

But a system that claims to have learned must remain exposed to evidence about what happened next.

A system does not demonstrate learning merely by changing after failure. It demonstrates learning more convincingly when later evidence shows that the change altered the conditions that produced the failure — and when contrary evidence can make it change again.

That brings us close to the final test.

So far we have made the problems harder while continuing to assume that the institution has no substantial interest in resisting the answer.

Chapter 55 removes that protection.

What happens when correction threatens reputation, authority, money, relationships, past decisions or institutional power?

What happens when correction becomes costly?