The Human in the Loop

Essay 1 of 5, Decision-Native 2040 series. Draft 1, 2026-04-23.

In 2026, when this future was first being imagined, Raj Reddy, a Turing laureate who had spent his career building autonomous systems, described the human role in an AI-abundant economy generously. The work that would remain, he said, would be “a human in the loop who will essentially fix bugs or provide guidance,” evaluating automated outputs for reliability, availability, dependability, and security.¹ The framing is accurate. What it does not ask is what happens to the human after ten years of not being asked to notice anything the system has not already noticed for them.

This is what that human looks like in 2040.

8:42 on a Tuesday

Daniel Fischer is at his desk in the Zurich office by 8:42. The coffee machine has finally been repaired. He pours his second cup of the day and opens his decision stream.

Fourteen items are waiting. All but one arrived overnight, recommendations from the credit risk engine about exposures that shifted during the Asian session. Each item carries the system’s recommendation, the confidence band, three comparable decisions from the last eighteen months, and a justification of about five hundred words.

Daniel is a mid-cap credit analyst. He has been doing this job in various forms for eleven years. The firm deployed its current decision engine in 2035. By 2037 the role had been rewritten around it. Daniel’s team of four covers the exposures that an earlier team of sixteen used to cover. The team’s Q3 and Q4 numbers are ahead of plan.

He opens the first item.

Velser

Velser Industries. A long-standing client, nine years on the books. The system recommends reducing exposure by 15% on the back of industry headwinds in the Austrian machinery sector. The comparables cited are crisp. The confidence band is tight. The justification ends with a sentence Daniel could have written himself.

He approves in thirty seconds.

As he approves, something crosses his mind. Velser’s CEO called him at 10pm in late 2039, during a slowdown, sounding worried. Daniel had stayed on the phone for nearly an hour. The call had not been a credit decision. It had been a conversation. He remembers the feeling of it more than its content.

The memory crosses, and then it is gone. By 9:15 he does not remember that the memory had crossed.

The morning

The next three items move quickly. A cross-sell recommendation for a long-standing commercial client. A margin adjustment on a trade facility. A rating reaffirmation on a mid-market exposure. Each takes under a minute. The system’s confidence is high on all three. Daniel approves.

At 10:15 an overdraft case arrives that the system has flagged low-confidence. Daniel schedules a call with the sector head for 10:30. On the call, he presents the case as the system has summarised it. The sector head offers a minor adjustment. Daniel implements the adjustment. The call ends in twelve minutes.

By 11 he has processed nine items and has some time. He drafts a note about workflow improvements for the Monday planning meeting.

11:30

At 11:30 a harder item arrives. A mid-cap exposure in Romania. The market data is thin and the comparables are messy. The system has flagged low confidence, human review recommended. The confidence band is wide enough that the recommendation could go either way.

Daniel could read the primary source documents the system has summarised. He could call the Romanian desk lead. He could reconstruct the credit from scratch, which would take him perhaps three hours, because the Romanian mid-cap market is one he has not followed directly for two years.

He schedules a fifteen-minute call with the Romanian desk lead. The desk lead provides a brief view. The view mostly aligns with what the system has suggested. Daniel implements the system’s recommendation with a small adjustment for sector balance. He does not read the primary sources. He does not feel that he needs to.

The call is over by 11:52.

Lunch

Daniel eats at the cafeteria on the second floor with two colleagues from operations. They talk about a hiking weekend in the Glarus Alps. At 12:40 he takes a phone call from his sister. They talk for eleven minutes about the live-in carer for their mother, whose dementia has been harder since spring.

He returns to his desk at 12:58. A small residual tension is with him. He would not call it tension. He would call it having had a difficult conversation. He opens the stream.

The afternoon

Four items between 1pm and 2:15pm. All routine. His throughput is slightly above the team average for a Tuesday. He notices this with some pleasure.

At 3:30 an older case reopens. A client flagged by the AML engine for additional review. Daniel reads the system’s compiled file, which runs to six pages and synthesises the last year of the client’s transaction patterns, external ownership changes, and news-feed mentions. The system recommends escalating to compliance. Daniel reads the file, agrees, and approves the escalation path the system has suggested.

He does not ask what the system did not include in the file.

At 4:45 his decisions for the day are cleared. He opens the portfolio-review dashboard and runs the end-of-day checks. His books are green on risk, green on compliance, green on throughput. Q4 is tracking ahead of plan.

5:30

Daniel leaves the office at 5:32. He walks to the tram stop. He thinks, briefly, about the sister conversation, and about his daughter’s mathematics homework. He does not think about any of the decisions he made today. None of them require him to.

What Daniel did not notice

Daniel did not notice that Velser’s recommendation had arrived with a comparables set that omitted the 2039 stress that had led to the late-night call. The stress was outside the system’s eighteen-month comparables window. The recommendation was still correct. The omission changed nothing. Daniel did not see the omission because he had not reached for the comparables window parameter himself.

Daniel did not notice that the Romanian desk lead on the 11:30 call was a 28-year-old analyst who had been in the desk seat for eight months. The analyst’s view had aligned with the system’s because the analyst was in the process of being shaped by the system’s cues, week by week. The alignment was not independent confirmation. Daniel felt it as confirmation.

Neither of these omissions produced a wrong decision on Tuesday. Most decision-native Tuesdays produce no wrong decisions. The decisions are within the band of acceptable. The outcomes will be acceptable. The books will remain green.

The loss Daniel will not notice, over a period of five to ten years, is that the thing that used to happen when a thought crossed his mind at 9:08 in the morning, the thought about a late-night phone call in 2039, stopped happening somewhere in the fourth or fifth year of the decision engine’s deployment. The thing that made the thought useful was that it would sometimes make him pause. The pause would sometimes make him call the desk lead himself, or read the primary source, or pick up the phone to Velser’s CEO. The pauses had become rare by 2038. By 2040 they had become absent.

Sources

1. Raj Reddy, “The Future of AI: Doomers vs. Abundance,” talk, 2026. Reddy is University Professor of Computer Science and Robotics at Carnegie Mellon and a 1994 Turing Award laureate. In the talk he predicts that in an AI-abundant future, human labour will persist in the role of “a human in the loop who will essentially fix bugs or provide guidance,” evaluating automated outputs for reliability, availability, dependability, and security.

A note on this essay. Daniel Fischer and Velser Industries are fictional; the Zurich office is a composite. The decision stream described here is a projection into 2040 of credit-risk decision systems in use today at European banks, the kind built around Moody’s Analytics, S&P Capital IQ, and the credit-scoring platforms of vendors like Finastra and Fenergo. The Reddy quotation is verbatim from the 2026 talk.