Can AI Teach Problems That Have No Right Answer? Six Years From The Good Place

This is the first chapter of a notebook that will become a book. It is not a results report. It is a record of one question I have carried and turned over for several years. So chapter 1 is not about a product, and not about technology. It starts with where the question came from.

The question is this. In domains that have no right answer, things like ethics, value judgments, or leadership, can AI build a kind of learning that makes a person less certain rather than more? I can write it in one line now, but it did not arrive in these words. It started with a single scene in a sitcom.

A note up front: this notebook does not begin with a finished answer. I have carried this question for years, judged this year that it was worth building, and actually started touching it. I plan to write down, in order, the thoughts that came up, the walls I hit, the outside perspectives I borrowed, and the judgments I changed. So the chapters may not land on clean conclusions. Some will end with an admission that I got stuck. Some will overturn the conclusion of the chapter before. That is the weakness of this format, and also the point I chose on purpose. For this subject, a record published in the middle of being figured out feels more honest than a memoir written after everything is settled. For now, in this first chapter, just where the question came from.

When Chidi Stood in Front of the Trolley

The Netflix sitcom The Good Place is about four dead people who land in the afterlife and try to learn how to be good so they can stay in the good place. At its center is Chidi, a professor of ethics. He taught moral philosophy his whole life, yet could never make a single decision in his own. The fact that the show casts an ethics professor as its lead and builds its plot around “learning to become good” is, in itself, a fairly serious question wrapped in a joke.

There is an episode in that series that takes on the trolley problem. Chidi, the ethics professor, lectures on the trolley dilemma the way he always does. Is it right to pull a lever and kill one person to save five? The problem from the first page of every ethics textbook. Chidi explains it calmly, theoretically, laying out the positions of different schools. If you care about consequences, you save the five. If you care about the rightness of the act itself, then directly killing one person becomes the sticking point. To Chidi, this is a problem on a chalkboard. He has taught it for years and knows every angle.

Then a character named Michael puts Chidi in the driver’s seat of an actual trolley. There are real people tied to the track, the lever is in Chidi’s hand, and the trolley is barreling forward. Chidi screams, freezes, cannot decide. Time passes while he fails to either pull or release the lever, and in the end someone gets run over. Blood sprays. Michael repeats the situation again and again, changing the conditions slightly each time. What if the one person on the track is Chidi’s friend. What if the five are strangers. And he says, blithely, that Chidi only talks so easily because he learned it from a textbook. Does he really think he could make that choice?

The scene is darkly, cruelly funny. Because it is a sitcom, it is played as a joke, blood spraying like a fountain in slapstick. But the joke stays with you. Watching it, I caught myself. Oh. It is not the same. The reasoning you tidy up on a desk and the state of standing inside it with your hand on the lever are not the same thing.

Looking back, this scene held the entire seed of what I would try to build years later. The point is not the trolley problem itself. The trolley is just material. The real point was the gap between a position you organized in your head and a choice you have to make while standing in the spot. The power of making that gap visible. Chidi does not get shaken by reciting five schools of thought. What shook Chidi was the lever. He had to face, with his body, that the clean theory he had just laid out was no help the moment his hand touched the lever.

There is a distinction buried in here that I only put into words much later. Knowing something and standing inside that knowledge are different. Chidi knows the trolley problem better than anyone. But the moment he stands inside the trolley, that knowledge collapses. The thought that the place where learning truly happens may not be where knowledge accumulates, but where accumulated knowledge falls apart. That is where chapter 1 begins.

I Dislike Black and White

Around the time I watched that scene, I was unusually alert to anything called “a new form of learning.” Virtual space was the hot topic then, and it was a period when I believed that, the way content was shifting from static to dynamic, a virtual space would inevitably be better in many ways. That is probably why the scene from The Good Place did not pass by as a simple sitcom joke and instead stuck with me.

Watching The Good Place, I got a sense that some new kind of learning could arrive this way too. So I searched. People who unpack ethics through stories. That led me to the most highly rated lecture out there: Michael Sandel’s Harvard course Justice. Apparently it was already world-famous as a book, but I did not know it until then.

I just liked watching the lectures. Not because of any particular trigger, I simply liked them. I am someone who dislikes black and white and is comfortable staying in the gray. A state where several positions all make sense at once is more interesting to me than a story that converges on a single right answer. I was always drawn less to “this is the answer” said quickly, and more to “here is the tension that lives here” laid open. Sandel’s classroom was exactly that kind of place. Instead of giving an answer, it makes students see the loose seams in their own answers for themselves. I liked that form.

How Sandel’s First Lecture Unfolds

Sandel’s first lecture usually goes like this. He throws out the trolley problem. A streetcar with failed brakes is hurtling toward five people, and you can switch the track to send it toward a side where only one person stands. Most of the room raises a hand to switch it and save the five. By arithmetic it seems obvious. Five is bigger than one.

Then Sandel tweaks the scenario slightly. This time there is no lever to switch the track. You are standing on a footbridge, and beside you is a large man. If you push him onto the track, his weight stops the streetcar and the five live. Would you push him? The hands that just voted to save five mostly come down this time. It is the same structure, sacrificing one to save five, yet people’s answers split. Sandel asks: between a moment ago and now, what changed? Why will you pull a lever but not push a man? Are they not both killing one to save five?

He does not stop there. The next variation comes. You are an emergency room doctor. Five patients are dying, each needing a different organ. In the next room is a healthy person who came in for a checkup. If you take that one person’s organs and divide them among the five, the five live. Would you do it? Almost no one raises a hand. Same arithmetic again. Sandel drives the students through this series of scenarios. Your answer changes from situation to situation. Can you explain, yourself, what the changing standard is?

And in the latter half of the lecture, a real case appears. The 1884 English case Queen v. Dudley and Stephens. Four sailors adrift after a shipwreck, starving, killed the weakest among them, a boy, and survived on his flesh. After being rescued, they were charged with murder. Sandel asks: are they guilty? What is the same and what is different between this and the trolley, sacrificing one to save the rest? Is the fact that the boy did not consent decisive? If he had consented, would it change?

What all of these scenarios do is one thing. They make a student’s just-given answer clash with their gut in a slightly twisted version of the situation. And they leave the student to discover that clash for themselves. Sandel does not say “you are wrong.” He does not hand over the right answer either. He just throws the next question, and the student trips over the contradiction inside their own self.

I learned later that this clash has a name. Cognitive dissonance, defined by the psychologist Leon Festinger in 1957. It is the discomfort that arises when two things you believe collide, and that discomfort is what moves a person. The interesting part is that I met this term afterward. When I watched the lectures I was not analyzing this. I just liked the moment of collision. The term came much later. And this order, where the experience comes first and the name comes after, repeats throughout this whole book. I am not a scholar but a maker, so I always run into things first and look for names later.

Why a Book Would Not Have Shaken Me

Here I stopped and thought about something. Sandel’s Justice exists as a book too. The content is the same. The trolley is there, the man on the footbridge is there, that ship from 1884 is there. But if I had read it as a book, would I have been shaken as much? Probably not.

I had long held a vague sense of one thing. A photo works on a person more strongly than text, video more than a photo, and the real thing more than video. The closer a medium gets to the real, the more something irreplaceable appears. That is the same reason the trolley scene in The Good Place stayed with me far longer than the trolley problem written in text. Writing “Chidi could not decide” is one sentence, but seeing Chidi freeze for one second on screen leaves that second whole in your body.

But when you push this all the way to learning a concept, the explanation suddenly gets hard. Moral reasoning is essentially something that happens inside the head. What does it even mean that “a medium closer to the real” is stronger there too? Honestly, I still cannot put this cleanly into words. But I felt, clearly, that The Good Place and Sandel’s classroom were pointing in the same direction.

The hint was in what the two had in common. What shook Chidi was not the explanation of the trolley but being put inside it. What shook the student in Sandel’s classroom was not reading an ethics textbook but receiving the next question right after answering in their own voice. Both share something. The receiver is not just sitting and listening; they had to do something themselves. They had to state a position, grab the lever, answer the next question again.

A book cannot do this. A book is written without knowing the reader’s position. A hundred people read the same page and meet the same counterexample. Whether that counterexample targets the weak spot in my position is pure luck. The person who thought they would save five in the trolley and the person who believes you can never use a human as a tool read the same counterexample on the same page. The counterexample in Sandel’s classroom, by contrast, starts from what that student just said. For the same trolley, the next question thrown at the student who would save five differs from the one for the student who cannot pull. The first gets the man on the footbridge; the second gets “what if all five were your family.”

Here I came to separate two things. One is whether the receiver moved inside it. The other is whether the system responded to that movement. A book is weak on the first and cannot do the second at all. A well-made video makes the first strong, but it only flows in one fixed direction and cannot do the second. Sandel’s classroom does both. And the moment it does both, the weight of the learning changes completely. How accurate the content is comes after that.

Behind that vague sense, that a medium closer to the real is stronger, there is probably the fact that we do not reason with the head alone. When imagining pushing the man in the trolley, unlike imagining pulling the lever, people summon the sensation of pushing with their hands. That is why the answers split. With the same arithmetic, the body reacts differently. It means moral judgment is not pure logic but rides on the body. If so, “a medium closer to the real is stronger” makes sense for concept-learning too. The more you set the receiver inside the situation, the more the body cuts into reasoning that was running in the head alone, and only then does the weak spot in their position surface.

Back Then, What I Imagined Was VR

The problem is what technology I imagined this through back then. It was a period when virtual space was the topic, so naturally I thought of VR (virtual reality). Recreate the trolley situation in 3D, have the user put on a headset, go inside, and grab the lever directly, and maybe you could recreate that scene from The Good Place. I saw it as a matter of building an interactive environment.

But there was a gap here that I only noticed much later. I had thought only the environment was interactive, while the content stayed static.

What I mean is this. No matter how vividly you build the trolley situation in VR, all the user can do inside it is the preset branches. Pull the lever, or do not. Then a prepared result A or result B plays. Even if the user asks “but what if that person were an innocent child?” or “if I did not pull it directly and the streetcar switched on its own, is my responsibility reduced?”, the system cannot answer. It did not prepare those branches. In the end the user is someone choosing one of three or five forks, not someone unfolding their own reasoning.

Why this is fatal: it cannot do precisely that second thing from the chapter before, the system responding to the receiver’s movement. No matter how immersed the user gets grabbing the lever, the counterexample that comes next did not come from what the user said but from a script the creator wrote in advance. Inside a gorgeous environment, you end up solving a multiple-choice problem. The immersion is high, but the structure of the learning is no different from a book.

The reason Sandel’s classroom is alive was the opposite. Sandel does not prepare the next counterexample in advance. He listens to what the student just said, finds the weak spot in it on the spot, and builds the next question. A different question goes to each student. This was the genuinely hard part. Drawing the difference between the two approaches looks like this.

%%{init: {'theme':'neutral', 'look':'handDrawn'}}%%
flowchart TD
  U[User response] --> S{System approach}
  S -->|Static content / VR| A[Play one of preset branches A·B·C]
  S -->|Generated each moment| B[Find the weak spot just said, generate a new counterexample]
  A --> A2[Multiple choice inside a gorgeous environment]
  B --> B2[Reasoning unfolds differently per person]

On the surface both are “an experience the user takes part in,” but look inside and they were completely different things. The VR I imagined was static content pretending to be interactive, while what Sandel does was reasoning generated each moment.

Back then there was no way to bridge the two. To generate a different counterexample per person in real time, a human facilitator has to sit one-on-one, and that does not scale. One Sandel cannot face everyone in the world one by one. So this idea went into a desk drawer for a while. As a vague hypothesis at the level of “someday, when the technology arrives.” Honestly, I was not even sure that “someday” would come.

A Million Segments

What made me pull this idea back out came from a somewhat odd place. Something I had long found lacking about segmentation in marketing (segmentation, dividing customers into similar groups).

I have long thought the very concept of segmentation is a bit strange. How many groups you can divide customers into is, in the end, a function of how many people and how much resource you have to do the dividing. With few people you split into four or five, with many you split into twenty. A bigger company slices finer. But just because they got grouped into the same box, are the people inside really alike? Are the millions in the single box of “women office workers in their thirties” the same person? Not at all. Inside that box are the married and the unmarried, those in debt and those not, the one who had a good day yesterday and the one who had a bad one, all mixed together.

To put it to the extreme, if you have a million customers, you could have a million segments. Because each person is different. The most accurate segmentation, in the end, goes all the way down to the individual. Yet in reality we do not split into a million. Not because we cannot, but because the return on investment (ROI, Return on Investment) does not add up. The cost of analyzing and responding to a million people one by one was greater than the profit it produced. So we grouped roughly. Segmentation was a compromise fitted to human limits, not because it was right. We always called it “the right unit,” but it was really only “the unit we could manage.”

At some point two thoughts met. Sandel throwing a different counterexample at each student, that too is in the end one-on-one customization. A human facilitator cannot do it for a million people. The ROI does not add up. The wall I could not solve in VR, “reasoning generated each moment,” and the wall that made segmentation a rough compromise, were actually the same wall. The wall that real one-on-one is only possible by grinding through human labor, and that cost is unmanageable. In education they call that wall “one instructor per thirty students,” and in marketing they call it “five segments,” but the essence was the same. The unit cost of making something fitted exactly to one person is too expensive.

Then the question changes. What happens when that wall comes down? If the unit cost of a response fitted to one person approaches zero, do education and marketing not have to be redesigned? If every compromise we believed for thirty years to be “the realistic unit” was really a temporary agreement made by cost.

The Moment the Cost Came Down

Early this year, I felt that wall come down.

Less a single specific event than many moments piling up, but the biggest realization was elsewhere. The first time I used Claude Code. It takes a request I jotted down vaguely, makes code on the spot, watches my reaction, and makes the next thing. Rather than pulling out a prepared answer, it makes something new each time fitted to the context right now. For the first few days it was just a convenient tool.

Then at some point a thought of a different weight came. This was not something happening to me alone. In the same way, simultaneously, it could make something fitted to a different context for a great many people. What I saw was less a new feature than the fact that work I could not do because of cost had become possible at scale. Making a result fitted to one person was no longer as expensive as one person.

As I used it, the six-year-old drawer in my head opened. What had blocked that hypothesis back then was not a shortage of ideas but the cost of scale. Making something fitted to one person, separately, for a million people, was too expensive. You cannot attach a million human facilitators. I watched that cost come down in front of me for the first time.

In technical terms this is inference-time generation. Rather than making all the content in advance and pulling it out, it generates at the very moment a request arrives. But I understood it not as a term but as a cost. The unit price of the one-on-one customized reasoning Sandel did had dropped to near zero, from a price that required grinding through human labor. Work that six years ago needed the wages of one human facilitator now became the cost of a single model call.

Then the hypothesis I had put in the drawer started to make sense again. What I could not solve with VR was not a shortage of environment but static content, and now content can be made anew each moment. The ROI wall that forced segmentation into compromise had disappeared, at least on the line item of “the cost of generating reasoning fitted to one person.” Throwing a different counterexample at each of a million people is no longer an arithmetically impossible thing.

By the time I got here, that opening sitcom scene looked different. Michael could put Chidi in the trolley because he was a being who could make a situation just for Chidi. He aimed at Chidi’s weak spot, made a scenario fitted only to Chidi, on the spot, changing the conditions again and again. Six years ago that was a fantasy inside a sitcom. Something only an all-powerful character could do. Now a very small version of that ability has actually come into hand. Of course the Michael in the sitcom was all-powerful and today’s tool falls far short, but the direction was the same.

Being Able to Build It Does Not Mean It Is Built Well

But I have to stop here. The fact that the cost came down does not mean a good thing gets built. Being able to build something and that something being built well are different problems.

Think again about a million segments. Doing a response fitted to each person for a million people at once means, put differently, that those million responses go beyond the range of human oversight. Back when people inspected each one, at least someone had seen the content. But a person cannot look through a million pieces generated anew each moment. Then a question arises. Can this be 100% delegated to AI?

It is not as if marketing is safe either. A single misfired message toppling a brand is not rare, and there was a case recently too. The difference is that in those cases there is usually still room for a person to see the message, take it down, apologize, and correct it. But when the subject is ethics or values, and the unsettling words go out one-on-one without ever passing a human eye, in the spot where a person is wavering over their own position, the weight changes again. The mark a single badly-made instance leaves on someone can be harder to erase than a misfired ad line.

So this domain has two layers. The first layer is that there is no right answer. Since the trolley has no right answer, the system has no standard to grade itself on whether it threw a good counterexample. The scoring sheet given for free to a coding agent, the standard of whether it runs or not, is not here. Without a scoring sheet, AI grabs the next-easiest standard. Does the user nod? Do they come back? But if you follow that standard, it goes backward from the original goal of “making a person less certain.” Because a comforting line nods a person along more than an uncomfortable truth. What you meant to build was a mirror, and it becomes a flatterer fishing for approval.

The second layer is the nature and responsibility this domain carries. Not only is there no right answer, but mishandling it leaves a mark on a person’s values. At a scale beyond the range of oversight, you send a judgment that has no right answer into a domain where marks remain. Solving the first layer alone is hard, and the second sits on top of it.

How to handle these two layers is the real hard problem of this project. “Can reasoning be generated each moment” is now solved. That is a starting line, not a finish line. In a spot with no scoring sheet, how do you tell good reasoning from merely pleasant reasoning, and at a scale a person cannot fully watch, how do you carry that responsibility? This is something I will write a whole chapter on each, later. For now, driving these two nails into chapter 1’s giddy conclusion is enough.

Why I Keep the Record in Public

This is usually where one would move on to “so I started building.” But this notebook is not written to show off what got built. I want to leave the process of thinking through this question in public.

There are reasons. First, this subject is not a question of whether it can be built. Whether it should be built is the far heavier question. A tool that deliberately makes a person less certain shares the same surface, used wrong, with a tool that manipulates a person. The reason Sandel’s classroom is education and not manipulation is that several safeguards were laid into it that I did not see at the time. That other students are watching. That the promise “this is a philosophy class” is set before it begins. And that Sandel himself sometimes hesitates and changes his position. What disappears when those safeguards move to a one-on-one screen. This risk I will take up in earnest in several chapters ahead, but the short version is that it is not the kind of thing I can settle inside my own head. Writing in public and taking the pushback is safer.

Second, I have already knocked on this hypothesis from several angles. Afraid of concluding on my own that it was good, I deliberately borrowed the perspectives of different positions and ran a kind of roundtable review. The eyes of an investor who thinks you have to grow fast and take the market, the eyes of a large AI lab that sees first the danger of a feature that shakes people, the eyes of an education operator who lays down the cold numbers of completion rates and learning outcomes, the eyes of a learning scientist who says measure first whether learning even happened, the eyes of a lawyer who asks first about minors and liability. That the evaluations these gave for the same idea did not line up was, if anything, what I liked. The weak spots I would have missed listening to only one side surfaced where the positions collided.

The one that changed my thinking most was a large AI lab’s safety perspective. They pointed out that the nice-sounding line “make a person less certain” actually shares the same surface with a risk they spend enormous resources reducing in training their models, namely persuasion and manipulation that shake a user in an unwanted direction. That good intent is a hair’s breadth from a dangerous feature. The point stung, and so it stayed. How I take in this kind of pushback and insight from the roundtable and change the design becomes one axis of this book. A piece that only lists the good points I came up with alone is neither interesting nor safe.

So this notebook is a record of the path, not the conclusion. Starting from a vague intuition, where it got stuck and where it came loose again, and what I still do not know, written in order. I plan to leave the wrong judgments in too. When it is all written it will become a book, and that book will not start from zero but be the sum of pieces already verified in public.

What This Chapter Did Not Answer

Chapter 1 only wrote down the starting point. There is far more it did not answer.

The biggest is this. Will Sandel’s classroom work one-on-one too? A student wavering in a classroom watched by a thousand people, and a person wavering alone in front of a screen with no one watching, sitting across from an AI, may not be the same. The classroom has the eyes of other students, the promise that “this is a philosophy class,” and the human gap of a professor who hesitates. What disappears when that moves to a one-on-one screen is something I only noticed much later. This story is important enough to write a whole chapter on later.

And one more. In this piece I wrote “the cost came down” as a big realization, but the cost dropping does not mean it is okay to do it. Between becoming able to do something and ought to do it there is a separate distance to deal with. I also plan to point out, in this book, what happened to the tools that ignored that distance and ran toward “it works now, so let’s do it.”

The third is a more fundamental question. Why ethics, of all things? There are plenty of domains without a right answer besides ethics. Career choices, negotiation, leadership calls, investment decisions, all of them spots without an answer sheet. Yet I started with the trolley. Among domains without a right answer, ethics is the heaviest, and the easiest to hurt a person with. What happens if you keep throwing trolley counterexamples at someone who obsessively chews on their own moral failings? Or at someone with the trauma of a real moral choice? If that weight feels too much, there is also a path of proving the same form first in a lighter domain like negotiation or leadership. Which is right, I have not yet decided. I started with the trolley, but whether it is the spot to carry all the way through, I am keeping open. This too was one of the points the roundtable split on most.

In the next chapter, I look more closely at the moment the vague six-year-old hypothesis turned, this year, into a form that could be tested. Between feeling in your head that “this will work” and making it into something that actually runs, there was one more gap. When I first tried to put the intuition on a screen, I ran straight into a few things. How much to build and where to leave untouched. Whether to start with the single trolley, or aim at the whole of the answerless domain. And above all, if this is a tool that shakes people, where to set down the person who got shaken. I will lay out, in the next chapter, the first things I faced when I tried to give that vague intuition a concrete form.