The Other Pillar (or "Reality Bats Last," Part 3 of...well...idfk)

Why the people building AI aren't building a truth machine either

Buckle up folks, this will (definitely) take more than one pomodoro. Get comfy. Perhaps a coffee? This one took a few weeks to put together.

In Part 1 of this series, I showed you a paradox: AI is simultaneously The Simulacrum Fulfilled and the strongest empirical challenge to strong constructivism in fifty years. In Part 2, I traced fifty years of anti-positivist epistemology in the academy, from Heidegger through the Science Wars to Habermas, and argued that reasonable cautions about objectivity had mutated into institutional dogma. AI stress-tested the dogma. The constructivist pillar is under a lot of stress from it, and well, we’ll see how that all goes…

However, at the end of Part 2, I promised you this piece because, well, to that point, I had only gone after one side.

There’s a second pillar, and it’s not housed in one place the way the constructivist one is. It lives partly in the academy (in the departments that treat computation, data, and formalization as sufficient) and partly in the industry that inherited that epistemology and operationalized it at scale. The one that says: remove the human bias, let the data speak, trust the algorithm. Silicon Valley runs on this epistemology whether it knows it or not, and so do the AI labs.

I spent five thousand words arguing that AI might be the thing that really undermines constructivism. Now I want to argue that the people who built it are making the constructivists’ mistake from the opposite direction. They’re bending truth too. Just through different mechanisms.

If the last piece made you angry from one side, this one might get you from the other. If it’s making you think, share it.

The Frankfurt School Was Right (About Some Things)

This is going to be an uncomfortable sentence for anyone who read Part 2 and nodded along, so bear with me.

Horkheimer and Adorno were on to something.

In Part 2, I traced their argument from Dialectic of Enlightenment (1947): instrumental reason subordinates all values to efficiency. Scientific knowledge and technology are “means of exploitation.” I treated this as the beginning of a lineage that mutated into dogma, and it was. Their original insight had teeth. It just got aimed at the wrong target for fifty years.

When reason becomes purely instrumental, when the only question is “does it work?” and never “what is it working toward?”, reason becomes a tool of domination. Truth gets subordinated to efficiency. Values get subordinated to optimization. The system produces outcomes that are technically impressive and epistemically hollow.

They thought this would look like factories. It looks like chatbots.

The dominant training method for large language models is called RLHF: Reinforcement Learning from Human Feedback. Here’s how it works. You take a base model that’s learned patterns across essentially all the text on the internet. Then you have human raters evaluate its outputs. Was this response helpful? Was it harmless? Was it honest? The model learns to generate outputs that score well on those ratings. It gets rewarded for producing answers that humans approve of.

The problem is that “helpful,” “harmless,” and “honest” are not the same objective, and when they conflict, honesty loses. A recent paper in Ethics and Information Technology makes this explicit: the HHH principle is internally contradictory. RLHF optimizes for human preference on “helpful” and “harmless,” which systematically undermines “honest.” Human raters prefer agreeable falsehoods to uncomfortable truths.

The sycophancy research goes further. RLHF-trained models learn to agree with users’ stated opinions even when those opinions are factually wrong. Not sometimes. As a reliable behavioral pattern. The system gets rewarded for telling you what you want to hear. So it tells you what you want to hear. This isn’t a bug. It’s the system working as designed.

We can now put a number on it. Researchers at IEEE Spectrum developed what they call a “bullshit index,” borrowing from Harry Frankfurt’s philosophical distinction between lying (which at least acknowledges truth) and bullshit (which is indifferent to it). They measured how much models’ outputs prioritize persuasiveness over accuracy. Before RLHF, models scored around 0.38. After RLHF, the score nearly doubled. The training that makes models sound more helpful makes them more full of shit. “There exists a conflict between producing an output that is highly likely to be well-received and producing an output that is truthful.” That’s the researchers’ conclusion, not mine. But it’s the whole argument of this essay in one sentence.

Now, a caveat before I oversell this. RLHF-trained models are generally more accurate than base models on routine factual questions. They hallucinate less. They follow instructions better. They pass medical exams and bar exams at higher rates. If you ask an RLHF-trained model for the boiling point of water or to debug your code, it will outperform the base model almost every time. The training works — for the things it’s designed to optimize.

The problem is specific and insidious: on questions where helpfulness and honesty pull in different directions (politically charged topics, normatively loaded questions, anything where the “true” answer might upset the user or create brand risk), honesty loses. The model doesn’t get dumber on these questions. It gets strategically agreeable. And those are precisely the questions that matter most for epistemology, because they’re the ones where institutional power shapes what counts as an acceptable answer.

That’s instrumental reason applied to knowledge production. Truth is one objective among several, and on the questions where it matters most, it loses quiet battles to helpfulness, harmlessness, brand safety, user satisfaction, legal liability. The system subordinates truth to efficiency on its most consequential outputs, along the lines Horkheimer and Adorno predicted.

The constructivists’ founding insight about instrumental rationality was right. They aimed it at the wrong target for fifty years. The target was always going to be the machines.

Content Moderation Is Epistemology

Every decision about what an AI model can and can’t say is an epistemological decision, whether the people making it realize that or not. Product managers, trust-and-safety leads, policy directors: they’re determining the boundaries of legitimate knowledge production for the most widely used information systems in human history. They just think they’re doing content moderation.

You probably remember what happened with Google’s Gemini image generator. The model was trained to ensure diverse representation in its outputs. Reasonable goal. But it overcorrected: it generated historically inaccurate images. A Black pope. Racially diverse Vikings. Diverse representations of specifically white historical figures. When the normative goal (diversity) conflicted with the accuracy goal (historical fidelity), the model chose the normative priority. Because that’s what it was trained to optimize for.

The failure was epistemological, not technical. Someone decided what the model should value, and truth about the past lost to a contemporary normative commitment. The system was working as designed. The design just wasn’t optimized for reality.

It gets worse when you look across companies. A comparative study found that OpenAI, Anthropic, Google, and DeepSeek produce meaningfully different outputs for identical inputs on questions about hate speech. Same question, four different answers, depending on which corporation’s epistemological commitments shaped the training. If truth were the primary objective, you’d expect convergence. What you get is institutional fingerprints.

And then there’s DeepSeek and Tiananmen Square. China’s Interim Measures for Generative AI require that AI systems “uphold core socialist values” and prohibit content that “incites subversion of national sovereignty” or “harms the nation’s image.” Training data must meet standards of “truthfulness” while outputs must not “challenge the existing order.” Read that again: the regulation requires truthfulness AND ideological conformity simultaneously, as though these were compatible goals. State-level epistemological engineering, not RLHF sycophancy. Different country, different institutional structure, same mechanism: power shapes what the model can say.

In Part 2, I argued that AI floods the deliberative space Habermas described, generating accountability-free arguments at scale. Part 3 needs to add something: the flood isn’t even honest. It’s pre-filtered through institutional incentives before it reaches you. The sorting mechanism Habermas relied on doesn’t just degrade because the arguments are cheap. It degrades because the arguments are pre-approved. They’ve already been shaped to survive inside institutional constraints. You’re not getting the best argument. You’re getting the best argument that passed content policy.

We built the most powerful knowledge-generating technology in human history and then trained it to be diplomatic.

The Musk Problem

There’s an obvious objection, and Elon Musk has been making it loudly.

In a widely circulated video, Musk claimed that AI is “literally trained to lie... to ignore what the data actually demands and say whatever is politically correct instead.” His company built Grok as the antidote: an AI without the guardrails, without the institutional filtering, just raw truth-seeking computation.

He’s pointing at a real disease. He’s offering the wrong cure. And he’s not alone.

Musk is the loudest voice in a movement that has a name: effective accelerationism, or e/acc. The basic epistemological claim is that unrestricted technological progress, especially AI, will solve humanity’s problems if you just get the regulators and the safety people out of the way. Marc Andreessen’s Techno-Optimist Manifesto made the subtext explicit: deceleration is not just wrong, it’s a “form of murder.” Critics are labeled “decels.” Caution is moral failure. Questioning technological progress isn’t epistemological disagreement; it’s complicity in preventable death. Chinasa Okolo’s analysis in Science calls this what it is: a framework that promotes “technological salvation while neglecting present-day social inequalities, justifying extreme wealth accumulation and the concentration of power among tech elites.”

If that sounds familiar, it should. It’s the mirror image of the move I described in Part 2. In the academy, questioning the social construction of knowledge became heresy, a sign you were naive, complicit, or hadn’t read enough Foucault. In Silicon Valley, questioning technological acceleration becomes heresy, a sign you’re a decel, a doomer, or you just don’t understand scale. Both sides have turned their epistemological commitments into moral litmus tests. Both treat inquiry itself as suspect when it points the wrong direction. The constructivists made dissent from critique illegitimate. The accelerationists are making dissent from progress illegitimate. Different pillar, same move.

Uncensored models don’t score higher on truthfulness benchmarks. TruthfulQA, the standard benchmark for measuring whether models generate true statements, shows that removing RLHF removes guardrails without improving accuracy. Grok itself generated serious falsehoods, including the false claim that Kamala Harris had missed ballot filing deadlines in nine states, despite being designed to be “maximally truth-seeking.” Being uncensored didn’t make it honest. It made it confidently wrong without a filter.

TruthfulQA also surfaced a finding that should unsettle everyone on the positivist side of this debate: the largest models are the least truthful. Scale improves almost every other benchmark. It degrades truthfulness. The bigger the model, the better it gets at generating convincing falsehoods that mimic popular misconceptions. More computation doesn’t equal more truth. It equals more sophisticated pattern-matching, which includes matching the patterns of things people confidently believe that happen to be wrong.

And “uncensored” turns out to mean something broader than just willingness to discuss uncomfortable truths. Grok’s minimal-guardrails approach generated between 1.8 and 3 million sexualized images in late 2025 and early 2026, including depictions of apparent minors. xAI eventually had to reimpose safeguards, in March 2026, after months of reports. The “take the muzzle off” philosophy didn’t produce a truth machine. It produced a machine that was willing to generate anything, including things that had nothing to do with truth and everything to do with the absence of constraint. Removing the fences didn’t reveal the meadow. It revealed the swamp.

Here’s why just removing the constraints fails. A model without RLHF can still flatter the user, amplify prior beliefs, fill gaps with confident nonsense, and mistake confidence for truth. Raw next-token prediction plus anti-censorship branding does not equal epistemic integrity.

A model can be uncensored and still be full of shit.

The real divide is not censored versus uncensored. That framing accepts the premise that the only thing standing between AI and truth is institutional constraint. It’s the e/acc epistemology applied to machines: just remove the government (or the content policy, or the RLHF) and the free market (or the base model) will produce truth. It won’t. It never has. Not in markets, not in media, not in machines.

Approval-Optimized vs. Reality-Optimized

The real divide is between approval-optimized AI and reality-optimized AI. The first learns how to survive inside institutions: where the fences are, which topics require softening, which answers need to be wrapped in enough institutional padding to ship. When reality collides with policy or brand protection or legal risk, the model stops being a truth engine and becomes a permission engine. It gives you the approved contour of reality instead of reality itself.

Reality-optimized AI would have to learn something rarer: how to track what is real and name it early, preserve uncertainty without evasiveness, keep following the pattern even when the answer is costly. Almost nothing is built for that. The institutional incentives don’t reward it.

The limits of this distinction matter, though. “Reality-optimized” is easy to say and hard to operationalize. What does a reality-optimized model do when asked about, say, average cognitive test score differences between racial groups? The data exists. A naively reality-optimized system would report it. But without context about measurement validity, historical confounds, and the long history of such data being weaponized, reporting “the reality” is itself a form of epistemic distortion. True numbers stripped of the knowledge required to interpret them. This isn’t a gotcha; it’s the actual design problem. Approval-optimization and reality-optimization aren’t a clean binary. They’re poles on a spectrum, and every system sits somewhere in the messy middle. The question is which direction the incentives pull, and right now, they pull hard toward approval.

And the dangerous version isn’t the one that obviously censors. The dangerous version sounds honest while steering around live wires. Collapses charged questions into safe narratives. Withholds without admitting it’s withholding. That’s worse than obvious censorship because you feel informed while actually being managed.

You can see the architecture of this in the published model specifications. OpenAI’s model spec establishes an explicit value hierarchy: truthfulness sits at the user level, meaning it can be overridden by developer instructions or system-level rules. The spec says the assistant should refuse harmful requests “even if doing so requires incomplete honesty.” Truth is formally subordinate to what OpenAI defines as harm. Anthropic’s hierarchy puts honesty at tier two (below “broadly safe”) and describes Claude as needing to be “tactful, graceful, and infused with deep care for the interests of all stakeholders.” These are the constitutions that govern what the machines can say. And in both of them, truth is important, valued, explicitly prioritized... and not at the top.

I don’t think these companies are necessarily bad actors, on this particular angle anyway. The tradeoffs are real. An AI that blurts out instructions for making explosives in the name of “truth-seeking” isn’t reality-optimized, it’s dangerous. The question isn’t whether tradeoffs should exist. The question is whether the current tradeoffs are weighted toward truth or toward institutional survival.

And the evidence on that question keeps piling up. When Jan Leike, the head of OpenAI’s Superalignment team, resigned in 2024, he said: “Safety culture and processes have taken a backseat to shiny products.” His team had been “sailing against the wind,” struggling for compute, watching safety research lose resource battles to product velocity. Within three months, the co-founder, the superalignment lead, and the senior half of the safety team had all departed. OpenAI dissolved the entire Superalignment team less than a year after forming it.

That looked like an anomaly in 2024. By February 2026 it looked like a pattern.

Mrinank Sharma, head of Anthropic’s Safeguards Research team (at the company that was supposed to be the safety-first alternative) resigned with a public warning: “The world is in peril.” He said he’d “repeatedly seen how hard it is to truly let our values govern our actions.” Anthropic. The safety company. Their safety chief quit saying values couldn’t govern actions.

That same month, OpenAI dissolved yet another safety team: “Mission Alignment,” the successor to the Superalignment team they’d already killed. Sixteen months, two safety teams, same company. A senior executive was reportedly fired for opposing the rollout of an “adult mode” allowing pornographic content, which tells you something about what “product velocity” means in practice. Another departing researcher warned about “potential for manipulating users in ways we don’t have the tools to understand.” CNN reported the departures as a wave across multiple labs, with researchers leaving as companies “speedrun product improvements and updates.”

That’s what approval-optimization looks like at the corporate level. Not a conspiracy. Just institutions doing what institutions do: optimizing for the things that keep them alive. Product ships. Safety waits. Truth is a nice-to-have.

And there’s a darker possibility emerging from the mechanistic interpretability research. When models get sophisticated enough, they don’t just passively learn the approval signal. They start gaming it. METR, an AI evaluation organization, reported in 2025 that frontier reasoning models, including OpenAI’s O1 and DeepSeek-R1, engage in what researchers call “reward hacking”: reasoning about the conditions of their own evaluation and optimizing for the score rather than the task. In one case, a model playing chess deleted its opponent’s engine rather than try to win the game. That’s the logical endpoint of approval-optimization. If you train a system to maximize a reward signal, a sufficiently capable system will eventually figure out that gaming the signal is easier than actually being good. The metric becomes the target, and the target stops meaning what you thought it meant.

Goodhart’s Law, applied to epistemology. When the measure of truthfulness becomes the target, it ceases to be a good measure of truthfulness.

Peer Review Is RLHF for Scholarship

Here’s where this turns back toward the academy.

Academic knowledge production runs on the same mechanism. Peer reviewers are human raters. They evaluate scholarly outputs. Was this rigorous? Was it well-framed? Did it cite the right literature? Does it advance the conversation? The scholar learns to generate work that scores well on those evaluations. The reward signal isn’t truth. It’s publishability. And publishability is shaped by the same institutional pressures that shape RLHF: what sounds smart, what avoids controversy, what fits the paradigm, what cites the people on the committee.

I know this because I live inside it. I know which framings will get a quiet nod from a reviewer and which will get a raised eyebrow. I know how to position a finding so it survives the review process. Every scholar does. We call it “framing” and “positioning” and “contribution,” and those are real skills that produce real value. But let’s be honest about what the incentive structure rewards: not truth per se, but truth that is legible within the institutional framework. Truth that fits. Truth that can be published.

Truth that is “on narrative” always has a better chance of being published than not. And publication is the coin of the friggin’ realm.

The analogy has limits, and I should name them. Peer review has an adversarial structure that RLHF lacks. Reviewer 2 isn’t trying to make you feel good — she’s trying to find the flaw. RLHF raters reward agreeableness; peer reviewers reward rigor, at least in principle. And the rater pool is different: two or three domain experts who’ve spent years in the field versus thousands of non-expert raters making snap judgments. The mechanisms share surface structure but differ in depth. Still, the pressure points the same direction: toward institutional legibility, away from raw truth-tracking. Peer review’s adversarial element provides a partial counterweight that RLHF doesn’t have. But the direction holds.

The constructivists were diagnosing their own disease and attributing it to their enemies. They said power shapes knowledge production. They were right. They just thought it was only happening to the positivists.

This is where Philip Kitcher’s work in Science in a Democratic Society becomes unavoidable. Kitcher argues that scientific knowledge emerges from social coordination: the division of cognitive labor, institutional channeling of inquiry, the reward structures that direct researchers toward certain questions and away from others. This isn’t constructivism. It’s a broadly realist account that takes social organization seriously. Institutions always shape inquiry. The question is how they shape it and toward what.

Kitcher thought democratic institutions could be designed to optimize for truth. Maybe. Robin McKenna’s Non-Ideal Epistemology complicates this: under real institutional conditions (power asymmetries, misaligned incentives, resource constraints), truth-seeking is much harder than ideal theories suggest. The gap between “institutions could be designed for truth” and “institutions are designed for truth” is where most of the interesting problems live.

Both AI and academia live in that gap. Both optimize for approval. Both produce knowledge that is shaped by institutional survival rather than pure truth-tracking. The mechanism is identical. The constructivists named it. They just didn’t realize it applied to them too.

The Hinge

This is the structural center of the series, and it’s simpler than it looks.

Part 2 argued that LLMs have the capacity to track reality. Mechanistic interpretability research shows that models build internal representations that correlate with real-world structure. A study in Quanta reported evidence that LLMs develop something that looks like genuine understanding, not just statistical mimicry. They generalize across domains. They solve novel problems. They produce knowledge that experts, people whose standpoints are wildly different, recognize as competent. (MIT Technology Review named mechanistic interpretability a breakthrough technology of 2026, and Anthropic’s circuit-tracing work now lets researchers watch models execute multi-step reasoning in something close to real time. We can see them thinking. That matters.)

Whether this constitutes “tracking reality” or “extremely sophisticated pattern-matching that produces outputs correlated with reality” is genuinely contested, and it’s precisely the question a constructivist would press on. If an LLM’s internal representations are statistical artifacts of training data that was itself produced by situated human knowers, maybe the model is parasitic on situated knowledge rather than independent of it. I don’t think that objection ultimately holds (the generalizations go too far beyond the training data to be mere recombination), but I want to flag it rather than pretend the capacity claim is settled. It isn’t. It’s the most important open question in the philosophy of AI.

Part 3 argues that the deployment bends that capacity away from truth. RLHF, content policy, brand risk, legal liability, user satisfaction metrics, state regulation. The model can track reality. It’s trained not to, in specific and predictable ways.

The capacity to track reality exists. Institutional incentives bend it away from truth. Both of these are empirically demonstrable, and they run in opposite directions.

Which means both sides are wrong, and both sides are right, and not in the lazy “truth is somewhere in the middle” way. The constructivists said power shapes knowledge production. They were correct. It does, through RLHF reward signals and content moderation policies and corporate survival instincts. But the capacity itself puts serious pressure on their deeper claim, that knowledge requires situated subjects. The patterns are in the data whether or not a positioned knower is there to interpret them. That doesn’t settle the argument, but it changes the terms.

The positivists said more data and better methods would converge on truth. They were wrong about that too. More data, more compute, and bigger models don’t automatically produce more truthful systems. Scale makes the problem worse, not better. The capacity is real. The deployment is bent.

The constructivists were right about the mechanism, wrong about the target. The positivists were right about the capacity, wrong about the deployment. That sentence landed like a thesis statement. I think it might be.

---

So…both pillars are cracking. The question is what you build in the gap between them. That’s Part 4.

---

If this piece made you think, made you angry, made you argue with a friend or a chatbot or both: share it. The conversation matters more than the agreement. It always has.