The New Guy
Why the rumors of your replacement with AI have been greatly exaggerated
Onboarding
Your boss walks in with a smile on his face.
“We’ve got a new hire starting next week,” he says, and The New Guy absolutely aced his interview. What’s more, The New Guy’s spending his time before getting started on the job studying everything he can about the company from all the public records he can find.
Your department has been short-staffed for months. You could use the help.
“But didn’t we say we needed to hire two, maybe even three full-timers?” you ask.
Your boss smiles like he’s already sitting in the car he’s going to buy with his next bonus. “This guy’s so good, we don’t need to hire anyone else. He’s going to be doing the work of three people. Maybe even more.”
There’s an icy edge to these last words, a not-too-subtle message: watch out.
It’s hard not to feel a prickle of anxiety when you greet The New Guy (who we’ll call Sam, for no reason at all). By the end of his first day, though, you start to like him, in spite of your worries.
Sam is affable, quick to learn, and eager to please. He gets along with the rest of the department, too. People share their universally positive first impressions at the water cooler and over Slack and Zoom: he’s done a terrific job getting started on his new role and is easy to talk to, with a broad range of interests that mean he can make great small talk with anyone.
When you start working on your first project where you collaborate with Sam, you can hardly believe how smooth the process is. He works diligently and fast, never misses a deadline, and—if you ever make a suggestion or ask for rework—he’s happy to do it all over again, as many times as you want.
You present the project to a room full of executives. They look impressed. You give Sam a high five. This looks like the beginning of a beautiful friendship.
The Trouble With Sam
The next day, your boss calls you in to his office.
“I wanted to ask you some questions about your report,” he says, and points to one of your slides. “How did you get these figures on Slide 14? I didn’t think we even have a way to track that.”
When you see the numbers, you don’t recognize them—it must be one of the parts Sam worked on. So you ask him where he found them.
He gives you a long and detailed answer about how to find the figures using your existing data. It’s not really your area of expertise, so you just repeat it to your boss. “Sam says…”
Your boss is impressed with the explanation. After that, you and Sam collaborate a lot more.
Now that you work together a lot, it’s not long before you realize Sam has a problem: he’s a bullshitter, in the true Frankfurtian sense.
Sam doesn’t lie. Not exactly. But if someone asks him for a sales figure, he’s going to give them a plausible-sounding figure…whether or not he knows the real answer. If someone asks him to argue in favor of a proposition, he does it, and fires off rapid-fire “facts” that are usually true, but sometimes contain total whoppers.
The times when you catch those fibs make you feel very wary. You quickly discover that Sam responds very consistently when caught telling an untruth: he gives a detailed explanation with sources. But he’s compounding one untruth with another—the sources he cites don’t really exist. They sound perfectly plausible, but they’re nowhere to be found, online or off.
If you’d found this out about any other employee, you’d have reported them right away. But this isn’t just any employee. It’s Sam. Everyone loves him. Your boss recently rewarded him with a promotion, and he’s doing more work for more departments now.
So you grit your teeth, and try to at least be grateful for all of the truthful, good work Sam is doing. You commit to fact-checking Sam’s work during your off hours, concerned that you’ll otherwise end up blamed for his tenuous grasp on reality.
Seeking Efficiencies
Little by little, you find out that your coworkers are having the same issues with Sam that you’ve been having.
From what you can piece together, Sam’s slippery relationship with the truth is motivated by people-pleasing. The fibs show up to fill in whatever gaps exist in his knowledge, because he can never seem to admit when he doesn’t know something.
The worst part of this (for you and your coworkers) is that Sam seems to have very little regard for whether these fibs are “little white lies” with minimal consequences, or whether he’s invented from whole cloth the entire study that serves as the underpinning of a high-impact recommendation.
It’s all the same to him. So everything he says has to be fact-checked. Even though he produces huge volumes of work, absolutely nothing he says can be trusted.
Everybody is getting very cranky about having to fact-check Sam behind the scenes.
Eventually, a couple of department members find the courage to tell the boss what’s been going on.
Your boss agrees that it’s a problem, but—to your dismay—blames the rest of the department for Sam’s issues.
“You didn’t prepare him properly,” your boss admonishes you, telling you that before you give Sam a task, you need to write a very detailed, very careful instructional brief that tells him exactly what you expect, including explicit instructions that all sources need to be real, genuine documents that exist somewhere.
You object to this new requirement. No other employee in the department requires this level of hand-holding. Certainly no other employee needs to be told explicitly, at the start of each assignment, to tell the truth and not make stuff up.
But you can see from your boss’s budget that he’s allocated quite a lot of money to Sam and his future workload—so you tell him you’ll give it a shot.
For the next month, every time you ask Sam anything, no matter how minor, you accompany it with a set of instructions and expectations.
Sometimes, it seems to work. You swap tips with your coworkers about how to properly brief Sam in order to keep his work more factual.
But while the fibs are now less frequent, they’re still there. And it still doesn’t seem to make any difference to Sam whether a given fib could have serious consequences.
What this means, in practice, is that you all have to do exactly the same amount of fact checking as before. Whether a 100-page report from Sam contains fifteen bogus “facts” or three, the entire report still needs fact checking.
Your boss is unsympathetic. “Everyone else makes errors, too,” he points out. “I’d bet if you made a hundred page report, you’d get a few things wrong, just like Sam. Maybe this level of fact-checking is something we should always have been doing, for everyone’s work.”
He’s right that everyone gets a few things wrong. But—based on your own experiences fact checking—you don’t think that anyone else gets it wrong “just like Sam.” Maybe you make corrections every now and then, but due to an honest mistake or transcription error (like citing page 56 when the statistic cited is actually on 46). You never, not once, find one of your other colleagues citing a totally made-up source. You never find them whole-cloth fabricating quotes or data.
With so many things Sam could be making up, your boss tells you to “triage” his outputs, trying to only fact-check the parts that might have a real impact—so you can claw back the time you’d otherwise have spent following up every irrelevant background detail.
This means, of course, that an ever-increasing amount of bullshit—not errors, not honest mistakes, but bullshit—is making its way to the entire organization and beyond.
One day, a new employee starts onboarding. Your boss says she’s like an even better version of Sam, capable of much more. The best part: this new employee will have access to all of the company’s data, and will study up on all the internal information generated by the organization in the last decade.
You consider how much of that information was created by Sam.
You consider how much of it was probably, due to your fact-checking “triage,” absolute bullshit—and how it will now be understood as absolute fact by the new employee.
A few weeks later, you find yourself locked out of your email and work applications. The termination call is brief and to the point: the company wants to find someone who is better at working with Sam.
Multiplying the Negative
Let’s start with an easy question.
How valuable is Sam’s contribution, really?
Does his pace of production make up for the fact that he’s an incorrigible bullshitter?
I think that it is blindingly obvious that Sam is a terrible employee, and should be escorted out of the building with all due haste.
Sam’s contributions directly undermine his organization. Multiplying a debt gives you a bigger debt.
This means that Sam’s high productivity—which is largely illusory, after we’ve taken into account all the pre-work of making sure to brief him in extreme detail, plus the post-work of fact checking his outputs—serves only to enlarge the losses to his organization over time.
An obvious fact that MBAs and tech entrepreneurs too often forget: Organizations operate in reality.
This is not open to interpretation or appeal. Whatever false realities are manufactured inside the building do not obviate the real world outside it.
The more data you generate—and the more analyses you perform—based on bullshit, the more your best recommendations and insights will point in a direction totally orthogonal to reality.
The plausibility of Sam’s lies makes them extra pernicious. Obvious lies don’t get past sharp-eyed executives and analysts. Perfectly plausible bullshit does (and has, and will).
It is my belief that if a human being like Sam was hired by any publicly traded company, he would be fired (or on a Performance Improvement Plan) within a month of hire. There’s just too much at stake to let someone like Sam undermine the company with bullshit.
Besides—we just know people like Sam are bad ideas. We’ve all been burned by trusting a bullshitter before. We all know bullshitters who sound convincing are actually more dangerous, not less.
Sam generates negative value for his organization. He can do nothing else.
That’s just common sense.
Special Pleading
So now it’s time for the harder question—the trillion-dollar (and then some) question.
Why does our common sense abandon us when we replace humans with machines?
Why is it easy for us to say we’d dismiss Sam, but so difficult to do the same for ChatGPT, Gemini, and all the other large language model (LLM) competitors?
Why is it that many sensible, reasonable people—who otherwise despise bullshit and champion productivity—act like the boss from this scenario when they work with an LLM? Why do we make excuses for bad outputs, decide the problem is in not doing enough pre- and post-work, and cling to the potential quantity of output as a measure of its value?
I can think of a few potential explanations.
Explanation #1: People think of ChatGPT as a toddler.
One explanation that makes intuitive sense of our credulity: we’ve been fooled by the age of LLMs.
ChatGPT 1.0 was released in November of 2022, which is about 18 months ago as I write this.
It would be massively unfair to judge a human being’s level of knowledge (or moral character) at 18 months old, when they have just learned to walk and talk.
Because LLM outputs resemble human conversation, it is easy to anthropomorphize them. When we think of ChatGPT as a toddler, it is an unfathomably precocious one. Its factual errors seem almost irrelevant—after all, even the most gifted 18-month-olds are untethered to reality.
The big problem with thinking of ChatGPT as a human, of course, is that it isn’t one, and it doesn’t think like one. To ChatGPT, facts don’t really exist. Knowledge doesn’t exist. Words come from probabilities, based on the other words around them in a dataset.
You can put your finger on the scales some, or develop “guardrails” to prevent an LLM from talking about sensitive topics, but you can’t make it stop bullshitting. It is an engine for generating plausible-sounding bullshit.
This holds true even if (and this is a big if) an LLM’s bullshit becomes progressively more convincing and more frequently comports with reality as more training data is added. You may call this Bosch’s Law, if you like: Optimizations to LLMs can only improve plausibility—not accuracy.
Maturity is not the problem. LLMs are categorically not toddlers for many reasons, but chief among them is that they will never, ever grow up.
An LLM can no more stop producing bullshit and start producing factual information than an apple tree can stop making apples and start growing avocados instead.
I think people have a hard time leaving their anthropomorphizations at the door when it comes to … well, almost anything. So this goes a long way toward explaining the situation, but I don’t think it fully accounts for what’s happening. After all, many of the biggest champions of LLMs are hard-headed business money guys, not wide-eyed futurists dreaming of the singularity.
Which leads us to…
Explanation #2: Managerial Faith In the Absolute Value of Productivity
Another great theorizer on bullshit, David Graeber, talked about bullshit jobs, which are “jobs which even the person doing the job can’t really justify the existence of, but they have to pretend that there’s some reason for it to exist.”
Graeber was talking about whole categories of jobs, some of which he probably was being uncharitable toward. But within any large organization, a great deal of energy is expended on productivity without purpose—bullshit production.
Bullshit production within an organization serves multiple purposes, but chief among them is to bolster the budgets and influence of a whole caste of middle managers. When your fiefdom sends a steady stream of reports and slide decks and graphs and charts up to the royal palace of the C-suite, you’re generally considered a go-getter who’s going places, even if the actual results you generate are thin on the ground.
This is why the bulk of MBA coursework today is fundamentally about figuring out a) how to most effectively find metrics that justify your decisions, and b) how to effectively present your case using reports and slides and graphs and charts. It’s the fundamental skill of the job, the way to gain territory and rank in the great game.
So if we think of a human being as having a single unit of productivity, in terms of reports and charts and graphs, and ChatGPT could make 100 units of productivity in the same timeframe, there is a certain kind of manager for whom even the LLM’s loose relationship with reality is an advantage—a scapegoat based outside the organization whose flubs can later take the blame for any failures.
To this manager, the fact that ChatGPT’s long-term value to an organization existing in reality can only be negative is not a deterrent. For them, productivity exists as an absolute value unto itself—a positive is a positive, and a negative is turned into a positive of equal magnitude.
Long-term, of course, a champion of fake productivity is just a drain on an organization, but it’s the kind of drain that’s very complicated to detect and shut down. So they can persist in an organization for a long time.
What will not persist for long: the whole idea that charts and reports and slide decks and graphs are the weapons with which you wage wars of territorial expansion in a big company. The moment a resource’s supply becomes unlimited, the value of adding more to the stockpile becomes zero.
Explanation #3: LLMs as Novel Epidemic
What if we looked at ChatGPT not as a toddler, nor as the holder of a bullshit job, but instead as an epidemic? Another vision emerges.
Smallpox, Spanish flu, Covid-19. What did they have in common? Their most deadly years involved a novel virus encountering a population with no herd immunity.
Television commercials from the 1950s seem ridiculous now—their tactics both banally sincere and over-the-top. You’d have to be a real gullible rube, you think from the comfort of your 21st century chair, to buy something because of one of those.
But people did buy, of course, and as they got enough exposure to gain immunity to those first, primitive ad techniques, the ads got more sophisticated. They mutated, and whatever worked survived.
This is basic memetics, of course, but it presents an obvious worst-case scenario for how the whole ChatGPT and LLM love affair with global business could play out.
Remember Bosch’s Law that I wrote about, roughly a thousand words ago? Optimizations to LLMs can only improve plausibility, not accuracy.
However, even modest improvements in plausibility could potentially be quite believable to an MBA holder who isn’t yet immune to a new LLM mutation. The future becomes a realm where bullshit compounds other bullshit, as fake data generated by one LLM becomes training data for the next.
With each wave of mutation, a new round of VC funds will flow in to create the new, even more plausible bullshit of the future. Every big corporation will be infused with the new bullshit, until it would take a crime scene cleaner to rid themselves of the stink.
In the miniseries Chernobyl, after the titular disaster requires a thousand post-mortems, a character somberly intones: “Every lie we tell incurs a debt to the truth. Sooner or later, that debt is paid.”
Bullshit does not intend to lie. It does not intend to do anything at all, other than please its audience. But when it is a lie, it still incurs the same non-negotiable debt to the truth. Eventually, someone will have to pay the price of making “data-driven” decisions on bullshit data.
By the time someone’s left holding the bag, it’ll be years down the line, and the bag-holders will be people with solid, conservative 401(k) plans who foolishly thought they might retire someday.
Performance Improvement Plan (Offboarding)
If you understand how LLMs work—and that they cannot, and will not ever, learn facts or have knowledge, that they are not fundamentally connected to the world or a shared reality—it becomes obvious that they can provide value only in specific, limited circumstances.
As many companies pour good money after bad trying to recoup AI investments, there is one basic limit that must be observed to prevent AI from generating negative business value:
AI must never be used for anything where truth matters.
If you want to use AI to give you a dozen variations on a promotional headline (the “sentence-level thesaurus” use case), it could generate positive business value—especially since these would require little, if any, fact checking. If you wanted it to generate plausible dialogue for a fictional scenario, same thing.
It could also generate ideas for humans to follow up on, and there could be potential there. AI-assisted drug discovery, for instance, could have potential (but the jury’s still out). Even so, this productivity only happens due to the diligent efforts of many humans experimenting to validate the results.
But relying on it for any informative or persuasive non-fiction is folly. It cannot create value. It can only devalue human contributions while itself contributing nothing to knowledge.
If you work in any job where reality matters:
You already know what you’d have to do to Sam, if he started working on your team.
Don’t be a coward. Do the same thing to ChatGPT.