Normal Cheating

7 min read
ai education law

My home state, Utah, is packed with law-breakers, scofflaws. They do it with impunity, on a daily basis. To make matters worse, law enforcement is complicit.

It’s illegal to drive faster than the speed limit. Utah takes this very seriously; the penalty is not trivial. Anywhere from just 1–10 mph over the limit starts with a base fine of $130. Criminal and security surcharges are added depending on the jurisdiction, pushing the cost north of $200. Obviously you get points added to your license, raising the cost of car insurance. You can do traffic school in some cases to remove the points but not the fine. In fact, traffic school adds to the final bill. All just for going 1 mph over the limit!

Despite the severity, what I described is not how it really works. The way it really works is that 95% of the cars on I-15 (Utah’s primary freeway) drive 5–10 over the speed limit. It’s arguably more dangerous to drive at the speed limit or below because of how it disrupts the flow of traffic.

Not only that, the Utah Highway Patrol is complicit in the daily law-breaking. You will definitely not be pulled over for driving 5 mph over, and rarely for driving 10 over. The much likelier reason you get pulled over for speeding is, again, based on the flow of traffic. I haven’t had a speeding ticket in well over a decade, and my last one was in Nevada, not Utah. I pretty much exceed the speed limit at some point every single day.

I promise not to end this essay by arguing that breaking the law is okay. I have a different point to make.

This past spring semester, Brown University’s economics professor, Dr. Roberto Serrano, gave his students a take-home midterm exam for his Welfare Economics & Social Choice Theory class. This was outside his normal practice of in-class exams, but students had expressed fear of being gathered in person because of a recent shooting and Prof. Serrano relented. In fact, two of his students were among the wounded. One who died had asked him just days before to be her academic advisor.

As for the results of the take-home exam, Inside Higher Ed explains the predictable outcome:

But by the end of the semester, Serrano regretted the decision. Dozens of students in the class likely used artificial intelligence to cheat and earn perfect or near-perfect scores on their midterm, he said. Serrano in turn made the final exam in-person, which led more than a dozen students to drop the course and even more to fail it. Administrators’ response to the widespread cheating event has been “meek,” he said, and the incident has raised questions about how universities can—and should—respond to AI-enabled cheating at scale.

From the article we learn that Prof. Serrano forbade students from using AI on the exam; it was closed-book. But there was no proctoring or other enforcement mechanism. Students were entirely on their honor.

In past semesters, the typical score on this exam averaged from 65 to 80 out of 100. For this midterm, the average was 96. These are the students’ midterm scores compared to their final exam scores, screenshotted from the article.

The mystery heroes here are students #1 and #22. Both impress for different reasons.

Given obvious cheating, it was smart and fair for Prof. Serrano to offer the final exam as a proctored alternative. He also lowered the required score to pass the class, but 18 students ended up dropping and 19 failed the class. It was an academic disaster. To make matters worse, Brown’s administration took a page from Highway Patrol and basically looked the other way. The story gives more detail, but the cheaters got away without further consequences.

The resulting public attention has been, essentially, a chorus of people lamenting both the harms of AI and a total lack of moral character in present-day college kids. I see it differently. I think Prof. Serrano, Brown University, and modern-day education share some of the blame.

I promise not to end this essay by arguing that cheating in school is no big deal. I have a different point to make.

And here’s where I must admit that I somewhat mistreated my fellow motorists by calling them all scofflaws. You see the law in Utah specifically requires driving at a speed that is “reasonable and prudent under the existing conditions,” for which a speed limit can be used as prima facie evidence. That is, driving at unreasonable and imprudent speeds is what’s actually illegal. The speed limit is there as one of multiple benchmarks. All of this means that a highway patrolman may watch me cruise by at 5-over the speed limit and judge that my driving is prudent enough, not a violation of the actual law.

In fact, what many don’t know is that speed limits are set by a process that includes observing the natural flow of traffic on a roadway, used as a guide for what drivers consider to be safe. After all, as long as they can judge the danger accurately, people don’t drive at a speed that puts themselves or others at risk. Not all drivers work that way, but most do.

This standard, though, is not always applied to every roadway. Near my house is a wide road on a steep hill, and a speed limit of only 25 mph. It is unreasonably slow. In fact, you have to aggressively downshift or ride your brakes to stay at that speed. Everyone goes faster, because going faster is not imprudent. But it’s technically part of a residential area, defaulting to 25, even though the homes there are more spread out and set back further from the car lanes.

There is always a degree to which speed limits are arbitrary. I expect that you now see where I’m going.

Here’s a non-exhaustive list of reasons why students consider it okay to use AI to cheat:

  • The class doesn’t cover knowledge needed for their careers, so actually learning it isn’t needed. (Take that, economists!)
  • Everyone else is doing it, putting a non-cheater at a disadvantage.
  • The demands of a heavy semester mean some things need to be sacrificed.
  • In the immediate future, we’ll all use AI every day for knowledge work, so why not use it for schoolwork.

Notice that the last justification is new and, more importantly, true! The future of knowledge work will involve regular and consistent use of AI, at a minimum as widespread as the historical adoption of calculators and then personal computers.

Also worth noting is that the second rationalization for cheating—that everyone does it—isn’t true. This study by Sam Illingworth and others showed that (at least in the UK and Australia) the vast majority of students do not habitually cheat with AI and:

the habitual cheaters who use GenAI most of the time or always when not permitted form 5.2% of all UK respondents UK students, compared to 6.3% in Australia, what may be termed a minority of Moriarties. While concerning, this figure does not back-up the CheatGPT homework apocalypse narrative that all students are using it to cheat…

But we’re getting a little off-track. More to my point, what counts as cheating is almost entirely determined by the professor. We faculty have a wide latitude about how we measure student learning, including what, if any, use of AI we permit. My department chair teaches Public Finance, and he gives students take-home assessments and allows use of AI. This works because he gets clear and fair signals of performance; the students who actually learned do better.

AI is just the latest (and hardest) problem in exam standards. Many of you probably remember wondering exactly what the professor meant by an “open book” exam. Open internet? Open neighbor? If you’ve taught, you know that not a year goes by without a student asking what’s allowed for the test. (This is true even if you had a detailed explanation in the syllabus. Sometimes it feels like I might as well have locked the syllabus in a safety deposit box for how many students actually read it.)

The goal, which dimmed once grades were invented, is that students actually learn. And not just that they learn in general, but that they learn things that are important to the eventual degree earned. A degree, when it works correctly, is a credential that others can use to judge a graduate’s competence.

This puts a burden on professors to do two things well:

  1. Teach what students and the world benefit from being learned, and
  2. Measure in a way that ensures it’s been learned.

Everything else about university teaching is downstream from these two things.

What do we make of the scandal in Prof. Serrano’s class? Back to speed limits.

Speed limits exist to ensure prudent driving, determined by a range of factors, like the road conditions, the area/people around, and what drivers reasonably consider to be a safe speed. When they are set in a way that ignores these things they create either unsafe conditions (if too fast or unpatrolled) or pointless burdens (if too slow).

Students still, and will always, have an obligation to do their work honestly. No conditions change that, just like drivers will always bear responsibility for prudent driving.

In the age of AI, professors have a more urgent responsibility than ever to wisely choose what’s being taught and how learning is measured. We owe it to students to not be careless about these things. Credentials mean something new and different when all the professionals in the field can do their work with the help of AI.

We also need to avoid expectations so tempting and unchecked that our arbitrary limits are guaranteed to produce cheating. I cannot stress enough that it’s not only student attitudes that make cheating “normal.” The blame also falls on the professors who create the conditions that normalize it. Don’t be the downhill road near my house where everyone speeds.

I’m working with a research assistant this summer to make my classes “AI native.” I don’t plan to allow AI in everything students do. But I’m also going to make sure I’m not setting them up to cheat.

Brown Professor Suspects Most of His Class Used AI to Cheat | Inside Higher Ed

Tips on Asking for Help

1 min read
work advice

Very good advice all around, and it almost all applies to people you know and people you don’t.

My last heuristic is stranger: make it easy to say no. You might think that the worst outcome is a no, but the worst outcome is a pressured, begrudging yes. Your coercion will have poisoned your relationship with this person while you feel the false glow of a hard-won victory. A person who helps you with gritted teeth is one who will never help you again. And even then, the help will be a half-hearted effort to get rid of the obligation you manufactured. By contrast, help freely given is effortless, the way you’d hold the door open for someone. Help willingly given keeps your conscience clear, free from the burden of having pressured someone. And help, when given from the heart, is the foundation of a relationship where both of you contribute to what you’re building.

How to Ask for Help from People Who Don’t Know You | Pradyup Prasad

Birthright

4 min read
government law religion

There’s a fascinating detail in my family history in how its lines crossed once, then not again until a hundred years later.

My great-great-grandfathers of the Miller line (Allen Jr., on my dad’s side) and the Todd line (Thomas Sr., my mom’s side) both emigrated to Utah in 1854 after their families in native Scotland joined the Church of Jesus Christ of Latter-day Saints. They didn’t live far from each other over there, but there’s no evidence that the Millers knew the Todds around the time they joined the Church.

They almost certainly met, though, on a ship called the John M. Wood, which carried both families that year from Liverpool to New Orleans. Allen was just six and Thomas was twenty-three. I wish I could go back and see it. I like to imagine young Allen running around on the deck of the ship while Thomas played with his two young sons. Allen sadly lost his baby sister during that voyage, and she was buried at sea.

Both families travelled north to Kansas City to join the Garns wagon train headed for Utah. While in the camp, Allen’s father died of cholera. The “Widow Miller” (as she was called) with her three sons crossed the plains anyway, receiving support from others in the camp throughout the journey. Perhaps Thomas and his wife Margaret were among the fellow travelers who helped them. Even though he was just six, Allen took turns with his brother walking and then riding in the wagon. More than 1,000 miles lie between Kansas City and Salt Lake City.

Upon arriving in Utah, the Millers immediately went hundreds of miles south to help settle the (still) tiny town of Parowan and the Todds found home in less-distant Springville, the families never to cross paths again. But a century later two of their descendants, my dad and my mom, met as teenagers at the Peach City diner in Brigham City where they fell in love and eventually married. So here I am.

Becoming American

Allen’s life inspires me. Before he turned twelve and in the same year, he lost his mom and older brother. Allen and Ninian, the last of the immigrant Millers, were orphaned. A kind foster family took in the boys to their little dugout home until they were old enough to strike out on their own. Despite his desperate beginnings, Allen went on to become a successful merchant, cattle rancher, and mill owner in nearby Panguitch, and he and his wife ended up with eleven (!) children. A detail I love from their life history is that the Miller home is where the teenagers in town came to hang out.

Allen Miller, Jr. (If only I'd inherited his full head of hair!)

The second-youngest of those eleven children was my great-grandfather Joseph, “Grandpa Joe.” I never knew him because he passed away shortly before I was born. I’ve heard he was a pretty gruff guy. My mom remembers him punching a horse because it bit him.

When Joseph was born in 1892, he was born an American, despite his father being Scottish. The 14th Amendment had been ratified 24 years earlier, Constitutionally guaranteeing Joe’s citizenship. That in turn gave citizenship to my grandpa, which meant my father was also a US citizen. I consider myself greatly blessed to be American as well.

When Allen came to the US, he was a reviled Mormon. But his immigration wasn’t against the law, even if he and his family only had refuge in territorial Utah. By the 1870s, though, anti-immigration efforts against foreign-born Latter-day Saints ramped up, culminating in an 1891 ban on Mormons. Otherwise contrary to First Amendment religious protections, it was justified by their practice of polygamy, even if that had been officially discontinued by the Church the year before. The ban’s enforcement carried on into the 20th century.

Being American

I’m not the only American with a family story like mine. Indeed virtually all Americans, except the Native ones, have a story like mine. American ancestors were hated for being Irish or Italian or German or Russian. American ancestors also of course came into the US as slaves, the national sin for which the 14th Amendment became partial atonement.

What should “birthright” mean? Outside of US politics, the word invokes lineage, rights and privileges endowed by the spinning of a roulette wheel. You had no say about your parents, or theirs. To me there’s something powerful in the way the United States—a country formed to eschew centuries of peerage—redefines birthright by granting its precious gift of citizenship to any and all born here, not just to the ones whose parents were already the lucky winners.

I hear the people making arguments against birthright citizenship that this time is different, that granting citizenship for a birth on US soil is a reckless mistake. But has there ever been an argument against the Constitution that didn’t claim that this time is different? I wonder if “this time is different” would persuade them to reinterpret the Second Amendment, or the Fourth, or the First…

All of this is to say that I’m grateful to be a citizen of the United States. I’m grateful for my inspiring ancestors. But I’m especially grateful this week for our shared, national heritage, descended from a world-changing document and its principles, not from any one ancestry. I’m grateful that our country justly preserved the best meaning ever conceived for the word birthright.

Some common misconceptions about LLMs

6 min read
ai

I was talking to my mother-in-law this week about how she downloaded ChatGPT to her phone two weeks ago. It’s her first experience with an LLM. We talked about all the ways she’s started using it, but also the common things that people misunderstand about how they work. It got me thinking that this would be worth writing about.

Send this article to someone who is just starting out with ChatGPT, Claude, or Gemini. I hope it’s helpful.

—

If you’ve been sent this article, it should mostly explain why AI has done stupid or confusing things. It will also help you think better about what AI can do reliably. It’s not a deep-dive, but just covers a handful of common misconceptions.

Quick note: when I say LLM, it’s short for Large Language Model, which is what Claude, ChatGPT, and Gemini all are. I’ll alternate between “LLM”, “model”, and “AI”, but I mean the same thing.

AIs are much more human-like and much less computer-like than most people think.

It’s natural when you’re using a computer program to assume it has all the predictability and precision that computers have. When you type on your keyboard, the letters you press appear on the screen. Unless something has broken, that happens just as expected every time.

AIs are a lot less computer-like than this. It’s not their fault; it’s primarily because they operate on human language. LLMs are created by having a very large computer program learn patterns in all different kinds of human language, including everything from French, English, and Chinese to computer code and math equations. They get very good, but not perfect, at predicting what to say next based on those patterns.

Language makes AIs surprisingly similar to people, because language contains a bunch of our tendencies, perspectives, and ways of reasoning. This is why LLMs can actually even have preferences, which is not the sort of thing you expect a computer program to have. Designers of LLMs wrestle with this problem in the development process, which is when they’re essentially teaching the models rather than programming them. This is why it’s sometimes described as growing rather than building an AI.

You will tend to get much better results if you lean into the humanness of LLMs. Treat AI the way you’d treat a smart intern instead of a computer program. You’d give the intern specific, clear instructions instead of assuming they know something unique to you. You’d ask them to try a task first so you can give them feedback. You’d assume they couldn’t read your mind. And you’d be (hopefully) nice to them. All of these things—even the being-nice part—have been shown to get better results from LLMs.

AIs do not remember.

With LLMs, every conversation starts fresh, as though you were talking to a person with memory-loss. Think 50 First Dates or some other movie about a person with amnesia. This is the unavoidable byproduct of how they operate, not an intentional design.

ChatGPT, Claude, and Gemini do have (relatively weak) ways to get their models to “remember” things by injecting what they hope are relevant details at the start of every new conversation (in a way that’s hidden from your view). These details are gleaned from past conversations. Sometimes it works and often it doesn’t. This also explains why ChatGPT might weirdly throw in a fact it knows about you but that isn’t really relevant to the current topic, and why it has no recollection of a conversation you had a couple of months ago.

The amnesia problem is why there’s so much work being done by people to create memory-style systems that LLMs can call on to help. These mostly operate by giving the AI a way to take notes and, in a later conversation, check its notes. Right now, though, there are no widely used, consumer-friendly ways to give AI better memory.

So with each new conversation, you have to give the LLM the information it needs. You can do that by just typing it in, or adding a document, or using the built-in things like projects, connectors, and skills. If you don’t know what those are, just ask ChatGPT or Claude to teach you how they work.

AIs don’t learn new things.

This next misconception is related to the memory problem. AIs have a way of being both incredibly smart and incredibly dumb, but why they’re dumb might feel mysterious.

What AI “knows” comes from two, and only two, places:

  • What the LLM learned during training by the company who made it.
  • What it was told in the current conversation.

That’s it. What this means is that LLMs don’t really learn anything from you. They don’t get to know you over time, except for what the software might save as a memory. They don’t learn your habits. In fact, they don’t even know who you are beyond what they’re told about you or by you during the conversation. If it feels like ChatGPT or Claude knows you intimately, most of that intimacy is either trapped in a single, long conversation you’re having or it’s imaginary and might be just a general vibe thanks to the model’s training to be friendly to users.

For the same reason, LLMs also don’t know recent events. All models have a “knowledge cut-off date.” This is the date at which the latest training data was collected. So a model with a January 2026 cutoff date doesn’t know that the US attacked Iran in February, or that Jessie Buckley won Best Actress for Hamnet, or that the US men’s team just qualified for the knock-out round in the World Cup.

The only way they can know recent facts in a new conversation is if someone or something tells them so. During the chat with you, the most common way they learn new things is by searching the web. All of the major AI companies basically give their LLMs a version of Google. This means the correct answer is only as good as the search results fed to the model. And again, because of the amnesia problem, if you start a new conversation and ask again about 2026 Oscar winners then they will need to search again for the same information.

AI drinks all the water.

It doesn’t. There are serious concerns to weigh, current and future, in a world with AI. ChatGPT taking all the drinking water is not one of them.

AI today can do far more than most people realize.

Large language models are surprisingly simple machines. They only produce one thing: text. But that makes a lot of things possible.

Text is how we give commands to computers. Even when you point and click, there was text (computer code) that made it work. To make AI more powerful, LLM products like ChatGPT and Claude have been trained to output text that takes the shape of software commands. This is how they make a chart for you, edit a document, check your calendar, or read your email. Basically everything done on a computer can be done by an LLM using text commands that triggers software.

Without these software tools, LLMs are nothing but chatbots. But with tools, they’re frankly amazing. (Note that Gemini is far more limited in the kinds of tools it has than you’ll get with ChatGPT and Claude.) To make the most of the available tools, do the following things:

  • Download the ChatGPT or Claude applications to your computer instead of using the web browser versions. With the software actually on your computer, they can get a lot more done.
  • Try Codex (ChatGPT) or Cowork (Claude). These are tools that make those two LLMs far more powerful. Just ask the AIs how to use these tools and for examples of what you can do with them.
  • Explore Connectors. These are ways for your LLM to talk to other software products you use, like Gmail or Canva. When you tell ChatGPT or Claude to draft an email for you, and then you discover the draft sitting ready to be sent, it feels pretty magical.
  • Pay for a subscription. Much of what I’ve described above is only available if you pay for at least the $20/month Pro plans. Try it for a month and see if it’s worth the money.

AI can teach you how to use it.

If any of this felt over your head, or if you’re not sure what to do next, just ask AI to teach you. If the explanation is too complicated, tell it to explain more simply. I’ve learned more in the last year than I have in any previous year of my life. (Including four years of grad school!) Claude and ChatGPT have been the teachers.

Of course, people can be helpful teachers, too. The best place to learn from people about AI is currently YouTube. So I’ll leave you with this post about my favorite AI YouTube channels.

Jesus Christ and Latter-day Saints

1 min read
religion government

All the discourse on social media over the last week+ has been expectedly pointless after the Dept of Defense listed The Church of Jesus Christ of Latter-day Saints as a non-Christian denomination for chaplaincy purposes.

If you'd like a thorough, expert treatment of what Latter-day Saints believe about Jesus Christ, I can't recommend this one highly enough. It will dispel common misconceptions but also won't sugarcoat differences that matter to Nicene Christianity.

To say that Latter-day Saints worship an utterly different Jesus is, at once, rashly uncharitable, and yet precisely true. On the one hand, it cannot be said that LDS theology teaches a Jesus who is totally foreign to the Christ of Christianity and completely devoid of any similarities whatsoever ... On the other hand, LDS theology irreversibly departs from traditional Christianity ... These differences are very serious and overshadow every shared conviction between Christianity and Mormonism about the nature of the Son of God. So, who is Jesus Christ according to Mormonism?

Who Is Jesus Christ According to Mormonism? | Kyle Beshears

A Fed, but for AI

1 min read
ai government

I'd looked forward to spending more time with Anthropic's Fable model today, but alas the government had other plans. As usual, Zvi has the most thoughtful, thorough commentary on things.

I am however taking the position that the implementation method chosen by the government, with no warning, was deeply terrible, even given our options with our current very terrible level of relevant state capacity, and reflects some combination of at least one of either malice or a deep misunderstanding by decision makers of how jailbreaks and cyber security work.

It makes me wonder if the answer is something like a Federal Reserve, but for AI. Certainly AI safety is a serious enough issue for government oversight, and too serious an issue for reactionary incompetence.

I know it's not a perfect analogy, but at the very least something with political independence, clear measures, and experts who actually understand what they're overseeing would do immense good. The Fed is one of America's best national government institutions that we have right now. It feels like a good watermark for AI regulation.

“What the University Is Now For”

1 min read

The threat that AI poses to the future of higher education would be less imminent if it weren’t for a list of other ways that universities have become worse for students and families. These include:

  • A cost of attending that’s grown faster than inflation for decades.
  • The concentration of donations to the most elite institutions where the marginal benefit is the lowest.
  • The underfunding and politicization of state universities by legislatures.
  • Degree requirements detached from the needs of the labor market and the students entering it.

Dr. Shehu notes much of the same in this provocative article. She has a unique vantage point with her background in education and AI. Higher ed has survived attacks on its legitimacy in the past, but AI is something new that poses a stronger threat.

As much as we want to think this moment is unique, we stand at a similar inflection. The question is the same. The pressures are different. The hedging that has carried the American university through the last half century is no longer available, and the institution will have to answer, in language its students and their families can recognize as honest, what it is now for.

The article doesn’t—and I don’t either—make the case that higher ed is doomed, but it will absolutely need to change to better meet the needs of our students and communities.

The university has told one story to its trustees, its accreditors, and the public, which is the story of the holistic education, the formation of citizens, the cultivation of judgment, the well-rounded life of the mind. It has told a different story to its students and their families and the labor market, which is the story of the credential, the ticket, the signal, the return on investment. The two stories were never quite compatible. They were held in suspension by an institution wealthy enough, slow enough, and culturally trusted enough that no one had to choose.

What the University Is Now For - Amarda Shehu

Becoming Improbable

Your life’s goal should be to become the most improbable person you can be. Your path, your character, your life, should be the most unlikely, the most unexpected, the least predictable version you can make.

This really struck a chord with me. I’ve had a unique and improbable career and I’ve often found myself feeling less than others for it. (A professor without a PhD, a law student who didn’t want to be a lawyer, etc.) This piece by Kevin Kelly—one of the great improbables—helped me appreciate my improbability.

Your Most Improbable Life | Kevin Kelly

Chesler Park & Fiery Furnace

Chesler Park & Fiery Furnace

2 min read
photography

Katie and I knocked out two bucket list hikes in one trip, Chesler Park in Canyonlands and Fiery Furnace in Arches. Easily one of my favorite trips we’ve ever done. Here are some of the photos.

We stayed in a perfect little cabin on highway 211, called Marie's Place. Strongly recommend.

Chesler Park loop has two standout features in addition to the Needles: the Joint Trail (one of the most impressive and varied slots I've ever done) and the meadow surrounding the Chesler Park formation. Look for the little hiker on the trail in the bottom right photo to get a sense of scale.

Fiery Furnace in Arches National Park requires a permit to hike it, which you need to get on the website a week in advance, right at 8am. You have to watch a training video, in part so you learn how to not get lost down in there. My favorite part of this hike were all the secret features hiding just past the dead end signs.

“Academics Need to Wake Up on AI”

1 min read
ai education science

The third of a series by Alexander Kustov on AI attitudes among academics. I found the entire series to be quite persuasive. I suspect before long that AI use in academic writing will be common as long as its use is disclosed.

But the idea that AI use in research somehow pollutes it needs to go. Researchers are at least as likely to produce slop as AI is.

Meanwhile, academics routinely cite papers they haven’t read beyond the abstract. At least AI hallucination rates are tracked and improving. Human hallucination rates in academia are not tracked at all. We just call them “contributions to the literature.” And if you’re a peer reviewer, you don’t even have to hallucinate on your own: you just write “please cite me” and move on.

Also, another smart point about the “stochastic parrot” metaphor I wrote about this week.

One of the most influential slogans in the AI debate has always functioned as a thought-terminating cliche. As Cate Hall observed, it is a potent coinage: fun to say, conceptually efficient, and it has permanently colonized many people’s minds despite not being true of today’s models. A genuine linguistic work of art. It is also empirically false: every major frontier model since GPT-4 has been trained on non-textual input, and the original argument’s own logic requires text-only training to work.

Academics Need to Wake Up on AI, Part III

Humanity’s Best Invention

4 min read
ai tech

“Stochastic Parrot”

If you’ve followed the AI discourse even just a little bit, you’ve probably come across this dig at Large Language Models like ChatGPT and Claude. As accused, LLMs are just unintelligent “fill-in-the-blank” machines with a layer of randomness to boot, i.e. stochastic parrots. Or, as another common criticism goes, LLMs are just an “average of all human expression”. And who wants average?

At a technical level, this isn’t incorrect. Even the frontier LLMs today are doing what GPT version 1 did eight years ago. Given a length of text, they produce the next bit of text (a “token”) repeatedly until they’ve made a new passage. Each next token is chosen with some degree of randomness, with the likeliest tokens having been learned during the LLM’s training process. The training is done on a massive body of text, mostly scraped from the internet.

Of course, the difference between GPT v1 and v5.4 (the most current) is like the difference between a weed trimmer and a Formula One car, even though both of them just compress and ignite gasoline in a cylinder over and over. What the AI researchers and developers have been able to do with next-token prediction is turn LLMs into skilled software developers, patient tutors, and all other kinds of human-like products.

Despite these abilities, many people still dismiss what a modern LLM can do with the right software around it. And it isn’t just the technical feat. Critics who scoff at modern AI as a mere slot machine that spits out text, or as an “average of all human expression”, are missing what’s really going on. What’s being overlooked is how the power of LLMs lies in the much older technology they’re invoking: language itself.

Our best invention

I’m convinced that language is humanity’s best invention. It’s how we store information across space and time through stories, histories, lessons, and facts. We use it to bridge the gap between our brains, which are otherwise completely opaque to each other. It gives us a way to externalize ideas so everyone can participate in them, whether in the form of sound, code, math, or letters. Language is how people coordinate and cooperate, and also how they compete and fight.

What’s the average of all of that?

Perhaps the problem is that there’s so much language that’s average (including, perhaps, this article). We’re more often bored or annoyed than moved by the language we encounter each day. Notably, we were surrounded by slop before LLMs came along, inundated with advertising, social media, and political entertainment. But even those mediocrities prove the point that language is powerful, otherwise why bother producing so much of it?

When we think of technology in the context of language, it’s typically media that come to mind, like the printing press, the telegraph, radio, television, and the internet. But language itself makes a strong case for being a technology, too. It’s an invention that we’ve refined over thousands of years. It’s useful, learned, and does things for us that we couldn’t do otherwise. I think it serves us well to think of language this way.

And like any other technology, it’s the foundation for more advances. In the case of language, there’s not a single human accomplishment that doesn’t rely on it. Medicine, law, engineering, computing, art, and so on. It’s the core technology of education. We’re hard-pressed to come up with a more important invention than this.

Language brought to life

With AI, we’re feeling the sharp edges of our best invention. LLMs harness the power of language, but with speed and ceaseless energy and often without good judgment. AI psychosis, for example, is a thing because of how powerful language is. We’re all far more persuadable than we believe ourselves to be. It’s genuinely scary to think about what happens when persuasion is automated.

This is why I’m far more sympathetic to the people who are worried about AI than to the people who are dismissive of it. I don’t think it’s wrong to be scared by the prospect of not knowing if you’re talking to a person. The Turing test is far away in the rearview mirror at this point. People have always used language well, and poorly, and dangerously. To be nervous about what LLMs can do is just to be nervous about what language can do, but at scale.

Ironically, the original source of “stochastic parrots” is a paper from 2021 where the authors discuss the risks of large language models that have the capacity of language without the human ability to understand it. The term is meant to draw attention to the perils of the technology, not to pooh-pooh it.

For me, the most interesting detail is this one: LLMs aren’t just a medium, like books and the web. AI uses language to do things. Each token in the flow changes what comes next. The closest thing we’ve had to this is software code, but code doesn’t generate itself in anything like the same way. It’s as though we brought language to life.

Reasoning models are perhaps the best example of what I mean, altering the way they work by talking to themselves. They generate tokens for their own benefit, to help them “think” about what the user wants, what the next step ought to be, and what they might be getting wrong. This is, frankly, incredible. Reasoning models have, in turn, opened the door to agentic AI, personal assistants that are now taking shape in exciting ways. And all of this only works because language with all its complexity and nuance and density makes such things possible.

I’ll finish with this. As I was workshopping this article with Claude, it gave me this gem:

“Just text" is how humans declare war, fall in love, spread conspiracy theories, write constitutions, and teach children right from wrong.

Not bad, if a bit dramatic. Also, not at all wrong.