Ben Riley: We Should Be Skeptical of AI

In which Dan chats with Benjamin Riley, founder of Cognitive Resonance, an organization dedicated to replacing AI hype with an understanding of how generative AI works using cognitive science. Ben's understanding of human cognition has him disturbed to see how some in education are running into the arms of big tech. 

As always, I welcome comments and questions on Instagram @_dankearney_

Mentioned in the show:

Cognitive Resonance, founded by Ben Riley
Gary Marcus on Substack
8 Things To Know About Large Language Models by Sam Bowman
Ex Machina, a film by Alex Garland
The Story of My Life by Helen Keller

Music by Soyb

Transcript

SPEAKER_01 0:00

The organizations and entities in this space that have been promoting computer science and sort of uh technology more generally were saying just a few years ago how everyone needs to learn how to code. And that was a prediction about the future, and the future is going to require that, and you need to learn how to do that. And it's like along comes this tool that quite literally now says, actually, you don't need to learn how to code. You actually don't need to do that. So to me, it's like maybe what we should be doing in education is moving away from the ultimately doomed uh notion that we can predict what the future will hold, and instead, like focus on what we actually know about our own development.

SPEAKER_02 1:06

I just attended a very cool one-day meetup here in Los Angeles, where I was part of a panel talking about how schools can build and preserve resilient cultures in a world of rapid change. The panel included some local heads of school and directors of technology, and while our chat was about change more broadly, the day's focus was clearly on AI. The presentations and conversations were a mix of optimism, worry, practical ideas, and regurgitated industry talking points. I guess that's where we are right now. Recently I spoke with Michael Wagner about the nature of assessment in the era of large language models, and then Eric Hudson about how to demystify AI for teachers. Today I have for you a great conversation with a skeptic of AI, someone whose understanding of human cognition has him, I think he would say, deeply disturbed to see how some in education are running straight into the arms of big tech. Ben Riley is the founder of Cognitive Resonance, an organization dedicated to replacing AI hype with an understanding of how generative AI works using cognitive science. In our conversation, he tells the story of Helen Keller's awakening as a thinker as a window into what AI can and cannot do. It's really fascinating, and I've been thinking about this framing that Ben proposed ever since we spoke. Now, my plan wasn't to do three episodes about AI in a short time. I mean, you'd be forgiven for wondering if I'm just turning this into an AI discussion podcast. It just kind of worked out that way. But I do encourage you to go back and listen to Michael and Eric after listening to this chat with Ben. My hope is that I've given a nice arc of different ways to think about what has quickly shaped up to be one of the biggest potential change agents in schools in a long time. Okay, here's Ben Riley.

SPEAKER_01 3:20

And I am the current founder of an effort, a virtual organization called Cognitive Resonance, which has a mission of building human knowledge to halt AI hype. And previously I was the founder and executive director of an education nonprofit called Deans for Impact that continues to exist and continues to work to improve teacher preparation, and as part of that, uses principles of cognitive science to improve teacher understanding of how students think and learn.

SPEAKER_02 3:52

Great. I think I saw you referred somewhere as noted AI skeptic.

SPEAKER_01 3:58

Yes. The New York Times uh called me just last week regarding the resignation. I don't actually resignation is not the right word, the forcing out of the superintendent of LA Unified in relation. It's a really crazy, very complicated story. But it does tie to AI. And the question there was why did it ever make sense for a school district to try to create a custom chat bot? And the answer is it never did. It didn't make sense then, it doesn't make sense now. Um so I was pleased, I have to say, that uh Dana Goldstein, the reporter, described me as an education policy expert and noted AI skeptic because indeed we all dream of being noted. That's just yes. Well I've known Dana for a long time. She used to write for the American Prospect and um and do so very well, and now she's reached that perch. And I I think I can say uh on air that I have at times pushed her a little bit because she's quoted people who I don't think understand the technology who are promoting it in education. And we during the inter during the the interview, I sort of said to her, you know, I wonder if I should aim my efforts more focused at journalists so they understand this tool. And she said, no, because there's no money in journalism. So I don't know. I don't know where to take it from there.

SPEAKER_00 5:16

But um but uh I was glad to at least you know be able to say to folks from the platform of the New York Times school district superintendents, if you read this, don't do this. This is a bad idea. Yeah.

SPEAKER_02 5:28

I've been enjoying your thinking on your Substack for a while. And and a lot of it is the type of stuff that I think educators should be wrestling with. You know, that LA effort always felt very superficial, very how quickly can we jump in? I think for a lot of people, there's sort of when they're thinking about AI and LLMs, there's before the first Chat GPT and then there's after. How has your thinking about AI evolved? You know, your understanding of what it can do. How has that sort of been shaped over the last few years?

SPEAKER_01 5:59

Yeah, it's a great question. And really everything that I'm doing is born of my own effort and desire to get some mental model of what was happening. So I'll tell the slightly long version of the story, but hopefully not too long. Um back in some point, I would say in the late 20, how do we call it, 20 teens, I became aware that there was something going on called deep learning. All of a sudden, this phrase deep learning started to pop up. And I have to tell you that artificial intelligence is considered like a discipline of cognitive science. And as someone who has spent a lot of their life and career thinking about cognitive science and considers himself kind of an amateur cognitive scientist, I always paid no attention to the artificial intelligence branch. Not because I didn't think it was worthy of intellectual pursuit, but because I'm interested in humans, first and foremost. But nonetheless, when all of a sudden there started to be these bubblings of this thing happening, I decided I needed to at least try to figure out what was going on. And um I ended up contacting Gary Marcus, who has become, I think, one of the most well-known um AI skeptics, quote unquote. Um, and at the time he was really talking about deep learning before we had large language models and these tools. And and for reasons I am grateful for but still didn't understand, he agreed to get on the phone with me and and sort of talk with me about these tools. And at the time, I was actually curious about whether we were tracking towards a place where they could be used by teachers, particularly teachers who are preparing to become teachers, teacher trainees, as ways of practicing teaching that were zero stakes, you know, when we know it's hard to become a teacher, and especially when you're young and you're a novice, you're dealing with real kids and you're worried about it. It's stressful. So could we create teaching simulators? And Gary was like, yeah, probably not. That's that's probably not gonna happen. Okay, so so then flash forward a few years later, and all of a sudden, November 2022, uh, Chat GPT is commercially deployed. And even then, I was sort of like, yeah, okay, like this is definitely more robust than I thought it would be. Uh, but I could still see it making errors very quickly, and it just was sort of like, you know, it's it's better than. I mean, we've had chatbots going back 75 years at this point practically. So it's not like this new thing, at least to me, but it was it was still clearly more versatile. But then it was in early 2023, and I really think this is like one of those accidents of history that kind of created this frenzy that we're in, that they release, you know, a new model that all of a sudden the jump in capability is everyone is stunned. And I remember there was this slew of articles coming out where people who had said first time around, a lot of educators, they were like, uh, especially at the university level, they were like, Yeah, I gave I gave ChatGPT my quiz, they couldn't answer it. Well, now they could. And and I will never forget, uh a good friend of mine, a teacher in New York City, a wonderful teacher, thoughtful teacher named Michael Pershawn, posted on Twitter this article that had come out from a guy named Sam Bowman, who was at NYU but also at Anthropic. I think he still might be at Anthropic. I haven't checked lately. And he basically said, here are nine things about large language models that we need to know. And it was very accessible. I could sort of read it. And he was making the claim that this progress that they'd made will continue indefinitely. I mean, he literally makes the claim that we now know, unlike any other software development in history, that if we just feed it more data, these capabilities will continue at the trajectory that we've just seen. And I thought, well, is if that's true, then something significant truly is happening. I remember reaching out to my father and it started a dialogue with him saying, Dad, have we done it? Like, are we on the way to creating digital sentience? You know? Um, and that that kicked me into gear to try to figure it out. I was like, now you've got my attention, artificial intelligence. I want to try to understand how you're doing this. And so I just began a self-education process that, you know, I won't overly describe, but I reached out to people I knew in the world of cognitive science. I reached out to people I didn't know. Um, I remember one person that was really helpful to helping me build a mental model of these tools, someone named Murray Shanahan, who's at Google Deep Minds and also an academic, and also, interestingly, was the consultant for the very interesting movie Ex Machina, which is uh yeah, worth worth watching. Sounds like you uh looks like you may have seen it. Um he's written he wrote some brilliant things and still continues to that sort of help me start to figure out really what this was. Another person I'd point to, excuse me, Eric Silvaggio, also was writing these blog posts, but also diagrams that were sort of like, here's how this works at a sort of high level, you know, sort of not getting so into the details of things like backpropagation and autoregression. And so at the end of that, I realized, man, I just gave myself an intense education. I feel like now I do get how they work and how different it is from human thinking and cognition. And I wonder if other people would, it would be useful for them too. And so from that, cognitive resonance was born.

SPEAKER_02 11:27

So cool. I love just the self-education element of that and just the whole community out there of people just wondering how this all works. I find myself swinging between really impressed with the things Chad or Claude can do and stunned by the idiotic mistakes it makes. I mean, just that last week I was planning on taking my kids to this big amusement park here in Southern California, Knott's Berry Farm, and I just Googled um the hours. But of course, now Google, it just immediately shifts to Gemini for everything. You've got to get past Gemini. And the first thing Gemini said was, oh, the park's closed on the weekends.

unknown 12:01

I was like, okay.

SPEAKER_01 12:04

It's really remarkable. There's a neuroscientist who I've written a bit about on my Substack. His name is Paul Sisek. Uh and he um he made a really important point when I interviewed him in one of the essays that I have on his work, where he says, you know, when it comes to science, it's the failures that sort of dictate what you know, sort of what matters. So he made the analogy to physics, where if you compare Newtonian physics to Einstein's theory of relativity, like from a use case scenario, Newton's physics are better or more useful in many ways, and we still use them to a large degree. But it's when you get to that point where suddenly there's a prediction that the theory of relativity can make that Newton's theory does not. And then when the empirical tests are done that show that Newton is wrong and that Einstein is right, that the theory of relativity displaces, you know, sort of the then existing understanding of the world. And so these failures that you find and you're experiencing are pretty important, I think, to recognizing that like whatever whatever utility you may be getting out of it, and you can certainly get that. And I sometimes think that the people in my world of AI skepticism who want to deny that they're ever useful are really fighting a losing battle on that front. Um, but it's the failure rates from a scientific standpoint that can make us conclude, I think, with a fair degree of certainty, that they are not capable like we are. They are not intelligent in the same way that we are. They are not cognitively impressive in the same way that human beings are.

SPEAKER_02 13:48

You brought up that statement from Anthropic about this trajectory that we're on. It's just gonna keep getting better. Interesting that it'd be for anthropic. Their lane seems to be transparency and safety, whether they're actually accomplishing that. Yeah, okay, you're shaking your head. But I think that statements like that have created this at least facade of inertia that has things, places like schools feeling like, damn, if we don't get on board with AI, we're gonna be behind very quickly. And a narrative uh in the workplace that this is inevitable and adoption and the money that goes along with adoption is just a you know, a given.

SPEAKER_01 14:29

Yeah, the the so the funny thing about this is there's a a paradox. I think it's kind of a stupid paradox that sits at the center of that contention. Because if it's true that these tools are going to continue to get more and more powerful and more and more functional, then like A, it where you're you're trying to hit a moving target in terms of being able to use them now, because already we've seen them change quite significantly in the last couple of years. But B, the whole value proposition is making it easier to use them, right? I mean, that's it's it's fundamentally strange to me that there's sort of like this effort within the education space to say, well, the future is coming and therefore we need to do all this. And it's like, and create AI literacy. And it's like, well, no, the whole point is the simplicity that you can communicate them in natural language function, that it's not a specialized tool. So I I find it fascinating that people in education feel like they have to do something about this. And it's particularly fascinating because this is a tool of software, and the organizations and entities in this space that have been promoting computer science and sort of uh technology more generally were saying just a few years ago how everyone needs to learn how to code. And that was a prediction about the future, and the future is going to require that, and you need to learn how to do that. And it's like, along comes this tool that quite literally now says, actually, you don't need to learn how to code. You actually don't need to do that. So to me, it's like maybe what we should be doing in education is moving away from the ultimately doomed uh notion that we can predict what the future will hold, and instead like focus on what we actually know about our own development, about our own cognition, about the role that knowledge plays in that. Stay anchored to what we actually have real robust, not only evidence, but I think experience over all of humanity to know how important sort of the the cultivation of the mind is. Get out of this skills prediction game.

SPEAKER_02 16:40

Yeah, you you like to write about the necessity that we resist the temptation to cognitively offload everything to these LLMs. And I that's something that we talk about a lot in schools. Once we get past sort of the first layer of our kids cheating, and we start to actually think think about AI, it's it is a lot about how can we promote deep thinking without turning it over to um the LLMs. Which leads me to one of the questions I sent you. I'm sorry. Uh one of the questions I sent you, my guests often tease me for the um giant questions I'll sometimes pose. They're like, Do we have two hours? But I wrote to you, what is human thought? But I did want to give you a chance, as best you can, Ben, to explain to me and my listeners, when we talk about cognition in humans, what are we really talking about?

SPEAKER_01 17:32

So it is a huge question. And as I shared with you before we started recording formally, it's one that derailed my day yesterday in the happiest of ways. I spent hours thinking about how I would answer it in a way that I thought would resonate and not get too technical. And it it led me to what I'm about to share with you. And it will, it will go at length. So um, if you need to jump in uh at some point, so I don't I don't drone on for too long, because I want to tell a story, and it's a story that um I think most of your listeners, if not all, will have some background knowledge of, but I want to to unpack it. And it's the story of Helen Keller. I think that most people are familiar with who she was. I think that they know her teacher was named Ann Sullivan. Um, I sometimes call Anne one of the few famous teachers in history. Um, and they know that moment about water and when she, you know, suddenly understood what was happening there. I think that's like, for a lot of people, that may be sort of the start and end of the story of Helen Keller. And it's so much more than that. And I'm talking now not about the full arc of her life, but I'm talking about um her childhood and sort of how cognition emerged in her. So what I want to do is kind of tell that story because I just think it's like, yeah, it's really um, we'll we'll walk through it here. So, first of all, Helen Keller was not born uh blind and deaf. It's something that happened to her after 19 months. And in fact, she was at an age where she was actually starting to say words in her autobiography, um, just prior to the illness that took away her sight and her ability to hear. She was saying words like tea, um howdy, which is weirdly spelled in her autobiography. I don't know if it's howdy or how is the day, and perhaps significantly water or wahwa. Then the illness befalls her and they thought she would die. Everyone thought she would die, but she lives, but now she no longer um has sight, she no longer can hear. And it's very interesting to me how she describes that experience. She describes it as uh being plunged into the unconsciousness of a newborn baby. Plunged into the unconsciousness by her own words she could feel uh a removal from the world around her, and she felt unconscious to it. She was still sentient, she was still thinking. Okay, she developed um rudimentary signs, a very basic sign. If she wanted food, she learned to sort of cut her one hand across her palm in order to show she was slicing bread. When she would fold laundry, she could recognize her own clothes and she would separate them out. But her life was miserable by her own description. Every day she dreaded there was no distinction between night and day for her. Um she would later come and describe knowledge as light. Light features so heavily in sort of what ultimately occurred. Okay, so she has this rudimentary vocabulary, but it's not it's not even vocabulary, it's the wrong thing. It she has like signs, very basic signs, um, according to some accounts, maybe forty or fifty of them. And then Anne Sullivan arrives, and there's a whole story to be told about that, but we're just going to skip to Anne Sullivan arriving. And she begins making uh motions into Helen's hand in relationship to objects. And this is very interesting. One of for Helen, it was a delight, right? Like this was fun for her. She was like, Oh, this is weird, fun new game. I have an object and then I I have to do this. And she learns quite quickly to pantomime the very same signs that Anne Sullivan, her teacher, is making. She has a doll in one hand and she feels the signs coming from um from Anne, and then she does it back. Okay? She's learned something here, but it's very interesting what Helen says in this moment as as a child. She says, I did not know I was spelling a word or that words even existed. I was making my fingers go in monkey-like imitation. I'm gonna call that the first cognitive transition. She's uh imitating. She's parroting, you might say. She doesn't know what's happening, but she knows something.

unknown 21:56

Yeah.

SPEAKER_02 21:56

And you say transition, what's the transition from what to

SPEAKER_01 22:00

That that she is now able to recognize from Anne that there's something she can do in relation to this particular object. And it's fun. It's a very limited transition. Okay? It's a very limited transition. Some might even say it isn't really a transition. She was just adding another sign as like those 40 or 50 that she already had. So, okay, it wasn't long after that, very close in time, that the famous incident occurs. She goes out, she puts her hand underneath the running water, and Sullivan does it again. And now there is the cognitive breakthrough. She understands that this thing flowing over her hand, this water, is the word for this thing. The water is impermanent, right? It's not like the doll. The water is gone. Suddenly she understands this is a word. This is a word for something. Her description of that moment was that the living word awakened my soul, gave it light, hope, joy, and set it free. From that point forward, that I'm going to call the second cognitive transition that she had. Her entire world changes. Her entire world changes. Okay, she begins to every day voraciously seek out the words for everything. She's naming every object. She's learning. She's been deprived of what children normally have in that moment. She doesn't have visual input. She doesn't have audio input, but she has this, and she's putting words and she's classifying, and it's just, she describes it again, her soul is awakened, but it's still object classification. It's still object classification. Now an interesting moment happens as she starts to get to more complex words. There's a very interesting moment where they're talking about love. I say talking, they're signing within each other's hands about love. And Ann Sullivan's explaining to her that love is something that happens between people, between things that are alive. And Helen wants to sign that she loves the sun. Because that feeling of warmth, one of the things that she can experience, she feels love for. And Anne signs, no, that's wrong. And this is very confusing to Helen because she feels it in her mind. There is love. I'm going to call this the third cognitive transition, which is that there are now two human beings who do not have the same understanding of what's happening in the words that you're using they're using. The cognitive scientists call this theory of mind, right? And Sullivan, as brilliant and wonderful and heroic as she was, could not quite understand why Helen would be saying love and sun and expressing love towards the sun, because she perhaps in that moment just couldn't imagine, oh, of course, for Helen the sun is something different than it is for me. It is one of the few things that she can feel differently. Okay, so now cognition is really flowing, but there's still one more final step, and this is actually, in some sense, I think what turned Helen Keller into one of the major figures of human history and why her story so neatly encapsulates what is unique about us as human beings. Remarkably, um, she's starting to learn math. She doesn't like it, it's not her favorite subject, but Anne Sullivan is making her work with a, I don't know if it was an abacus, but a math tool to do sums, and she's really struggling. She's really struggling. And Anne Sullivan makes the sign within her palm think. And it's at that moment I'm going to read to you exactly how Helen Keller uh said it happened for her. That word was the name of a process that was going on in my head. This was my first conscious perception of an abstract idea. Suddenly she understands think. Think does not exist like an object, like a doll, or even uh an experience like water flowing over your hand. It's an idea. Thinking is abstract. Human beings uniquely are capable of forming abstractions that we can then use to navigate our worlds and to encounter new problems. I'm gonna call that the fourth cognitive transition. And at this point, at this point, Helen Keller, despite the limitations of sensory input, she is now at the same level of cognition as any human who has been given the opportunity to have this education. I'll stop there, but I want to quickly turn to where we might think AI sits within those cognitive. I just I wanted to tell that story because I think it it's it's so so perfectly encapsulates what happens to us, although we don't realize it because we have so much more. We can take in sight, we can take in sound. The cognitive scientist Alison Gopnick talks about, you know, children are little scientists. They're constantly gathering evidence and forming hypotheses and testing these hypotheses and knocking them over and facing consequences. And Helen Keller describes just how deprived her experience was of that and how much harder it is for someone who is deaf or someone who is blind. I think any teacher who has students who have any special needs or disabilities of any sort or even linguistic um impediments, like it takes more effort. The amount of effort that had to go into educating Helen Keller. Um, I both love the movie, but hate that the movie about her that I remember watching in elementary school in the 80s is called The Miracle Worker, describing Ann Sullivan, because it's not miracle, it's not miraculous. It was a human teacher dedicating themselves so that Helen Keller could think.

SPEAKER_02 28:13

I mean, is there is there something about the way that she developed that actually gives us insight that we would maybe wouldn't get from observing a child that you know didn't have disabilities?

SPEAKER_01 28:26

Helen's story speaks to the process that I actually think we all go through. It's just that what what she actually was it was happening to her between the ages of 19 months and seven years old. For us, we sort of go through that when we are pre-conscious or pre-sort of fully aware that we eventually become. It's why we struggle to memory to remember things from that point in time. We we lack the language in order to articulate them. Um, and then, as I think we're about to turn to or should, I think it has some important implications for how we think about what large language models really are.

SPEAKER_02 29:15

It's so inspiring, but that she's able to articulate later these realizations that she comes to. And you know, as teachers, we we dream of the aha moments, whether it's in skills or whether it's in abstract thinking. And um, you know, it sounds like Keller had the most aha moments a human can could conceive of.

SPEAKER_01 29:39

Exactly. Exactly. For us, it happens gradually, but for her, it was, you know, she was advancing, you know, intellectually in her own sort of subscribe, you know, circumscribed, darkened world. But then for for it really was this unlocking of cognition. We don't, I don't think, typically have that as sort of like a singular moment where we suddenly understood what words were. That happens for us gradually. I mean, human beings are unique as a species that we have this extended adolescence period that, depending on how you look at it, lasts into our 20s. I mean, that is unreal compared to all of the other species on this planet. And it speaks to the incredible cognitive development that we go through and inculcate in our cultural practices, like public education. And we say, no, actually, everybody has to do this. That's under an extraordinary amount of strain right now, and I'm quite concerned about it. But but nonetheless, we have made as a species, by and large, a commitment to saying the cultivation of your mind, the building of knowledge, the improvement of your cognition is something that you are entitled to as a human being.

SPEAKER_02 30:51

Aaron Powell You talked about unlocking cognition. I think companies like OpenAI and Anthropic would probably have us believe that in the last few years they've unlocked some kind of cognition at our fingertips. But that's not really true, is it?

SPEAKER_01 31:06

Well, so I think this is the question. And I think the answer to it will vary depending on what perspective you take and sort of what you think can happen from a process that we have developed as a form of software. So to recap, there I identified four cognitive transitions for Helen Keller. So the first one is when she starts making the same finger movements for Dahl, she parrots them uh as Ann Sullivan's giving her those finger movements. So I think large language models 100% do that. In fact, one of the most seminal papers written about them is called, you know, On the Danger of Stochastic Parrots. Uh stochastic referring to kind of the statistical process that they use, the forecasting for what words they should produce, but parroting meaning that much like at the time Helen was just signaling back the same things without any real connection forming, consciousness forming, um, that they they lack that consciousness completely. Okay? So I think large language models are there. And that's why I say it's even debatable whether or not that was a cognitive leap or if it's just sort of a basic state. But it is true that not all species can imitate or mimic. I do think that humans and other hominids are more capable of that. Certainly parrots are incredibly capable of that, uh, some of them when it comes to certain noises. So mimicry is a cognitive skill, okay? Now that second cognitive leap, remember, was when she suddenly realized that there are words. That's when the water is flowing over her hand. And now she's object classifying. She's going around and making connections of the words to the objects, okay? And it's very straightforward. This thing is connected to that object. Now, is that similar to what a large language model does? Well, not really, because they're not going around and actually figuring out that this word is connected to a physical object out here in the world. They're only doing it by words to other words. But as I think we can see, it turns out having all that data they've trained on and all of the words ever, you know, uh put up on the internet um available to them, that there does seem to be quite a bit of value in terms of just, if nothing else, linguistic fluency and coherency to what they say in just making those connections. I mean, that really was the technological breakthrough that got us here. You know, Google was literally working on translating French and German to English, and suddenly they realized, well, wait a second, if we do this with these now powerful chips, we can we can look across these linguistic data sets in much more rich detail. So maybe, maybe they're there. I would say for me, I don't think they are, but I think there's a plausible argument to be made on the other side, right? Now we get to that third transition where we have the mismatch between Ann Sullivan's understanding of what can be the object of love and Helen's belief in that she could love the son. Here you have two human minds with abstract ideas. Those understandings of those abstractions are not exactly the same. When I say justice to you, you might agree with what I think of as justice, but you might not. I mean, we are certainly undergoing a moment in the United States, if not the world right now, where disagreements and misunderstandings about what we are talking about conceptually and abstractly are dividing us into very scary and dangerous places. But here in that micro example, you have two human minds that have not met, we would say. Now, there is no mind on the other side of the large language model. There is nothing over there that it's trying to express other than you have put in some text into the device. I have been trained to output text using a very complicated statistical process using words like backpropagation and auto-regression. Now I will output that. So that's where I get to the point where we're not ever going to see large language models be able to develop a robust theory of mind because they don't have minds of their own. Now they can they can emulate it, they can perform it, because there's text written somewhere about what theory of mind is, but they are not a living thing thinking and trying to share ideas with another living thing. They're not. And then you get to that last and final fourth transition, the understanding that of abstract ideas. In some ways, it's the same one. It's just now sort of moved off a specific between two humans, but we're just totally internalized. Here's where I really think scientists, even some very smart ones, get really confused, even ones that have won Nobel Prizes, like Jeffrey Hinton, who is responsible for some of this, they sort of think that we can get backwards to it. That, like, if we just keep feeding it the data and the words, that somehow that abstraction will emerge from these, that it will have sparks, it's said, of what they call general intelligence, artificial general intelligence. I just think they get it exactly backwards. The process works the other way. We have a world that we live in, that we are a being that has will that acts upon it, and for that reason we are trying to experience it. And it turns out that there are some things that are made more uh powerful and more deeply resonant once we start moving to those abstractions. And so we we develop the words in order to get to those thoughts, and we connect up those words in order to empower our thoughts. I just don't think you can get their reasoning backwards from using statistics and training data, and an awful lot of money and an awful lot of power is lining up on the other side. So it's very much a David versus Goliath situation.

SPEAKER_02 37:20

Yeah, I think I've am I correct that I've I've read somewhere that Meta and and OpenAI are putting a lot of money, tons of money, into people that they hope can develop AGI, I guess. I guess like that third and fourth cognition, they're actually their hope is that that breakthrough is coming.

SPEAKER_01 37:43

Yes. I mean 100%. It's it's some of the salaries, and this is um uncomfortable to talk with with teachers given that their work is so much important in my view than than these people who do this work. But their salaries, they're like athletes now. They're getting poached at the price of $100 million or more, and it's it's ridiculous. It is interesting you mentioned Meta just because um for a long time their chief learning scientist, or I'm not sure what the exact title was, but their chief AI guru um is someone named Jan Lakun. And um he's won the Turing Award for his work in this area. He's someone who spent his entire life thinking about how to potentially emulate the process of cognition uh or even recreate it artificially. And he has been very consistent in saying large language models ain't it. And for a while I actually thought, you know, I wonder if Meta will actually get this right, because Jan actually has pretty sharp insights, um, very sharp insights into the limitations of just using these statistical processes. Even in the peak of the initial wave of hype, he was giving lectures saying large language models suck. And um, he's left Meta. They have they've gone another way with the team, and he's starting uh his own new thing where he's he's very interested in figuring out well, how how could we digitally recreate world models? Um and and and taking that as his approach for at least trying to advance their capabilities. I think he's got a tough road ahead, but he's he's sure thought about this a lot longer than I have.

SPEAKER_02 39:19

You brought up recently in one of your Substacks that what people often forget about education is the socialization aspect. And in in schools, we would talk about relationships. We talk about this a lot at schools, that education often comes after those strong relationships are formed. And teachers know that the real work comes with sitting down with students, going through their processes, iteration, feedback. Um and it feels like the LLMs are in a completely separate world from this, from what from how education really works at the table with the student. And in a way, the just the Silicon Valley nature in which this all came about feels you know, cartoonish at times, so disconnected from the real world. But it really feels disconnected when you're in that classroom with living, breathing kids who are developing and learning at all different rates. Um and that's where I think for us educators, that's that is where we get stuck. We can see value in certain things LLMs do, but when the rubber meets the road on a living, breathing human. Yeah.

SPEAKER_01 40:41

Yes. Well, I mean, look, I I have a short answer, and then I want to run a thing I've been kicking around in my mind. You can tell me if it makes sense to you. This is a we'll beta test something, because I had this observation the other day. I mean, so the short, pithy response to that is right. If you have a really powerful hammer, everything's a nail, right? And that is just very much the attitude. And it's not, frankly, unique to large language models. For literally, since the invention of the computer, and frankly, going back even further than that, there's just been this belief that surely technology can deliver education better than humans can, because humans are messy and imperfect, and so there must be something that's wrong with them. And so the, you know, I like to point out Thomas Edison, 1913, was like, Yep, books are done, scholars will be instructed in the eye. You know, B. F. Skinner in the 1950s creates teaching machines, and they'll just, it's it's fundamentally based on this input-output, and it and it lacks that relational piece. And so here's the the sort of conceptual thing I've been playing around with, and I would love your feedback on it. So I I like going to um many of our fine local breweries here in Austin, and um, you know, most of them, the big ones, have a big outdoor area and a pretty large indoor area. And that makes sense because it gets really hot here in the summer, and sitting outside during July and August can be pretty brutal. And then we sometimes have at least a week or two of cold in the winter where you also don't want to be outside. But on most days, especially this time of year where it's our spring and March and April, we have this lovely weather. And so I was there at one of my favorites just a week ago. It's called St. Elmore Brewery. Perfect day, Saturday or Sunday afternoon. Outside, every single table is jammed. People are sitting next to each other. Like people are asking. I had to ask, is it okay if I said it this? Yeah, of course. Kids are running around. I mean, it is popping, man. But you have to go inside to order the beer. And I walk inside to order the beer, then what would you predict? There's one family sitting at the table inside, and they're doing so because they have a baby who's in a high chair. Okay? So, why am I telling this story? Is it because I like beer in part? But here's the thing. Here's the thing. I was thinking about it. I was like, this actually is exactly what Silicon Valley gets wrong about education. Because why do we choose the outside on a beautiful day? It's I it's like you almost don't even have to ask the question. Of course we would, because that's that's our natural way of being. We like being around each other. And we like being outside. We like being in the world. So much of modern Malays, we might say, is because we force ourselves into offices and places, because we have to, and it's we do have to do that. I'm not saying, you know, schools should be abandoned and everybody should be doing outbound nature courses. That's not what I'm saying. I'm saying that we can be either inside or outside, but under circum certain circumstances, the optimal circumstances, we're always going to choose to be outside. And what I would submit is that is exactly the same thing for learning from one another. It's not that we can't learn from using technology. Surely we can. You know, if you watch enough Khan Academy videos, you'll probably pick up something. But because of our speech, Species, deeply ingrained from literally hundreds of thousands of years, if not millions, we want to learn from one another. This is our superpower. And it is unbelievable that so much power and wealth is being wasted trying to destroy it, trying to erode it. It's un the amount that we are able to do because of our relationships with one another and the knowledge that we learn from one another. And I'm not just talking about academic knowledge, although that's very important, but that relational knowledge of what is school also about? It's about learning how to get along with people you disagree with. It's about learning how to have a conversation that doesn't end in tears. You know, I mean it's so deeper, and it's just like it's just baffling to me that as we see the ravages that technology have brought on sort of some of these profoundly human institutions outside of schooling, and as we see sort of democracy itself under a strain that I never imagined in our lifetime, at this very moment in the education sector, the posture is by and large, well, we've got to do something about AI. And it's like, yeah, we do. We need to keep it out. We need to keep it out.

SPEAKER_02 45:14

Yeah, I uh a guest I had once said that school at this point might be the last place where people are forced to be with uh others they disagree with. To be in a room with someone that you don't share their beliefs. Because once you leave school, and and and the online world has only accelerated this, and the sacophancy of AI even even further so.

SPEAKER_01 45:40

Yeah, exactly right. And it's like note how even that is under strain, the notion that no, I it well, yeah, they need to be schooled, but it needs to be in a micro school, or I need a voucher to send them to the school that I want. It's it's the it's the acceleration of the hyper-individualism that what matters is what is what's good for me. And democracy and frankly, social cohesion is at least partially premised on. I need to care about what's good for you because that too is good for me. And if we have schools that do not reflect that, if we are not doing public education in the true public sense of it, and we have not ever, we have not ever done it perfectly. It's always been imperfect. We have at times made better strides. We, you know, the country almost ripped apart not that long ago over saying actually black kids and need to be able to sit next to white kids. Private schooling in the South erupts in the wake of Brown v. Board of Education, right? And we are now seeing that accelerate. So I I have some hope that the commitment to learning from one another is so deeply ingrained in our species, and the commitment to democratic ideals is at least culturally ingrained over the last several hundred years, that this current moment, if we can get through it and when I say current moment, I mean the moment of American fascism, that there will be a restoration and a redemption and a return to ideas of caring about one another, not just myself, particularly in education, because to your point, you're absolutely right. It's one of the last places where twenty-five, thirty, sometimes unfortunately even more than that, teachers are together and you just get who you get, who's in your community. And and you learn. You learn about differences, you learn about how to be in a relationship, both with the adult who cares about you not because of blood, but because of professional obligation and the calling that they've they've come to.

SPEAKER_02 47:53

Yeah, I I'm so glad you you come you come back to the theme of democracy and civics, because that is such an important part of this. And one of the i part of those ideals is the power of education, but the eroding of trust and education in this country, the eroding of funding. It leads schools to look for silver bullets. It leads schools to look for things like, well, if we just go all in encoding. I mean, I I was at a school once that thought that if you just kept seeing growth mindset all year, all the time, test scores would go up. Like literally, they just were like, just growth mindset. And now you and you've written recently, scathingly appropriately, about partnerships between Google and um districts. Yeah. Why is this happening? Um and what are the potential consequences?

SPEAKER_01 48:43

So my scathing essay that you're referring to was actually about Google partnering with ISTE plus ASCD. Uh I'm just going to call it ISTE because that's a long acronym to get off the tongue, um, wherein they have vowed that they will provide free training, quote unquote, on how to use AI to every single teacher and educator. So one thing that has emerged that was entirely predictable, although even to me, I'm I'm surprised at how not only how big the uptake is, but how uh some of these companies seem to be embracing it, is that one of the primary use cases of AI in the form of large language models, is students using it to cognitively offload, some say. I prefer cognitive automation. OpenAI openly boasts that half of its users are students, and they openly boast about how usage rate skyrockets during school years across the world. I remember watching a presentation where Leah Belski, who I think is public enemy number one for me, put, you know, this thing. Look at what happens in the Philippines. School starts up and all of a sudden they start using it. And so we know why. So the battle is on, and what's interesting is we now know that these things usually shake out in a winner-take-all um scenario, right? Like Google wins the battle for search of the internet, right? Uh, Facebook wins the battle for thing where you connect with people in your life, and then after a few years, you get ads like jammed down your throat. So all of these tech companies have rightly assessed that someone's probably gonna win here, and education is one of the primary vectors of usage. I see education, I should put that in quotes. Students, children are one of the primary users of our tools, and so we better capture this market, quote unquote. And it's I don't even think it's motivated primarily financially, because if you think you're gonna make a lot of money in the quote unquote education market, um you're probably in for a disappointment. But ideologically, ideologically, that's what they're after. OpenAI says to teachers, hey, we'll give you our premium version until June 2027. So it's like, we'll get you hooked. It's free and just sign up and you can use it for as long as you want, right? Google counters. Well, uh free online courses, which will be, I don't know if I can swear, they will be very bad. They will not be good, they will not be educational, they will be a waste of time. So I'm not too worried about it. I mean, some teachers will surely sit through it and click through it, but um, since Google doesn't understand pedagogy, and since Google doesn't understand how humans really think and learn, and has shown an ongoing commitment to eroding student agency and creative thinking, I'm not too worried about it getting uptake. But how embarrassing, how embarrassing for the sector that it's not resisting but instead embracing. How embarrassing that the CEO of ISTE is saying, yes, let me help you do that.

SPEAKER_02 51:36

Well, he's he's you you have a quote from him in the piece. It's basically like, teachers are telling us we're not prepared, we need training. Well, it's like, well, you're telling teachers they need this or they're gonna be behind, and then feedback loop comes back to you.

SPEAKER_01 51:50

That's right. I mean, it's exactly right because it's sort of like teachers are telling you that because they're being told they need to do something, but it's like maybe what they're telling you and telling you that is this tool doesn't do what you think it does. You know, that this is not some, you know, magical unlocking of cognitive capability that you're promising it will be. You know, it's not that it can't be used for certain things. Again, I go back to what I said much earlier in the podcast. Absolute denialism of some utility is, you know, not great. I um you I don't know if you read Stephen Fitzpatrick, who writes about AI in education. I think he does so extraordinarily thoughtfully. He's more sympathetic, and I would call him an AI realist because he sees the usage that his students have and has sort of said, you know, uh, well, that's just going to be the reality. And he's written about how he has used it for his own sort of testing of his lessons and in and refinement. And, you know, if you're going to make the case for it being something that's useful to teachers, okay. I would still have a lot of things on plan to push on with Stephen and and in the in the broader public. But man, cognitive effort is hard and students will choose to avoid it often. That's the that's the reason we set up these institutions and have adults called teachers sort of pushing them in the same way that we have trainers at the gym who push us, because it's hard. It's hard to motivate to do that. And and there's just very little question in my mind that even for the small fraction of students who can actually perhaps learn something through interacting with AI, that's not going to be the students that we should be thinking most about in terms of the challenge their cognitive development.

SPEAKER_02 53:32

So actually, I want to talk about utility and we can sort of end here. But in sort of more practical terms, how teachers should be thinking about this administrators. You know, you have we, you know, we have these extreme, these alpha schools that are like, well, let's educate kids entirely with AI, and we're already seeing what a joke that is. And but then on the other end, there's schools that are maybe saying we're not even going to allow this in our building, which also feels, as you say, this denialism. Um, it missing.

SPEAKER_01 54:00

I don't know that there's many schools saying that, but if there are, point me, point me their way.

SPEAKER_02 54:04

Okay.

SPEAKER_01 54:05

I'm very sympathetic to that position. You know, there's a wonderful teacher, uh, Sinee Bond, who's here in Texas, and she became kind of teacher famous a few years ago because she's like, Yeah, we're not doing AI in my classroom. And she's written for Edutopia, both about that initial decision, but what she's learned since. And I I really recommend to educators that they find that. Google Edutopia, Sinee Bond, C-H-A-N-A-E, I believe. Um, because it can be done. It can be done in the context of the classroom. I I have I have given up, people would ask me, you know, make the best case for its usage. I'm not going to do that. There they can make a case. Like I'm here to oppose it. I'm here to oppose it in what it's doing in the education space, while not judging if people do use it. I just there is a there is a bit of a complexity and nuance to that, because I'm not saying you're a horrible person if you've ever used it, teacher. I'm not saying that. But I'm saying that like as a as a sector, educators, human educators should see this for what it is. It is a debasement of the calling that has brought you to what you're doing. You should be offended by its push into your life as an educator.

SPEAKER_02 55:17

A few episodes ago, I interviewed Michael Wagner at Drexel University, and he's sort of writing about how AI, in some ways, maybe ironically, is ushering in, could usher in, um, a rebirth or a growth in authentic assessment, authentic learning, where you okay, you don't you want to resist AI, so you need to go back to things like the oral exam, you know, the in-class writing task. And it is kind of funny to me now to see college professors on like Instagram like discovering that they can do in-class work. It's like, wow, if I just have them do it in-class, they can't use AI. This is incredible. But I mean, so I guess like if you're in if you're in front of a uh a school, you're talking to a group of school administrators, what what do you say, what would you say to them about how they should be thinking about AI in this moment?

SPEAKER_01 56:09

Well, I think you've left, we can leave on this point because I think it's a great one to leave on. So the best thing that may come out of it is exactly what you've just described. And I don't I don't push it too hard because I think it's quite offensive to say, oh, AI is this tool that's going to save teachers all this time and then turn right around and say, oh, actually what we really mean is you have to redo everything that you've ever done because now you can't trust that your students' work is actually their own work and a product of their own thinking. It's more work, it's creating more work. Like Sinee Bond talks about it, and she's hardly alone. It's like, if I want to authentically assess, meaning I want to actually know that these thoughts are your own, that is now much more work for teachers to do. But that said, there's no doubt in any walk of life, you sort of get stuck in patterns and practices. And certainly through both education policy and I think a number of other things that have happened, we have drifted too far away from what you've described here as authentic education and the human relational side and getting away from the technocratic approach to education that has dominated for at least the last several decades here in the United States and frankly uh in many other places. That that gets back to my my hope for a restoration of what education can and should be. So we'll hope. We have a great hope.

SPEAKER_02 57:30

Ben, thanks so much. Uh your substat cognitive residence is awesome. Like for educators, for everyone who's just thinking about the place of AI in our world. It uh great stuff. And uh, I realize, you know, in this hour we've just probably scratched the surface of the things that you think about on the daily, but I appreciate you taking the time to chat.

SPEAKER_01 57:49

I really appreciate you um inviting me. And I think you have to teach today, right? You are you uh this is we do this literally before and the engine away. We just did an Allen podcast, and I'm gonna go, you know, take a take a nap or something from all the mental exertion, and you're gonna have to kick it into even higher gear. So I just want to say thank you for this, but thank you for being a teacher. Thank you for being thoughtful. And um I hope uh our conversation in one way, shape, or form continues.

SPEAKER_02 58:18

Huge thanks to Ben Riley for coming on the show. I really appreciate his independent thinking about artificial intelligence. And I think that's actually one of the most difficult aspects when learning about and thinking about large language models is that there's a lot of industry chatter and talking points that's sort of seeped into the way we think about the technology. Um, you know, I'm someone that I use these chatbots to help me as a collaborator. But when it comes to how schools should be using it, you know, I'll just go back to this conference, this one-day conference I was at that I mentioned at the top of the show, I was so struck by the keynote speaker who had some really compelling arguments for how to use AI. But I was struck by how many of her slides, her graphics, her quotes came from industry like OpenAI or Anthropic or Microsoft or a viral essay written by somebody who literally is in the AI business. And so voices, I think, like like Michael Wagner, like Eric Hudson, like today's guest Ben Riley, I think are vital because these are not people that have a vested interest in seeing AI succeed. They are not financially tied to the success um of these language models. And I think it's really important that we keep seeking out voices like this and not succumb to uh industry claims of safety, of potential, of how we're gonna be replaced if we don't learn how to use these models, of how we have to get the paid model. That was it, that was actually one of the things that the speaker said at that conference. You know, you really have to have the paid model. I was thinking, uh, says who? Sam Hallman. So uh thanks again to Ben Riley. Uh would love to hear from you uh how you're using this, what you what you're thinking about this. I mean, it's clearly there's no there's no head in the sand on this one. We need to take it on and think about the what AI is doing or not doing for our students, our fingers for us. Um, so uh thanks for checking out this episode. Hope you've been able to hear all three on AI. And uh check us out next time. Appreciate you.