Eat Brains. Boost Profits.

͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌   ͏ ‌  ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­ ­

On the logic of embedded systems

Data and attention are some of the world’s most valuable commodities, and our personal tech products are wildly efficient extraction tools.


AI companies are the latest horde of prospectors in this cognitive gold rush, and their new frontier is the classroom.

Screengrab from the Miami-Dade County schools website

Disclosure! I use LLMs all the time.


I’ve made show tunes about Strategic Lawsuits Against Public Participation, hairless podcasting cats who sound like late Bob Dylan, and collaborated with ChatGPT on hundreds of poems.


Just this Saturday, my son was using ChatGPT to discover new Balatro strategies, my daughters were using Gemini for song-writing prompts, and I was working on a “Press Release Generator” out in the garage.


So, you know, I really like Large Language Models.

And yet!


The recent headlines about Big Tech’s plans for public schools and Trump’s Executive Order “Advancing AI Education for America’s Youth” have pushed me to think more carefully about the costs and benefits of AI integration for students and communities.


Should we really let the unrestrained interests of private companies have even more influence over public education under the Trump administration’s watch?

After all, these tools aren’t just helping students learn. They’re collecting data, shaping behavior, and optimizing for engagement (aka dependency).


You don’t even have to look as far as Myanmar to see how the makers of these tools don’t have a great track record for collective uplift (but, you should look as far as Myanmar anyway, because it’s horrifying). Closer to home, during the pandemic, we saw the private Big Tech savior story play out in public education, and what happened?


Students spent more time on screens than ever before, learning gaps widened, and class engagement dropped off a cliff.


Oh and Big Tech made billions of dollars, because they weren’t just giving helpful tools to kids in need, they were embedding systems of extraction.


As Jessica Grose wrote recently in the New York Times, the pandemic should have been a teachable moment for all of us that education is not just about delivering information but about building relationships, maintaining structure, and being present.


Brookings Institute fellow Rebecca Winthrop has argued similarly that AI probably will help with rote tasks like math drills or grammar correctionin the same way spellcheck raises the floor for coherence but AI shouldn’t replace education’s more human, and arguably more important, elements like socialization, empathy, and resilience in the face of failure.


Alas, parents, administrators, teachers, and students are so desperate for any relief from budget shortfalls and time-crunches, it’s not surprising there’s always some induced magical thinking at play when Big Tech comes calling (“Yes, test scores are down. WHAT IF WE HAD AN APP?!”).


Any parent who has been forced to sign up for dozens of platforms just to figure out when assignments might be due or why the bus hasn’t shown up on time can imagine “full AI integration” being an implementation disaster.


For all the promise of AI as an educational utility, we need some way to ensure a minimum of accountability for companies that want to get further involved. Simply a promise that they’ll “respect student privacy” isn’t going to cut it.

An educational slide for Miami-Dade County Public Schools teachers. Credit: NYT

AI is already in classrooms, whether we admit it or not. The real danger going forward isn’t that it won’t actually work but that it will work REALLY WELL — for private interests, not the public good.


I’ve seen in my kids’ school district how difficult it is for students to learn when they’re constantly being pulled toward dopamine-optimized digital distractions, and I’m confident AI will only make this worse.


(SMASHCUT TO: Spring semester 2028 when rising fourth-grader Maverick Cash Deforester is so addicted to chatting with his fully bespoke Spanish tutor bot, made in partnership between DuoLingo and Grok, that he no longer has any human friends)


Expecting K-12 teachers to compete with these cognitive mercenaries is like asking them to teach social studies through a network of PS5s while #Looksmaxing influencers hand out free Takis.

Really hard, is what I’m trying to say.


And the latest web traffic research shows LLMs are not only siphoning traffic from search engines but also from old student stalwarts like Wikipedia and SparkNotes, which says the shift to LLMs isn’t just a fad.


We’re clearly undergoing a fundamental shift in how people use the internet, and how the internet uses people.So fully banning AI in schools would only widen the digital divide — students with AI access at home would learn to use it, while those without would fall further behind. But we can’t just give away privacy, autonomy, and focus for the promise of universal uplift.

Queen Victoria Giving the Bible to an African Chief (The Secret of England’s Greatness) by Thomas Jones Barker (via WikiCommons)

In her new book Empire of AI, Karen Hao draws a compelling parallel between the logic of today’s AI companies and the logic of colonial empires. Colonial powers once claimed land, labor, and culture under the banner of “civilizing” the world. Today, AI companies extract data, underpay human workers, and claim ownership of online content, all while promoting a narrative of innovation and education.


While the current number-goes-up market caps of Microsoft, Google, and Apple are proof enough that this model of exploitation still works just fine (for the tech companies), it still seems marginal at best for the rest of us.

If schools don’t teach students how to use AI with clarity and intention, they will only be shaped by the technology, rather than shaping it themselves. We need to confront what AI is designed to do, and reimagine how it might serve students, not just shareholder value.


There is an easy first step for this: require any AI company operating in public education to be a B Corporation, a legal structure that requires businesses to consider social good alongside shareholder return.


This would create enforceable obligations: companies could no longer maximize profit by optimizing data extraction and designing tools based on that data to hold attention at the expense of student well-being.


This would help create a structure where students would get more than just access to technology. The focus could then be not just how to use AI, but how AI works for and against them — what data it collects, what assumptions it makes, what biases it replicates.

They won’t just get “digital literacy” and “job training.” They’ll get an education.



Fine! Okay! Enough useless vibrating airflow, Mr. Instrumental! What should we actually do?


Good question, imagined cyborg reader! Thanks for asking!

I think a start would be to:

  • Hold companies accountable. Mandate B Corp status—or an equivalent public-interest structure—for any tech vendor in schools.


  • Teach AI as a subject, not a shortcut. Build curricula that help students understand the design, impact, and limitations of the tools they use.


  • Protect focus. For those few hours kids are in classrooms, limit AI use where it competes with instructor attention. Don’t let the promise of “customized learning” be an excuse to foster even more device dependency.


Solved? Sadly, no. But as AI becomes more fully integrated into education -- both in America and in many places around the world -- we should prepare students to use AI with intention, or we’ll just be preparing them to be used by it.


We are embarking on a “Summer of ABBA” in my household —> largely because some of us are obsesses with Mamma Mia! —> which features Colin Firth —> who also smolders delightfully in the 1995 BBC production of Pride & Prejudice —> which features a Mrs. Bennet played by Alison Steadman —> the ex-wife of Mike Leigh, director of the greatest movie of all time, Topsy-Turvy —> the director’s commentary of which includes this tidbit:

”Gilbert and Sullivan communicated mostly by letter…and, of course, there were 16, 18 mail deliveries a day in the London metropolitan area”

—> a fact which complicates the idea at least a little bit that “real-time” comms channels like email, Slack, Teams, or whatever have ruined a previously pristine mode of correspondence.

Next
Next

You Don’t Make Friends with Factoids