The Renegade Coder – Code First. Ask Questions Later.

As generative AI is thrust upon as at an alarming rate, I've found that even subtle usage of these "tools" could have profound impact on the way we think. As a result, today's public service announcement is a plea to get you to stop using generative AI for summaries. The Summary Takeover I think the first time I really noticed the infiltration of AI summaries was during search. Google was toying with them a while back, and while people clowned on it, Google didn't really back down. Now, no matter what you search, you'll get an AI summary of a supposed answer. Of course, Google isn't the only offender here. Even the more "ethical" search engines like DuckDuckGo give generative summaries now, which is really annoying. The entire purpose of a search engine is to make it possible to sift through the slop, and now all the major search engines center it. To me, summarizing search results is stupid for more than the following reasons: 1) the response could be wrong, 2) the response is inherently reductive, and 3) the response takes away from the enjoyment of reading what people have to say. As a result, I wasn't really bought into the idea at all. In fact, I have it turned off in DuckDuckGo. Then, Apple introduced summaries for texting, and I found that even more stupid. Texting is already a reductive technology, so reducing texts seemed completely insane. Not to mention that Apple also shipped AI replies, so you can easily respond to a summary with AI generated text that would then be summarized. What a hilariously stupid "innovation." The Problem With Summaries Despite what seems to be a widespread push for AI summaries and its subsequent adoption, there are some major problems with relying on AI summaries. Rather than just listing them off, I want to challenge you to reflect for a moment on a few questions. How Do You Trust the Summary to Be Unbiased? Frankly, I am not interested in anything being unbiased. The reality of being a living, breathing human is that we carry biases—even scientists. With that said, a machine cannot be biased because it does not have a worldview, a set of beliefs and values, or lived experience. It has no decision making or reasoning capabilities that would give it the ability to construct a biased narrative. It cannot be held accountable, so it has no fear of repercussions. Yet, machines clearly display biases because they're constructed by humans. Therefore, the question becomes: what informs the biases of generative AI? To start, we do not know what data these models are trained on, so we have no idea what biases they might have. These biases could be unintentional as a result of mindlessly feeding the model all of the world's data. In contrast, these biases could be manufactured by feeding the model curated data. Even if the model is robust, it's possible that the prompts are manipulated on their way to the model. Perhaps certain keywords are added or subtle instructions are injected. As the client, you have no way of knowing what happens to your prompt once its entered. Even if the prompt goes in clean, responses could be pruned on the backend as they're deemed undesirable. Conspiracy theories aside, even if we were able to determine a model's biases, there is no guarantee that they would be static or even consistent. This makes it really difficult to interpret any summaries provided by generative AI—something that is often a given with someone we know. For example, suppose you ask me who the greatest current hockey player is. I'm a Penguins fan, so it would be unsurprising if I said "Sidney Crosby." However, if I said "Connor McDavid," you might be surprised and take my opinion more seriously. Ultimately, there is no way for a generative AI summary to be unbiased, and their biases cannot be pinned down. Without a consistent understanding of a model's biases, I would argue that AI summaries cannot be reasonably interpreted and should not be trusted. How Do You Trust the Summary to Be Accurate? Right now, we seem to assume that these glorified chat bots are capable of producing accurate summaries, but how can we be so certain? After all, it's already clear that generative AI is capable of hallucination (or outright lying) while presenting information as definitive fact. In my opinion, the idea that a summary can or even should be trusted to be accurate seems odd. If you task five friends with summarizing a book for you, you'll get five different summaries. While the overall message of the summaries might be the same, each of them will latch onto different ideas, themes, and plot points. Intuitively, you'd expect this, yet you'd trust a single summary from generative AI anyway. To stretch this example a bit further, imagine one of your friends hadn't read the book at all. Maybe they forgot about it until a few hours before book club, so they skimmed the book and crafted a summary. You would probably expect their summary to be comically bad. That friend is how generative AI seemingly behaves. After all, Apple Intelligence once incorrectly summarized a news article in four words: "Luigi Mangione shoots himself." Not to mention that over half of all generative AI responses to questions about BBC news stories had significant issues. Needless to say, I wouldn't personally trust any summary to be accurate, but I definitely wouldn't trust generative AI to give anything resembling accuracy. How Can You Trust the Summary to Give a Good Interpretation? For whatever reason, we seem to think that communication is purely mechanical. It absolutely is not. After all, language is not strictly logical. Every time we're told something, we have to make an interpretation based on what we know and believe. In a conversation, it's not just the words that we must interpret; it's the tone of voice, the facial expressions, and the body language. When we inevitably moved to texting, a lot of the back-and-forth in a conversation was lost. As a result, we swapped out all of the non-verbal communication with emojis, images, and gifs. As you can imagine, despite our best attempts, text is still an extremely reductive form of communication. As a result, every message that's lost in translation is a fight that could have been avoided. In fact, I am willing to bet that half of a therapist's job these days is helping clients make sense of text messages (a job which is already being outsourced to generative AI). So, imagine my surprise when we swapped out this reductive form of communication with an even more reductive form of communication: generative AI text summaries. Obviously, a summary by definition is reductive, but I think we're underselling what's lost here. Generative AI does not know the sender, so it cannot make a meaningful interpretation of a message. An area where I find this particularly funny is Twitter, where you'll see people comment under a tweet something like, "@grok what does this mean?" If you don't know, how the hell is a chat bot supposed to know? It's just going to spit out some milquetoast "interpretation" of the tweet, maybe using the user's post history as context. While funny, I do worry about this sort of thing. What do people even do with grok responses? Does it tell them what to think? Do they use it to uncritically form their opinions? I am so confused. Should I ask grok? All of this is to say that, like communication broadly, summarization isn't a purely mechanical task. In an effort to boil text down to its essence, you have to determine what is most salient, which is a task of interpretation. While generative AI might have access to all of human knowledge, it isn't capable of interpretation (only something that looks like interpretation). Therefore, it shouldn't be trusted to produce summaries, especially in interpersonal communication. One Big Problem In this article, we looked at a variety of problems with AI summaries. We talked about how they're biased in ways that we cannot know. We talked about how they're inaccurate to the point of promoting misinformation and disinformation. And, we talked about how they can't give a good interpretation of the underlying data. To me, all of these are wonderful reasons to stop relying on generative summaries, but there is one big reason that is reiterated throughout this series. Each time you rely on generative AI to summarize something for you, you become ever so slightly less capable of doing it yourself. You become less capable not only because you lose the skill to summarize but because you become more tempted by it in the future; you become more dependent and more lazy. This combination of dependency and laziness is a death spiral for critical thinking. Why do the work of reading and synthesizing a handful of sources when you can just ask chat? Why discuss an idea with one of your friends when you can just ask chat? Why read a book or watch a movie when you can just ask chat? Why form an opinion at all when you can just ask chat? I don't think I'm fear mongering when I say that our ability to think critically is tied to our ability to summarize thoughts and ideas. To put together a good summary, we have to be able to actively read, watch, listen, and interpret multiple sources of information. We have to be able to determine what ideas are salient and what can be ignored. Ultimately, we have to be able to sort through the noise. If you think you're somehow above the ability to summarize, there's an entire mode of research dedicated to the craft called qualitative research. If you think an LLM is capable of quantifying the human condition, you're completely cooked. All of that is to say, tech bros will call you a luddite for this take, but they will let generative AI form theirs. There are so many ideas I have about the problem with outsourcing summaries to bots that many of them didn't make the cut. That said, I like to leave this area for some of my musings, so I'll share more down here as they come up. For example, one that came to mind for me is the importance of the audience when you're constructing a summary. Who is the summary for? Do any of these tools actually consider YOU when they "write" the summary? Another that comes to mind is this idea of the profit incentive. Do you think the profit incentive will necessarily align with the good of humanity? In other words, do you think a summary will always provide the best possible result or will it begin to behave in more nefarious ways for profit extraction? Not to beat a dead horse but people are already so addicted to offloading their brain that they've started dating these bots. I'm sure there will be fascinating ways of making summaries worse (e.g., sponsored summaries, paywalled summaries, propaganda summaries, etc.). Anyway, thanks again for taking the time to read this post. Overall, I've really been enjoying this series. Being a hater feels pretty good. Jokes aside, if you liked this, there is plenty more where that came from: Generative AI Has a Short Shelf Life No, Generative AI Is Not Just Another Innovation Generative AI Makes It Feel Bad to Be an Educator Also, if you have the means, I'd appreciate it if you took a moment to check out my list of ways to grow the site. Otherwise, take care!

Welcome to The Renegade Coder, a site for people interested in Python programming and college teaching! Join me on Patreon for even more content.

Latest Articles