Are we cooked, chat?

There is a specific kind of dread that comes with reviewing code you do not fully understand, and I know this dread well.

I stopped struggling at some point in my third year of Computer Science. The coursework had not become easier, nor had I suddenly become a better programmer. Instead, the tools had become good enough that I could outsource the part where I got stuck. Whenever I encountered an error, I would paste it into a chatbot and receive a solution in seconds. The assignment got done, and I moved on. It felt like efficiency at the time.

However, I am less sure about it now.

Artificial Intelligence (AI) researchers use the term “hallucination” to describe a model’s output that sounds right but is wrong—fluent, confident, yet subtly broken. The phrase stuck with me because it describes not just the model, but also the person using it. When a person accepts the output without asking why it works, they hallucinate mastery. Understanding exists only on the surface, concealed beneath the performance of artificial competence.

Recalling the past, the coursework that felt most punishing in my first year was not just about learning isolated technical tricks. Tracing the logic of a crashing program, understanding why memory was being claimed and never released, following an error backward through layers of abstraction until I found the assumption that broke everything. This taught me how to hold a system in my head and reason through it under pressure—a mental discipline that stays with me long after I forget the specifics of a language itself.

And that is precisely what AI cannot do reliably. A model can produce code that looks right, but it cannot always tell you when it is wrong, why it failed, or what the downstream consequences are. Recognizing the gap between producing code and evaluating its failures are not things anyone can learn by merely accepting outputs. It is learning by producing failures—often enough and slowly enough—to develop an intuition for where things go wrong.

This phenomenon exists beyond Computer Science. It is concerning how our generation has absorbed AI tools into education: not that we use them, but that we have stopped noticing the difference between using a tool and learning from one. For instance, a calculator does not teach arithmetic, a grammar-checker does not teach writing, and a model that generates a working function from a prompt does not teach how systems fail. Understanding that failure, however, is exactly what a Computer Science degree is designed to teach.

The issue with these advancing technologies is not their existence, but how we choose to rely on them. There is a difference between using AI to move faster through work that a user understands and using it to avoid the work that makes understanding possible. The latter borrows against a debt that eventually comes due: it usually hits the moment when we move past predictable class assignments into larger, interconnected projects, where things break in ways a simple prompt cannot diagnose.

Computer Science education is not simply about producing code, but also learning to understand the systems that fail in complicated ways. A version of this degree that skips the struggle may still produce graduates who are adept at generating output quickly. However, it also risks producing graduates who are too poorly equipped to question it. In a field where the most important professional skill is knowing when not to trust the system in front of you—be it code, a model, or a legacy process someone built before you arrived—that is a significant gap.

Our generation is the first to go through this without a clear consensus on where the line should be drawn. Most of us are drawing it poorly, or not at all. Looking back, I know I was no exception, especially as I still use these tools. What becomes clear, after learning some things the slow way and skipping others entirely, is that the slow way left something behind: judgment and memory. The shortcuts left nothing that you can carry forward.