Cursed ChatGPTists: The Cult of AI and the Risks of Unchecked Automation

There is a new type of tech enthusiast circulating through social media, WhatsApp groups, and digital entrepreneurship events. They have never written a line of code in their life, have never delved into system architecture, and most likely don’t even know what a stack trace is — but that does not matter. After all, they have ChatGPT. They have AI. And, in their view, that is enough for everything.

Welcome to the era of ChatGPTists.

---

The Cliché That Is Real: We Live in Radical Immediacy

Yes, it starts with a cliché. But clichés exist because they carry truth. We live in a time of fragmented attention, shaped by 15-second videos and content designed to deliver maximum impact in the shortest possible time. In this environment of chronic immediacy, artificial intelligence has found extremely fertile ground to grow — and, more importantly, to be misunderstood.

Every week a new AI tool emerges that promises to "revolutionize" something. Every week a tech influencer asserts that a certain profession is "doomed." And every week, the army of those who firmly believe in these promises without questioning anything grows.

---

The Phenomenon of the Accidental Developer

One of the most curious — and concerning — phenomena emerging from this context is the accidental developer. These are people without any prior training or inclination towards software development who, by using generative AI tools, manage to produce minimally functional applications in just a few hours.

The process is almost always the same:

1. The person describes a simple problem to ChatGPT or a similar tool
2. The AI generates seemingly functional code
3. The result "works" in a controlled environment
4. The person concludes they have found a great business opportunity
5. The product is launched to the market without any serious technical review

The central problem in this equation is brutal in its simplicity: the same ease that allowed for the creation of that software is available to absolutely everyone. If anyone can generate a customized solution for their own needs in a matter of hours, what is, in fact, the commercial value of this product?

The answer, in most cases, is: very little.

---

The Illusion of Sophistication

Artificial intelligence is extraordinarily efficient at one specific thing: conveying a sense of productivity and sophistication. AI-generated code appears clean, organized, complete. An analysis produced by AI seems deep and well-founded. A business plan created with AI assistance looks robust and detailed.

Seems. That is the key word.

Appearance and real value are completely distinct concepts, and confusing them has serious consequences. Developing commercially viable software involves much more than generating code that "works on my machine." It requires:

• Robust security against attacks and data leaks
• Continuous maintenance and bug fixing in production
• Reliability and availability (the famous "uptime")
• Integration with legacy systems and third-party APIs
• Real technical support for end users
• Automated and manual testing in multiple scenarios
• Legal and regulatory compliance (LGPD, GDPR, sector regulations)
• Capacity for evolution over time without breaking what already works

It is these invisible layers — that AI rarely mentions spontaneously and that the ChatGPTist often ignores — which represent the bulk of the real work in professional software development. These are what differentiate a sustainable project from a digital house of cards.

---

The Concrete Risks of a Market Flooded with Well-Intentioned Amateurs

In the coming years, we will witness an explosion of digital products created with the help of AI. This is inevitable — and, in many ways, even positive. The democratization of creation tools holds real and undeniable value. The problem arises when this democratization occurs completely detached from the necessary technical knowledge.

The risks are concrete and already materializing:

• Data leaks in applications built without any concern for security
• Critical security failures in systems handling sensitive user information
• Digital scams structured around seemingly legitimate but technically fragile platforms
• Loss of client data due to a lack of adequate backups
• Systems that stop working without the "developer" knowing how to fix the issue

More seriously: many people are delegating critical business and even personal decisions to tools they do not fully understand. They create a false sense of technical competence that can be extremely dangerous — both for themselves and for the users of their products.

---

It’s Not About Defending Technical Corporatism

We need to be honest here to avoid the overly simplistic argument: this criticism is not a corporatist defense of developers afraid of losing their jobs. It is not about building artificial walls around knowledge to preserve professional privileges.

The issue is more fundamental than that. It is about responsibility. When you build something that other people will use, when you handle third-party data, when you market a solution as if it is reliable, you assume responsibilities that do not disappear just because AI helped you write the code.

AI is an extraordinarily powerful tool. In the hands of qualified professionals, it exponentially amplifies productivity and opens up previously unimaginable possibilities. The problem is not the tool — the problem is the arrogance of those who use it without understanding its limitations.

---

What to Do with All This?

The answer is not simple, but some points are clear:

• Learning to use AI is valuable; learning to question AI's outputs is essential
• The democratization of development is positive when accompanied by minimal technical education
• Regulation and accountability must keep pace with the speed of adoption of these tools
• Consumers need to be more demanding regarding the quality and security of the products they use
• The market, sooner or later, will punish projects built on technically fragile foundations

Artificial intelligence is here to stay, and its transformative impact is real. But genuine transformation requires more than well-written prompts and visually impressive results. It demands critical thinking, contextual knowledge, and, above all, intellectual humility to recognize what you do not know.

The most fervent ChatGPTists will learn this lesson. The question is only how much damage will be done before that.

---

This article was inspired by reflections originally published on Medium by Afegão Médio.