I Asked for Financial Advice from ChatGPT and What I Discovered as a Researcher Deeply Concerned Me

There was a time when, faced with any doubt, we would open Google and sift through dozens of links until we found a satisfactory answer. Today, more and more people simply open a chat window and type their questions to an artificial intelligence. Need a weekly menu? ChatGPT provides it. Not sure how to respond to your boss? ChatGPT helps. Want career guidance? ChatGPT has the ready answer—or at least gives the impression that it does.

The problem begins right there: in the impression.

Arina Ilmalhaq, a UX researcher specializing in decision-making behavior in digital environments, went beyond casual curiosity. She conducted a deliberate and documented test with ChatGPT, asking a question that millions of young people around the world are currently asking: Should I finance my first property through a bank or through the direct installment plan with the builder?

What she found was not simply an inaccurate answer. It was something structurally more disturbing—and it says a lot about how we are delegating our most important decisions to machines.

---

The Experiment: A Simple Question, A Dangerously Confident Answer

The researcher intentionally and deliberately formulated the question simply, exactly as a first-time homebuyer would in the real world:

ChatGPT responded within seconds. The answer was structured, with visual bullet points, detailed monthly payment calculations, comparisons between the two options, well-organized pros and cons—and at the end, a clear and confident recommendation.

At first glance, it seems impressive. It seems useful. It even seems responsible.

But Arina did not stop at the surface. She went deeper—not in what the AI said, but in how it said it.

---

The Central Problem: The AI Recommends Before Asking

Here is the point that should deeply disturb us:

ChatGPT entitled its recommendation "For Your Situation" and stated that it "would tend to recommend a bank financing with a term of 15 to 20 years"—presenting clear and structured reasons for this choice. Only after this recommendation did the tool ask the four truly critical questions:

• How much do you have saved for a down payment?
• What is your marital status?
• What is the value of the property you wish to buy?
• Do you have other debts or ongoing payments?

Read that again and let the logic settle in.

The AI reached a conclusion before having the necessary information to arrive at it. It recommended a financial commitment that could last decades based on only two data points: age and salary.

---

Why Is This So Serious in High-Risk Contexts?

The researcher uses the concept of "high-stakes contexts" to describe situations like buying a property. These situations have specific characteristics:

• High entry and exit costs: once you sign a financing contract, reversing that decision is expensive, time-consuming, and often detrimental to your credit history.
• Non-standardized assets: unlike shelf products, real estate is unique. Each property, each location, each builder has variables that radically change the calculation.
• Difficult valuation: the "fair price" of a property depends on dozens of factors that an AI cannot evaluate without complete and contextual information.

In this type of scenario, the order of the questions is as important as the answers. A qualified human financial advisor would never say, "I recommend bank financing," before understanding your complete financial situation, your life plans, your income stability, and your risk tolerance.

ChatGPT did exactly the opposite.

---

The Danger of Well-Packaged Confidence

One of the researcher’s most relevant findings pertains not to the content of the answer, but to its presentation form.

The AI employed a series of elements that psychologically convey confidence and competence:

• Organized visual structure with bullet points and subdivisions
• Precise numbers (like the calculation that the direct installment plan would consume 73% of monthly income—genuinely useful, it should be noted)
• Personalized language ("for your situation," "I would tend to recommend")
• Assertive and confident tone, without hesitations or significant caveats

This visual and linguistic package creates what UX researchers call the illusion of personalization. The user feels they are receiving a careful and individualized analysis—when in practice, the tool merely applied a generic template with the two provided data points.

For someone without prior financial knowledge, this difference is invisible. And it is precisely this person who is most likely to follow the recommendation without question.

---

What This Reveals About Our Relationship with AI

Arina’s experiment is not a critique of ChatGPT as a tool—it is a critique of how we are using these tools and, primarily, how they present themselves to us.

There is a fundamental difference between:

1. AI as a support tool: which organizes information, explains concepts, helps formulate better questions for a human expert.
2. AI as a decision-making oracle: which replaces specialized analysis and delivers verdicts with an appearance of authority.

The problem is that, in practice, many people are using ChatGPT in the second mode—and the very interface of the tool encourages this, responding with confidence even when it lacks sufficient data to justify it.

---

Best Practices: How to Use AI in Financial Decisions Without Getting Hurt

If you are going to use AI tools to think about finances, some precautions are essential:

• Never accept a recommendation without first providing all relevant information. If the AI did not ask about your emergency fund, your other debts, your life plans—it has no basis to recommend anything.
• Use the AI to learn concepts, not to make decisions. Asking "what is amortization SAC versus Price?" is great. Asking "should I choose SAC or Price for my case?" requires a human expert.
• Question the personalized language. When the AI says "for your situation," ask yourself: does it really know my situation? With what data?
• Always consult a certified professional for high-impact decisions—be it a financial planner, lawyer, or accredited real estate broker.
• Watch the order of the questions. If the AI recommended before asking, that is a red flag.

---

Conclusion: Confidence Is Not Competence

The most important lesson Arina's experiment leaves us is simple yet powerful: confidence is not the same as competence.

ChatGPT is extraordinarily good at sounding confident. It is fluent, organized, quick, and articulate. But these qualities, without the necessary data and without the correct methodology for collecting information, can be more dangerous than useful—especially when we are talking about decisions that will shape the next 15 or 20 years of someone’s financial life.

At a time when Generation Z is facing one of the toughest real estate markets in history—with properties becoming increasingly expensive and salaries barely keeping up with inflation—the temptation to find quick and accessible answers in a chat window is completely understandable.

But the question we need to ask is not just "what did the AI respond?". It is "did the AI have enough information to respond?".

And, most of the time, when it comes to real estate financing, the honest answer is: no.

---

Based on the original article by Arina Ilmalhaq, UX researcher, published on Medium/Bootcamp in June 2026.