Google Search vs. ChatGPT
Why you shouldn’t trust a Large Language Model (LLM)

Published: 23/07/2024
10 minutes read

Google Search vs. ChatGPT

Why you shouldn’t trust a Large Language Model (LLM)

Published: 23/07/2024
10 minutes read

“A computer can never be held accountable” is a statement attributed to an IBM meeting slide that offers a perspective on what artificial intelligence really is. It’s a fancy name for a tool that analyzes human behavior and uses detailed mimicry to respond to us. Is it really intelligent, or is it just using the data it gathers about us to give us the impression it is?

And since AI today mimics our language, it is called a Large Language Model (or LLM). But can it be held accountable for our decisions, or at least the answers it gives us?

The artificial intelligence we have today is heavily based on machine learning, a jargon from the 2010s that refers to using statistics to compute probability.

Before machine learning, there was big data – extracting and analyzing information from huge volumes of data and drawing certain assumptions from it.

So, what we have today in the form of ChatGPT in the form of a large language model is a combination of machine learning and big data meant to talk with humans. Of course, this is a watered-down version of how things are, but it essentially involves training a computer on a lot of data to respond to a human prompt. We don’t need to delve into the Turing Test here since it has been proven that ChatGPT can behave like humans can indeed pass it.

The predicament of a GPT

Well, it’s all nice and exciting to collate big data in digestible, meaningful sentences, and Large language models (LLMs) have become adept at that. However, we’re in a pickle when people use AI to make decisions or we take what it says at face value. Most of the time, the produced answer to our prompt is true, but how can we know when it isn’t the case?

Well, the most common way to fact-check a GPT is to look at the data on which it has been trained. Or, look at the references or citations. The problem is that a GPT is created to sound natural rather than to provide sources for all its prompts. So, not to be logical, but to analyze the language patterns of a language, fit the subjects, the verbs, and the objects into a sentence, and provide something that mimics the language we speak.

So, factual answers are taking a backseat; all importance is placed on sounding natural. So, can an algorithm with no logic (apart from language logic) be held accountable? Still, no.

*Disclaimer: Microsoft’s Copilot provides citations, but they’re still blogs that might also provide inaccurate information, so a fact-check is required.

Google Search

Google was created with an exciting idea: provide accurate search results on the internet. Accurate, fact-checked, and relevant results.

The algorithm they devised is called PageRank, which ranks webpages by importance, similar to how scientific journals operate; articles that are more often cited throughout academic circles are more relevant to a specific publication.

Of course, there is also SEO – a whole process in marketing where people try to make their websites more relevant, but Google still updates the algorithm regularly to keep things controlled. They issued a core update earlier in 2024 to address AI-generated content and give more validity to human-generated content.

With Google Search, you are still doing your own research. By doing your own research, you can quickly stumble upon new and relevant data, understand the topic you’re researching better, and develop your opinion. Secondly, you can access many fact-checked repositories (Google Scholar, Science.gov, CORE, etc.) and find the latest information on the topic. While researching, you can often find related prompts that people also searched for.

What Google does is that they index the internet; the PageRank algorithm provides data points and suggestions for further research. With ChatGPT, you can only rely on its training data, which may be irrelevant now.

What does it mean for decision-making?

With Google, you still decide what you need and where you want to go. It doesn’t ‘talk’ to you. It doesn’t lead a conversation. Instead, it provides the information found on the web based on the things you search for. ChatGPT instead retells the facts and how it ‘understands’ them, which can be outdated or untrue. It also rarely quotes its sources or how it found them, leaving you without the ability to double-check what it’s telling you or having to just trust it if you can.

Even OpenAI states in their ToS that the output may be inaccurate, incomplete, inappropriate, and incorrect and shouldn’t be taken at face value. However, it doesn’t provide sources to double-check the output for accuracy!

So, should you base decision-making on ChatGPT? In short, no. You depend on its ‘will’ (if it even has one) to sometimes see the data. You can discuss the data with it (or any rubber duck will suffice), but it should never, ever, EVER be allowed to make a decision instead of you. No computer can ever be held accountable, regardless of its neural networks, its understanding of data, or its moral obligations. Only humans can still do that.

As a tool, AI is great, just like Google Search. However, as a decision maker, it lacks the capacity to think; it can only repeat information, no matter how complex, without comprehending it. Only humans and animals are still able to do that.

Can you use both Google Search and ChatGPT as tools? Yes. Should you trust them?

Oh, and one more thing. Here’s what you can find on Google Search for the term ‘Yandex ranking factors’:

And here’s ChatGPT’s response to the prompt ‘Yandex ranking factors’:

The good and the bad

Whether we like it or not, ChatGPT and numerous other Generative AI tools have found their way into many people’s daily work. These tools do help in our daily work by giving us rough ideas we can build upon later. As a marketing and branding company, we often use it to brainstorm ideas, do preliminary qualitative research about a topic, or create first drafts or rough sketches for internal projects. However, we see the use of Gen AI as a starter – never as a finished product.

So, we use Google Search to get the depth of information, and ChatGPT to create an angle for said information. For example, here is a process for quickly adding a custom widget to our presentation or a test website we’re working on with a client.

When building a custom widget, we can design said widget and then add the design to a Gen AI tool, like Cursor.ai or Blackbox.ai, to see how it would code it. Then, a developer and a designer build the widget together, add functionalities, customize the code, and reiterate until they get what they need. But even then, mistakes can go unnoticed if you don’t have an expert two steps ahead of Gen AI who knows what they’re doing. In our case, a developer who understands the code must be present throughout the process. Otherwise, we can easily get spaghetti code that doesn’t do anything, messes up a website’s SEO, or makes some other site functionality crash.

Here is how we stop this scenario from happening.

Expert assessment is necessary

To the untrained eye, a code is a code. People whose job is not in software development can make code work through trial and error, but they won’t follow industry standards or be mindful of numerous pitfalls when that piece of code meets users. Even expert developers overlook some bugs in the code, and then come back to fix them once users report the bugs. Robust systems like Meta or Google have bug-hunting bounties. For example, Meta paid out 22 million USD in total bug-hunting rewards.

According to GitHub, if a person understands the code, understands how it should work, and follows best security practices, they can finish more in almost half the time. But, if a bug appears, they need to deeply understand the code to fix the issue. Another study by Bilkent University states that Gen AI tools (ChatGPT, GitHub Copilot, Amazon CodeWhisperer) generate correct code only 48% of the time. Without an expert developer checking the code for quality, the said code isn’t valid, reliable, secure, or maintainable.

Content speed ≠ quality

In-depth topic knowledge is necessary if anyone plans to use Gen AI when writing. We can provide an example for copywriting.

Anybody can go to ChatGPT and prompt it with a question to write an ad about a product or service in the style of David Ogilvy (one of the greatest copywriters, by the way!). But, to prompt it this way, you need to know how Ogilvy wrote, what copywriting formulas he used, and why using a specific formula makes sense. You need to then refine the copy, adapt it to your target audience, and add keywords because Ogilvy didn’t follow digital marketing practices, but you have to.

Still, the better way is to use the old-fashioned way – find an ad by Ogilvy via Google Search, copy it by hand, read it aloud, then try to use that knowledge to write about your product or service.

Or is it a hallucination?

With both these examples, we rely on people’s expert, in-depth knowledge to act as fact-checkers when using Gen AI, because it has been proven that Gen AIs have a tendency to make up information. Essentially, they are large language models, programmed to produce one sentence after another, not act as an entity guided by ethics. If taken at face value, the information we unconsciously spread can harm the brand, digital property, and the people who rely on it. To minimize these risks of hallucinations, make sure you prompt ChatGPT or other Gen AI tools to cite their sources. Although they can make mistakes and even hallucinate citations, having a link to follow allows you to fact-check the response quickly.

ABC - Always Be Checking

Today, it’s no use comparing Gen AI tools to Google; these are two very different tools. If you have experts on the job, they have experience with both. However, we noticed that Google still doesn’t hallucinate when it produces search results. Sure, people have found numerous ways to skew the results slightly in their favor (that’s how SEO started, essentially), but you don’t get one result; you get multiple. Depending on your query, you can ask Google Scholar for scientific search results, YouTube for video how-tos, or the good old Google Search for almost everything else.

However, you always need to be one step ahead of the Gen AI you’re using because you’re responsible for potential AI slop that it can produce.

DISCLAIMER: This text hasn’t been AI-generated since no AI will say things like ‘we’re in a pickle’ or ‘conundrum.’