dandi8

Well, I suppose we can at least agree to disagree.

I have seen so much incoherent but confident nonsense produced by LLMs (mainly by frontier models trying to do even basic software development) that I would not be able to say in good conscience that thought was involved. Junior developers would have done better. The experience definitely fits the behavior of a word predictor, though.

Having seen what LLMs claim about software development, my stance is that absolutely no one should trust at face value what these models output. They’re Dunning-Kruger machines.

As for producing new ideas, these models are as creative as a random number generator. Coincidentally, that’s what is responsible for faking their creativity (the “temperature” parameter).

I guess that’s all I feel like saying in this particular thread.

I trust AI far more than I do a random person. They have access to far more information, and are more likely to be correct about any particular question asked.

That is a terrifying stance. And, frankly, embarrassing.

“OpenAI admits AI hallucinations are mathematically inevitable, not just engineering flaws”: https://www.computerworld.com/article/4059383/openai-admits-ai-hallucinations-are-mathematically-inevitable-not-just-engineering-flaws.html

OpenAI, the creator of ChatGPT, acknowledged in its own research that large language models will always produce hallucinations due to fundamental mathematical constraints that cannot be solved through better engineering, marking a significant admission from one of the AI industry’s leading companies. […] The research proposed “explicit confidence targets” as a solution, but acknowledged that fundamental mathematical constraints meant complete elimination of hallucinations remained impossible.

I think the Wikipedia definition of thought is quite good.

However, I have a feeling whatever definition I came up with, you’d just claim LLMs fit into it because their output is sometimes somewhat coherent.

You can claim that technically LLMs “think” because the output text sometimes contains conclusions, and sometimes they’re even rational, even though the LLMs still struggle with counting Rs in “strawberry”.

I find that disingenuous because it implies that the LLM is in any way aware of anything, that it can passively form ideas.

Most importantly, it implies that you can trust it for even basic reasoning. That you can trust the plagiarism machine that tells you that you should put glue on your pizza, eat rocks and walk to the car wash instead of driving, or that you will be able to trust it at some point in the future.

Whatever definition of thinking we use, it should include a simple rule - that the allegedly thinking entity should demonstrate that intelligence by being able to reliably answer simple queries correctly. Humans, by and large, can do that. LLMs fail at it miserably. If the LLMs were truly thinking, that should be shocking. Understanding the underlying technology - and that it is not truly reasoning - makes it obvious and expected.

Even OpenAI admitted hallucinations are an unfixable mathematical inevitability - something you handwaved as a matter of time to fix. No, the fact that humans can have hallucinations is not comparable.

I hold a MSc from what is arguably the most prestigious University in Europe

Good for you. Have a cookie, I guess?

LLM do not simply regurgitate existing content, and are in fact capable of creating wholly new content not seen before.

Citation needed.

Hallucinations occur when their context buffer is too small, and as time goes on, it will largely be a thing of the past.

A whole book of citations needed. That claim is wildly inconsistent with the consensus about AI hallucinations.

Magic Eight Balls, as I’m sure you’re aware, have a limited, predetermined number of responses.

You mean like how LLMs keep hallucinating the same passwords and nonexistent dependencies to the point that bad actors are using that fact to compromise vibe coded systems via techniques like slopsquatting?

I would disagree with you, and would suspect you are basing your assessment of their abilities on dated usage.

In fact, I keep experimenting with frontier models (including Fable when it was available) just so that the “but we’ve made so much progress in the past few months” argument can’t be used against me. You’re wildly overselling their capabilities.

Except LLM output is largely gibberish. Just confident gibberish. There’s a reason we call it “AI slop”.

LLM responses are only ever “sound” when they’re regurgitating existing information they were trained on. Beyond some simple transformations, they are unable to create original ideas. They very frequently break down on somewhat unique tasks, as evidenced by the ever-prevalent code-slop which is eroding our software.

They don’t have a memory of previous conversations (unless you literally copy-paste it into the prompt), they don’t learn (Claude “memories” is literally just copy-pasting a summary into the prompt, only automatically). They don’t have any “thoughts” of their own between prompts (OpenClaw just keeps prompting them to pretend they are autonomous).

The underlying implementation of “reasoning” in LLMs is literally “hallucinate some more text which vaguely looks like thoughts and hope that influences the answer”. LLMs are probabilistic models which we figured out how to make so they produce somewhat correct-looking answers at a rate a little higher than chance.

Magic 8-balls sometimes give sound responses. Do they think? Where do we draw the line with this interpretation of “thinking”?

No, humans are not word predictors, and my claim is absolutely not an oversimplification.

LLMs are word predictors. No amount of attention heads and backpropagation is going to change that. Scientific researchers agree.

The human brain works in a completely different way to how LLMs do and to conflate the two like you did is disingenuous.

The PS5 is subsidized to get you into the ecosystem. Valve let’s you play anything you like on the Steam Machine, so no incentive for them to sell at a loss.

I imagine they went for the less powerful GPU because hardware prices are insane across the board right now and Valve has no negotiating power with the manufacturers, as they said.

Word predictors don’t think any more than a magic 8-ball does.