Quick Introduction We have all been on forums, chats, reddit, discord, youtube, or somewhere and heard “Oh! Model XYZ is AMAZEBALLZ!zomgwtfbbq” then downloaded it (or more likely, some quantized form of it) and said “eww… This sucks!” This post is going to be a rather technical series of experiments to demonstrate the impact of implementation-specific hazards with inference. I will be using the term “reference implementation” to describe the lab that published and offers first-party hosting of ...
  • h0tbeef@lemmy.zipEnglish
    9 hours

    If you ascribe any level of sentience or intelligence to a computer, you definitely feel much less dumb than you actually are.

      • brucethemoose@lemmy.worldEnglish
        9 hours

        I mean, I’m all for local LLMs, but they are as sentient as a really good weather prediction model.

        aka not at all.

        Its just a fact of how they operate, mechanically. They are missing too many characteristics for it to even be an entertainable question.

        • hirihit640@sh.itjust.worksEnglish
          7 hours

          They might operate differently but they still appear the same as us, 99.9% of the time. Imo if it looks like a duck and acts like a duck…

          • h0tbeef@lemmy.zipEnglish
            5 hours

            Do you not interact with any other actual humans?

              • kadu@scribe.disroot.orgEnglish
                36 minutes

                I come back to Lemmy after 6 months and immediately read this.

                So that was a mistake.

              • h0tbeef@lemmy.zipEnglish
                3 hours

                Well, that may very well be the most depressing comment I’ve ever read.

                You either need to work on your listening skills or find some new less-vacant people to interact with, because what you’re saying is insane

                • hirihit640@sh.itjust.worksEnglish
                  2 hours

                  Sounds like you haven’t tried modern AI yet. I could train an AI on your comment history and copy you, and nobody would be able to tell the difference

          • brucethemoose@lemmy.worldEnglish
            7 hours

            I mean… No? Absolutely not.

            I dont even know where to begin. Maybe with their state being fixed in time; LLMs do not change. An sci fi analogy might be the “no timers” in orion’s arm, who are but a single thought frozen in an infinite loop in time:

            https://www.orionsarm.com/eg-article/47f4311eaef31

            Except they arent even that, because its just a next word completion model, not something that thinks. It is not self aware.

            https://arxiv.org/pdf/2503.09211

            https://openreview.net/forum?id=klU4737opt

            https://arxiv.org/abs/2504.09762

            This becomes (to me) very obvious if you ever use an LLM in raw completion mode. It very smart, but at the end of the day its no different than a weather prediction model spitting out probabilities for a storm system.

            Chat finetuning is meant to get humans to anthropomorphize them by training on human preferences, and the interface further reinforces this. Its all a trick, albeit a very elaborote one.

            Will future architectures be closer to “thinking?”

            Maybe.

            But we are a long way away.

            • SlimePirate@lemmy.dbzer0.comEnglish
              4 hours

              The fact they are trained to complete sentences and are thus word predicting models only tells us vaguely how they work. It does not tell us anything about their sentience.

              The sentience, if there is, comes from the emergent reasoning that develops in the massive neural network to offer actual good predictions: the more you think, the better the predictions.

              For example, models trained to produce textual board games predictions can be sounded to show they developed subnetworks corresponding to an inner representation of the states of the game. Models trained to produce images have emergent subnetworks on depth-maps, contours and lighting.

            • hirihit640@sh.itjust.worksEnglish
              3 hours

              Maybe with their state being fixed in time; LLMs do not change

              Not with agents and chain of thought. Agents can run for hours continuously. So sure, an AI agent is not the same as a sentient human on the time scale of a year, but on the time scale of a few hours, perhaps the AI is “sentient”.

              The first paper you linked is focused on the mechanism and not the output. It creates a definition of “thought” and then talks about how the AI doesn’t “think in the feature space”. It only addresses chain of thought at the end, and says that the issue is that it constrains AI to think in natural language only (instead of, say, pictures).

              But again, why would thinking in pictures define sentience? The paper gives a rock-paper-scissors example and says that the AI thinks about it in a different way than a human. So what? If a human plays rock paper scissors against the AI and the AI’s output is indistinguishable from a human’s, 99.9% of the time, why is that not sentient?

              This is like saying python programmers are not real programmers. If a python programmer can implement the same program in python, who cares what language they use.

              I’d love for a better definition of sentience than “it works differently than humans”.

            • rayboy@lemmy.worldEnglish
              6 hours

              Are we not just prediction systems with more neurons?

              Now im not saying LLMs are human at all because to be human is to have gone through our exact evolutionary path, but we really are in a lot of ways much more similar than you think. At least the speaking part of our brain is.

              The argument that LLMs are fixed in time does not contradict consciousness. Say a human was stuck in a time loop and the human is asked the same question every single loop, at the exact same time, in the exact same way, with every single atom being in the exact place. Would their answer change? I would argue no, the same exact neurons would be stimulated every time that would cause the same exact answer every time.

              We don’t even have a solid definition of what is concious. Are insects conscious? Like the fruit fly, we have modeled all of its neural paths and it simplifies down to essentially being a robot that responds to stimuli.

              What does make humans special is our ability to learn. But what when an LLM trains its self? Is that not learning? I think the question of consciousness is too complex to simply say one way or the other and it is foolish to simply it this way.

              • brucethemoose@lemmy.worldEnglish
                5 hours

                But what when an LLM trains its self?

                Thats what I’m saying. They cannot do this, not even close. Even a fruit fly is somewhat adaptive and can respond to novel stimuli, but an LLM effectively cant.

                There are tiny experimental models closer to a fruit fly than large LLMs now.

                …What about future world models with some adaptive learning loop, sampling stripped out, and so on?

                Yeah! That would be fascinating. Then sentience starts to become more of a question.

                But thats not what LLMs are.

                People like Altman and Modi have grossly overexaggerated what current LLM architectures are capable of, they trained the LLMs to reinforce the illusion as much as they can, and they arent to inclined to change it. Hence all the good researchers have fled to research world models, and are saying transformers LLMs are not a viable path forward.

                • h0tbeef@lemmy.zipEnglish
                  5 hours

                  Even in your highly optimistic scenario, the AI wouldn’t even come close to being comparable to actual sentience.

              • h0tbeef@lemmy.zipEnglish
                5 hours

                I can’t believe how many pro-AI arguments essentially boil down to “I do not understand words”.

                We have a very solid and explicit definition of what conscious means:

                https://www.merriam-webster.com/dictionary/conscious#medicalDictionary

                The best neuroscientists in the world do not even fully understand the complicated intricacies of the workings of the human mind. We do understand 100% exactly how computers work. There is no comparison.

                • SlimePirate@lemmy.dbzer0.comEnglish
                  4 hours

                  Discussing the consciousness of neural networks is NOT being pro-AI. I’d say it’s even the opposite, as it displays the danger of such a technology.

                  Most of the words in the definition have multiple interpretations, it is not as easy as you wish it would be.

                  Finally, we do NOT at all know how trained neural networks work. We know how training and inference work. We can read the neurons, but the shear size of weights make it a rather impenetrable black box we have to study, partially, to understand the goals of the emergent sub networks.

                • rayboy@lemmy.worldEnglish
                  4 hours

                  I haven’t even argued that AI is concious, I have said that we do not understand consciousness enough to define it. You have completely missed the point. This is Dunning-Kruger at its peak. You are vastly oversimplifying things, at what size of brain is an animal considered conscious or capable of thought? Define thought. We don’t even have a solid understanding of life, much less thoughts. Yes we have a word definition, no we do not know where the line begins and ends. It is a philosophical question that remains unanswered.

                  This definition is already making so many assumptions and over simplifications. https://www.merriam-webster.com/dictionary/thought

                  We understand that the brain is what gives us our cognition, we understand that the brain is made up of chemical and electrical signals. We are not special.

      • HubertManne@piefed.socialEnglish
        9 hours

        I 100% say as a sentient human being that no llm I have used has any sentience. It can’t even understand its own output.

        • SlimePirate@lemmy.dbzer0.comEnglish
          4 hours

          Having “real intelligence” is not a sufficient requirement to decide over which systems are sentient or not e.g. there are very low IQ “real intelligences”.

          Dictionary defintions serve for a partial understanding but are not enough to understand scientific concepts, especially definitions that are not under scientific consensus. Moreover, current AI systems, including tool-assisted LLMs can satisfy these definitions.

          Since you like exercising your brain, I’d recommend reading about sophisms.

          Note that I am not asserting that LLMs are sentient, but that your statement is too confident and that the subject requires actual care.

          • h0tbeef@lemmy.zipEnglish
            3 hours

            LLMs cannot fulfill these definitions.

            Try to explain to me how an LLM could fit the definitions of “conscious” or “sentient”.

            If you understood how computers work and also had a firm grasp on the English language, you would understand that there is no way to use either of those words to accurately describe a computer program.

      • squaresinger@lemmy.worldEnglish
        8 hours

        Sentience is roughly defined as “Line in the sand between humans and things we are allowed to eat”.

        It’s a concept created to justify treating animals differently than humans in a world where the old justification (“You can eat animals because God said so”) isn’t applicable anymore.

        Since the concept is defined backwards to justify preexisting actions, it doesn’t have any actually solid definition, and whether something’s sentient or not depends pretty much entirely on the opinion of the person deciding it.

  • brucethemoose@lemmy.worldEnglish
    9 hours

    This is true. But VLLM’s quantization support (as outlined in the article) is quite primitive.

    Thats not what its built for. Its made for maximum speed with many concurrent users, not cramming LLMs into a desktop rig.

    If you are running an LLM for personal use, you are usually better off with a backend optimized for higher fidelity quantization, like MLX DWQs, exl3, ik llama GGUFs and stuff like that.

    If you want raw speed on a desktop, use one of the ninfer project branches.

  • one_old_coder@piefed.socialEnglish
    13 hours

    inari@piefed.zip: Are you a bot? You are posting multiple times every hour of the day.

    • inari@piefed.zipEnglish
      6 hours

      Nope. I post things I find on Reddit or Hacker News sometimes, though.

    • brucethemoose@lemmy.worldEnglish
      9 hours

      Nah, I dont think so. There are comments mixed with posts, there arent that many posts, and they seem to be reasonably curated.

  • very_well_lost@lemmy.worldEnglish
    11 hours

    Having used these extensively, I can say with confidence that they feel exactly as dumb as they are.

    • Franconian_Nomad@feddit.orgEnglish
      8 hours

      Do you mind sharing what models you used and what your experience was? In my opinion the Qwen 3.6 models or maybe the 3.5 were the first local models that were actually useful, but I don’t have that much experience.

      • MagicShel@lemmy.zipEnglish
        7 hours

        Qwen is really good. For coding, it’s the best local model I’ve found.