I saw a talk recently of somebody translating manuals for new medical devices. She said translating software is not helpful because it’s a very specific field and the devices are brand new. Maybe give it a try.
Alibabas Qwen were among the first open weights models that were actually useful and can be run on consumer hardware without too much difficulties.
If they continue with that, they will hurt the business model of the big AI companies significantly, accelerating the burst of the bubble.
Yeah, a higher quant would be nice, I actually try not to go below Q5, but you can domino’s so much with 16GB of VRAM and the ddr4 system RAM.
But I must say I‘m pretty impressed by Qwen3.6-35b, not only from its capabilities but also from hardware requirements. MoE for the win I guess.
RWKV sounds interesting, have to look into it, thanks!
Yeah, I wouldn’t use it for coding. It’s a bit dumb unfortunately.
Looping was a problem after reaching a certain context window size. The llama.cpp flags - -flash-attn on and looping penalties helped.
I‘m not a coder, so I don’t know exactly. It is able to code, but I would say somebody with experience should guide it and have an eye on the results.
Have you tried qwen3.5-9b? It’s pretty solid for its size.
I don’t host it exactly, just use it when I don’t use my graphics card for gaming. I run Qwen3.6-35b on my 16gb vram RX 9700 xt with 34t/s. I use it as an IT advisor, admin and Linux teacher for my cachyOS gaming PC.
To be fair, humans anthropomorphising everything, of course they do that with software that can fucking talk to them.
Various AI features for Ubuntu Linux are expected to land over the next year with a bias on local inferencing by default. Canonical engineers will be working on integrating agentic workflows into Ubuntu for those that want it. There are areas being explored for AI use on Ubuntu both for the desktop as well as for Ubuntu servers such as for assisting in interpreting system logs
Sounds actually reasonable. As long as it doesn’t get shoved down the users throat it could turn out fine. And sifting through logs is in fact a good task for LLMs in my opinion.
Mark Zuckerberg: „Why put in all those effort with burner accounts when you just can pay to stalk someone!“