• 0 posts
  • 7 comments
Joined 10 months ago
Cake day: August 27th, 2025
  • I have a simple slow model running on CPU in my cluster for karakeep. I’ve tried running a variety of models on my 7900XT but even with 16GB their performance just isn’t there. My new work m5 Mac book with 48GB of ram is the first time I’ve seen usable performance for local models and it has been pretty impressive.

  • This is really great, especially as a jumping off point. You might consider a ranked approach, like good, better, best. Most marginally privacy conscious services are going to be better than their Google analog, but some are better.