Paul

@solarkraft.bsky.social

#FOSS nerd with a recent interest in #LocalAI, anti tech bro tech bro club

Current #LocalAI model picks on my 32GB M1 Max: Main model: Jundot/Muse-Glimmer-30B-oQ4e Small model: UraionLabs/MiniCPM5-2B-oQ4e They fit together because both are efficient at long contexts (most are not). It's already tight, so incorporating a Jev-style model will be a challenge.

How are y’all keeping your agents happy? I haven’t found myself able to give them work for more than an hour before without intervention. I’m such a massive bottleneck and guess J need to orchestrationmaxx …

I’m looking for an LLM-friendly task tracking system that I can dump ideas into, optionally prioritize and just let the LLM attack tasks that are unblocked. Could beads fit? Anything better? I find that this is a pretty strong bottleneck.

Has anyone been able to get Jev-style decisions out of a fully usable LLM? I mean a hybrid model that can generate both text with full decoding AND efficient decisions. Should be possible, right?