Hacker News
points by ohmahjong 2 hours ago | 0 comments
Man, my M1 Pro unfortunately only has 16GB of RAM; it has been such a limiter to using local models. I'll give kapitsa a try, I see I should be able to squeeze some smaller models on
[–] yieldcrv 1 hour ago
kapitsa is customized well to just a small selection of models that are in vogue right now, focusing on qwen and nemotron

compared to llama.cpp that tries to cater to all but does so only averagely

check the catalogue and see if one of the quantizations fit