Pushing Local Models With Focus And Polish

I tried playing with local models and this article resonates with my experience. There is a ton of complexity and pretty much from the get go you hit a wall where you have many many many choices to make.

The solution proposed is on the right track but only half of it. As the article says 128GB of RAM isn’t a realistic requirement for widespread adoption. I am wondering if the way to go is to pick models that can fit in 16GB of RAM and push them to the limit of the useful things one can do with them. That probably doesn’t include agentic code yet but maybe we don’t need to start from coding