Running AI models locally started as a productivity experiment for me. With Ollama, I could test different LLMs on my own hardware and use them to summarize documents, work with notes, analyze PDFs, and understand code. But somewhere along the way, I became more interested in trying new models than improving how I actually used AI.
Every new LLM release felt like something I had to test and keep. My local setup kept growing, but my productivity didn’t improve. Eventually, I realized I was treating AI models like Pokémon. I was trying to collect them all when I only needed a few good ones.
My AI setup slowly turned into a model graveyard
I was downloading models faster than I could actually use them
When I first started experimenting with local AI models, I wanted to try almost everything. I downloaded Llama, Mistral, DeepSeek, Qwen, Gemma, GPT-OSS, and several variants of these models. Every new release sounded promising, and since Ollama and LM Studio made pulling a model so easy, I rarely thought twice before adding another one.
The problem was that I barely used most of them after the initial testing. I would run a few familiar prompts, compare the response with my current model, and then move on. The model would stay on my SSD for weeks or months, waiting for a use case that never came.
At one point, I had a long list of models but still relied on the same two or three for almost everything. I wasn’t building a better AI setup. I was simply collecting models and wasting storage on a growing model graveyard.
Benchmark chasing made the problem worse
I was comparing numbers that had little to do with my work
In the early days of self-hosting local LLMs, benchmark scores had a strong influence on how I judged an AI model. I would look at reasoning, coding, and instruction-following scores and assume the model sitting higher on the chart was automatically the better choice. The comparisons looked objective, so it was easy to trust them.
However, those scores rarely reflected how I actually used AI. I spend more time summarizing long documentation, extracting useful information from PDFs, brainstorming ideas, and understanding existing code. A model could perform brilliantly on a popular benchmark and still produce an unnecessarily long summary or take much longer to complete a basic task.
I also started noticing that small benchmark gaps were almost impossible to spot during regular use. What I noticed was response speed, resource usage, and whether I had to rewrite my prompt three times. I was paying attention to the wrong numbers. My own daily tasks were a much better benchmark.
I gave every model a job
My model list finally started making sense
The biggest change I made was simple: every model in my setup needed a clear purpose. Instead of picking one based on what I felt like using that day, I started assigning models to the tasks where they worked best.
I use DeepSeek 14B for most of my everyday tasks because it runs smoothly on my hardware and responds quickly. For longer documentation and PDF analysis, I keep GPT-OSS 20B around. It is slower, but I find its responses more detailed and nuanced when the task needs deeper analysis. Qwen 2.5 Coder is my choice for understanding unfamiliar code, debugging, and refactoring existing snippets.
This small change removed a surprising amount of friction from my setup. I no longer spend time switching between five models to see which one gives a slightly better answer. I already know which model to open for a task. More importantly, every model now has to earn its place in my setup.
I focus more on my AI workflow than model stack
Better connections made a bigger difference than better models
Once I stopped spending so much time thinking about models, I started improving the surrounding workflow. That turned out to have a much bigger impact on how useful local AI was for me. I connected Ollama with Logseq, Home Assistant, Paperless-ngx, Obsidian, and VS Code so I could use AI closer to where my actual work was happening.
I can work with my notes without copying everything into a separate chat window. In VS Code, I can ask questions about existing code while I am already looking at it. I also use Open WebUI as my main interface for a regular AI chat experience.
These changes may not sound as exciting as testing a newly released model, but I notice their impact every day. Removing copy-pasting and constant app switching saves me more time than a small jump in model quality. I now care more about where AI fits into my workflow than how many models are installed on my machine.
I still test new models, but don’t collect all
I haven’t stopped experimenting with new AI models. I still get curious when a promising model is released, especially if it claims to be faster, lighter, or better at a specific task. The difference is that I no longer keep everything I test.
A new model now has to offer a meaningful improvement to stay in my setup. If it works better for my needs, it can replace an existing model. Otherwise, I remove it and move on. There will always be another LLM release around the corner. I still enjoy trying them, but I no longer feel the need to catch them all.
