Running local AI has become associated with hefty hardware that most people don’t have lying around. A lot of online content references impressive setups that use something like a 4090 and 64GB of RAM just to run AI at home, and those stories had me thinking it’d be pointless to try to run a local model on my old laptop.
My 10th-gen Lenovo with 8GB of RAM and integrated graphics had been sitting in a drawer for the last two years. I pulled it out just to see how bad it’d be. Turns out that with the right lightweight tools, it can run a local AI model surprisingly well. It certainly won’t replace ChatGPT or Claude for any serious work, but it can handle basic tasks like summarization and email drafting without a fuss.
LM Studio is a good fit here
It’s probably the easiest AI tool to get set up
Ollama is constantly recommended for local AI, and I do like it, but I wanted to keep things as simple as possible on this old hardware, so I went with LM Studio. It’s free, and everything is configured through a GUI. You get to use Hugging Face’s model hub instead of pulling models through a terminal window.
The one-click installs are what initially drew me to LM Studio. I also like the chat interface, which looks and feels like ChatGPT’s website, except that everything runs locally. I loaded Qwen3 4B Instruct as my daily driver, since anything larger just isn’t realistic on 8GB of RAM with no dedicated VRAM to lean on. It suffices for simple queries like summarizing documents and answering quick questions.
Another advantage of LM Studio on weak hardware is that it exposes a local API server, so once you’ve loaded a model, other tools can talk to it. This mattered for my setup once I started layering more software on top.
Picking the right model is the most important part
LLMFit matches your hardware with a capable model
Before I found LLMFit, selecting a model was more of a trial-and-error process. I’d download something that looked promising, then have it crawl when answering a question, or simply fail to load at all. For a machine like my laptop, which has very little headroom to work with, finding the best model to run is crucial to the setup.
LLMFit is a terminal tool that detects all the hardware your PC is runningand then recommends a list of AI models it can run reliably. Each suggested model has its own score, and the tool tells you exactly how you can expect each model to run on your hardware. It gave me a handful of models that my Lenovo would be able to run comfortably, so I downloaded a few of the top picks. You can also download the models directly from the tool itself.
Giving local AI a proper memory
You can have conversation context even on outdated hardware
Without AnythingLLM, LM Studio would be significantly less useful out of the box. Given my laptop’s hardware, I didn’t expect my conversations to retain much context. AnythingLLM gives you a fuller chat experience and persistent memory. It pulls facts from past conversations in the background, so I don’t need to re-explain things to the model every time I open a new chat.
It’s a free and open-source frontend that connects to whatever’s already serving your model, which, in my case, is LM Studio’s local API endpoint at localhost:1234. The reason this works so well on my setup is that AnythingLLM’s features impose almost no extra system load, since it’s just a wrapper that sits on top of LM Studio. When you don’t have any resources to spare, it’s a great way to give the local model a better memory system.
Why not just use the free tier of a cloud AI?
Local AI has some inherent benefits over the cloud
It’s true that the free tiers of ChatGPT and Gemini are more capable, and they don’t require you to resurrect old hardware. For any complex tasks that I need to throw at AI, they’ll beat my 4B local model every time. However, local AI has a few natural advantages that cloud AI doesn’t offer.
Cloud AI comes with rate limits, requires an internet connection, and means you’re handing over whatever you type to a third-party server. For many everyday tasks, a small local model running offline on hardware I already own is good enough, and it’s one less thing I need to put on someone else’s servers.
My old hardware found a new purpose
That Lenovo laptop went from drawer-dweller to a machine that I reach for at least a few times a week. What surprised me was that none of these tools required anything the laptop couldn’t handle. If you’ve got similarly modest hardware gathering dust, it’s worth checking what models it can run and giving that system a new purpose.
