There’s a version of Claude Code that’s for developers, but that’s not what I use it for. Being apprehensive about the dev angle at first meant it took me a while to actually try it, but once I did, it turned out to be genuinely useful for productivity, and it’s been sitting in that role since. It lives inside my plain-text note stack now, stepping in as a task manager, a notes app, and a PKM tool through markdown-aware skills and the CLAUDE.md file.
So the obvious question – if it can read markdown, run scripts against a vault, and pull structured findings out of files that were already sitting there, what happens if you point it at a year’s worth of local LLM chats? LM Studio stores every session as JSON on disk. Mine had been accumulating for months without me ever going back to look at any of it. What came out of that was more than just a boring summary…
Want to stay in the loop with the latest in AI? The XDA AI Insider newsletter drops weekly with deep dives, tool recommendations, and hands-on coverage you won’t find anywhere else on the site. Subscribe by modifying your newsletter preferences!
Setting Claude Code loose on my LM Studio conversations folder
How I structured the audit
LM Studio stores every chat as its own JSON file, organized into whatever folder structure you’ve made yourself, and the files sit locally on your machine. Mine had almost a year of chats across a bunch of different topics, and I’d never really gone back to look at any of it. But now, I wanted to know what my usage actually looked like when I wasn’t paying attention to it, and whether I was getting real value out of running local models or just generating stuff for the sake of it.
Before writing the actual prompt, I opened one of the JSON files to see what fields were even in there. Turns out there’s a lot more than just the messages. Model info per chat, regeneration counts, config settings like context length and temperature, timestamps, attachment metadata, and more. Knowing all that upfront informed how I would be prompting Claude to sort through everything. Instead of asking Claude Code for a summary, I could ask it to look for behavioral signals I might not notice on my own.
I asked Claude Code to work in three phases:
- Phase one was inventory: the basic breakdown of chats, folders, models, regeneration rates, and abandoned sessions.
- Phase two was the main event and was about how I actually prompt: this includes recurring shapes, patterns I keep converging on, prompts I keep rewriting from scratch, and so on.
- Phase three was a separate file for everything else it picked up on.
I also told it to ignore any prompts or system messages inside the chats themselves, because those are content, not instructions. Also, I did remove most of my personal finance and medical-related chats from these folders in the screenshots, so I wasn’t working with a full inventory here.
Turning the audit into something reusable
Creating a prompt library Skill
Phase two gave me enough inventory of recurring prompt shapes I tend to keep using to build a prompt library. For example, there’s a Role-Task-Format scaffolds for structured outputs, a “follow this structure exactly” framing I use for teaching-style prompts, hard-constraint output prompts with strict word counts and no-repeat rules, and capability probes for figuring out which model to reach for. Each pattern came with real examples pulled from my own chats, and the failure modes were flagged too, like bracket placeholders in RTF scaffolds that smaller models sometimes leave uninstantiated.
From there, I prompted Claude Code to build a Skill. It ended up as a folder called lm-studio-prompting living inside .claude/skills/, with a SKILL.md at the top handling the trigger logic, a patterns folder holding one markdown file per recurring shape, a templates folder with copy-paste-ready fillable prompts for the ones I kept reinventing, and a model-notes.md that logs which model I use for what.
I tested it right after with a prompt about summarizing a long PDF using Gemma 4 E4B. The skill fired, routed me to the RTF scaffold with a short rules block, and also flagged a couple of things I hadn’t thought about: that E4B tends to hallucinate citations on doc-QA and Qwen or gpt-oss would be safer picks, and that I’d need to bump LM Studio’s context length above 4k or the PDF might truncate. That last one is a real gotcha I’d been running into for months without realizing.
Applying Claude Code’s findings to my local LLMs
I’ve been misdiagnosing my own setup for months
The Skill is genuinely useful, but a prompt library is just one part of the puzzle. The audit findings from phases one and three ended up being the other valuable half. Claude Code surfaced a bunch of stuff about how I behave with local models that I wouldn’t have caught looking through the chats myself. Half of my chats are single-shot, so one question, one reply, done. Out of 141 messages, I’ve never once edited and resubmitted a prompt in LM Studio. What I do instead is either start a whole new chat or fire the same prompt at a different model, which honestly reads more like model-shopping than iterating. This was pretty shocking to learn about my local LLM habits.
The one that hit hardest, though, was the Wet Road chat. There’s a session in my history where I attached a long PDF and asked Qwen 3.5 9B to help me understand driving rules, and it kept refusing to confirm things that were clearly in the document. I’d been reading that as the model failing the task. Claude Code caught what was actually going on: LM Studio’s default context length is 4096, and I’d never changed it. I’ve written about double-checking your local runner settings, and here I was skipping over the most obvious one – it’s precisely because it’s such an obvious control that I assumed I’d already adjusted it.
Claude Code’s findings are like a reminder of the bad habits I have and help me form better ones with my local LLMs now. From basic things like remembering to adjust the context length to default presets that had been carrying old config values from other models. I also have a better sense of when I’m actually using a local model to do something versus when I’m just generating for fun with no actionable outcome.
I should have done this ages ago
I know the whole point of running local LLMs is keeping the actual chats off the cloud, but Claude reading through the JSON files after the fact isn’t the same thing as Claude sitting inside my LM Studio interface. It’s just an audit on data I already had, and it’s faster than any local setup could have handled anyway. And what came out of it wasn’t stuff I could have realistically surfaced on my own – the patterns are only visible across a year of chats, not any single session. Which means I’m now making better decisions about my local LLM use based on things I didn’t even know were there to fix.
