If it hasn’t hit you just yet, AI tools are only going to get more and more expensive to use. The usage you could once get out of the free tier is half of what you now get on the $20 plan, and the $20 plan increasingly feels like the on-ramp to a $100 or $200 tier that didn’t even exist eighteen months ago. Besides tools getting pricier to use, the funny bit is that they’re getting more limited while they do it. You would expect paying more to mean fewer walls, not more of them.
Unfortunately, the meters keep multiplying. Every provider is quietly re-drawing the same line — the thing you were doing yesterday still works, it just costs more today, or comes with a usage meter that empties faster than you remember it doing last month. This is why, now more than ever, it’s worth asking yourself: how much of what you use these tools for actually needs the cloud at all? After asking myself this very question countless times, I decided to install a free local LLM on my phone and ditch my reflexive habit of reaching for the cloud every single time.
The stuff I do on my phone was never complex to begin with
The work is small and often personal
Frankly, I don’t use AI all that much on my phone for actual tasks. The real work that actually needs a capable model always happens at my desk where all the powerful features actually exist. For instance, with Claude, Claude Code and Cowork are the two features I use fairly often, and I only use them on my desktop. Sure, Dispatch exists, and you can use Claude Code on your phone through the mobile app. However, doing agentic work on my iPhone’s screen is only something I’d do out of sheer desperation, and I’d much rather wait until I’m back at my desk than squint through a multistep task on a 6-inch display.
My phone, though, has always been where AI goes to do the small stuff. I’ll use it to search up a quick definition, an analysis of something I read on Instagram, a quick math question, perhaps a random question I had about my finances, and so forth. Simple, yet often slightly personal. The kind of throwaway query I don’t think twice about, but which, now that I say it (I mean write) out loud, I’ve been routing straight to someone else’s server every single time. A definition, sure, who cares. But a half-formed question about my own money or a screenshot of a DM I want a second read on? That’s the stuff I’d rather keep on my own device, and it just so happens to be exactly the stuff a local model handles without breaking a sweat.
I’ve been running Gemma 4 on my iPhone, and setup took minutes
Local LLMs, as the name suggests, run entirely on your own device. AI labs have been directing their efforts towards making these models smaller and more efficient without gutting their capability. While my poor MacBook with 8GB of RAM collapses anytime I try to run a local LLM with another app running, I’ve been using my iPhone 15 Pro Max to run Google’s Gemma 4 models locally.
The setup process was as simple as it gets. I installed Google’s Edge Gallery app via the App Store, and all I had to do was download the Gemma 4 model from within the app. You don’t necessarily need to use Google’s Edge Gallery, though! For instance, XDA’s Nolen uses PocketPal to run Gemma 4!
The model itself is a chunky download, so I’d do it on Wi-Fi rather than burning through mobile data, but it’s a one-time thing! Once it’s installed on the device, it stays there, and everything after that happens offline. Gemma 4-E2B-it, the model I run, is a 2.54 GB install, so do keep that in mind.
What I actually do with a local LLM on my phone
Turns out my phone was overqualified for the small stuff
Once I stopped thinking of it as a novelty, the local model quietly absorbed a genuinely large chunk of my daily phone queries. The kind of thing I’d have reflexively opened Claude for now just happens on-device, and often faster, because there’s no round-trip to a server involved.
Take the boring stuff first. A unit conversion, a math question someone fires at me, a “what’s another word for.” These are all quick queries I once used to route all to a cloud tool out of habit. Now it goes to Gemma 4 running locally, and the answer starts streaming before a cloud model would’ve finished loading. For low-stakes, high-frequency queries, the responses I get from Gemma 4 work fairly well!
The part that actually changed my behavior is the private stuff. Say I want to draft a quick message to my travel agent, the kind that includes my passport number. Normally, sending that to a cloud LLM means either censoring the confidential bits first or just accepting that my passport number is now sitting on someone else’s server. With everything running on-device, that whole issue disappears. Nothing leaves my phone, so there’s nothing to redact. Same for a screenshot of a DM I want a second read on, or a half-formed question about my own finances.
Since the model lives on the device, an internet connection is completely optional. I could be several thousand kilometers up in the air on a flight with no Wi-Fi and still ask my phone to walk me through the immigration process for wherever I just landed.
The multimodal side holds up better than I expected, too. Edge Gallery’s Ask Image feature lets me point my camera at something or feed it a photo and ask questions about it, and it’s good enough that I’ve stopped reaching for a cloud tool for quick visual questions — what is this, what does this label say, read this handwriting. It’s not going to match a frontier cloud model on hard visual reasoning, and I wouldn’t pretend otherwise, but for the everyday questions, it holds up well. Again, since it’s all local, I never think twice about what’s in the photo.
There’s also Audio Scribe, which handles on-device transcription and translation, so that’s one more thing I no longer need a separate AI tool for. Through the app’s Agent Skills, you can augment the model with tools like Wikipedia for fact-grounding and interactive maps. While this is obviously not a full Google Maps + Gemini replacement, it’s a neat way to pull it a little closer to the capabilities you’d expect from a cloud assistant.
Oh, and the limits just… don’t exist
If you’re a Claude user, you know how frustrating the tool’s limits are. While practically every cloud LLM now has limits, Claude’s limits are especially annoying. That’s the entire reason why upgrading to Pro over the free tier is less of a luxury and more of a necessity if you actually lean on it day to day! Back when I was on the Pro tier, I’d constantly hit the session limit on my phone (without using Claude Code or the fancy stuff), since I’d keep asking it questions within the same window. All of those questions are now routed to Gemma 4, meaning they never touch my Claude quota at all! The trivial stuff stays on my phone, and my Claude usage gets reserved for the work that actually justifies it.
While I’m not insinuating that the local LLM is better (because it isn’t), offloading the trivial queries to something free and unmetered means I stop hitting walls on the queries that actually matter. My frontier-model quota lasts noticeably longer when I’m not spending it on tasks that don’t really matter, and the small stuff is effectively unlimited because it’s running on hardware I already own. It’s less about the local model winning and more about using each tool where it makes sense!
