July 2026: I have a desktop with an Nvidia 2060 with 6GB and was curious whether I’d get any usable performance if I attempted to run an LLM model locally What have I tried so far:
Update Aug 2026: since I started writing this I’ve upgraded to a 5070ti with 16gb vram, so I have a little more realistic vram to play with.
tldr; 6gb vram is enough to run smallest models for general Q&A but not enough to be practical, definitely not agentic coding, but 16gb is doable with a small context.
ollama, opencode with gemma4:e2b
The smallest model I tried first (in terms of number of parameters in the model). Runs reasonably ok for trivial questions. Struggles with any coding tasks on a local repo with opencode – it keeps repeatedly asking the same question, although this might have been before I worked out how to increase the default context size and I may need to take another look.
ollama, opencode, with qwen2.5-coder:14b
Fails to invoke tools (listing the directory for files, gripping file content, running anything outside of the model itself), just outputs json with the tool name and params, but doesn’t seem to actually invoke anything. Read online that it’s tool support was older, and to try the 3.6 models instead
ollama, opencode, with gemma4:12b, 32k context
First actually useful local model. Asked it to review unit tests on a Java project, and create new tests for classes without tests, was was able to call Java and maven to do what was needed.
At this point I felt the model could do enough that I could try one-shotting a request to build a Java web app using JSF 2.3 and run on Tomcat 9 in a Docker container, and older framework on an older version of Tomcat, but a combo I know has some interesting gotchas (JSF 2.3 requires CDI if you use the javax.inject.* annotations, and this is not provided by Tomcat 9), so it would have to know this or work it out in order to reach a working solution). Claude Code Sonnet/Opus 5 can easily one shot this. Here’s my prompt:
Build a Maven based Java 8 webapp using JSF2.3 to run on Tomcat 9. Include a helloworld.xhtml page. Include a Dockerfile to build and run the app on Tomcat 9.
While it recognized it needed to also add a CDI impl like weld, it kept referring to weld versions that were no longer in maven central (I think the groupId and artifactid had changed over time and it was working with knowledge within the model that was out of date.
Also some other minor issues, when adding the weld dependency it overwrote the existing pom.xml rather than adding it to the existing file. When I asked it to out back what it had created before and then add weld it did, but it seemed to require more hand holding and interactive prompting to keep it going in the right direction and point out it’s mistakes. It was not able to produce a working JSF app.
I was curious if I was able to ask factual questions, although without a web search feature. More often than not it responded with out of date answers, but without a web search I guess to be expected. For example, asking about Qwen 3.5 models, it said I was probably asking about 2.5 as 3.5 models have not been released yet.
ollama, opencode, with gemma4:26b-a4b-it-qat, 32k context
Recommended MoE (Mixture of Experts) type model in Reddit subreddits. Definitely capable of coding tasks, able to execute any local tools.
While this model got closer to a working webapp, while creating files and testing starting up the Docker image to rest the results it would random lose track of it’s task and reply:
"I am ready to help you with your software engineering tasks. Please let me know what you would like to do"
Asking it to continue with it’s task does seem to pick up from where it left off, but it’s “a bit flaky”.
It also ran into the “javax.faces.FacesException: Unable to find CDI BeanManager” error and just stopped, saying it would need to investigate. At this point Claude would just do this and work to find a solution, whereas Gemma4:27B seems to need to be told to do this.
ollama, opencode, with Qwen3.6-27B-A3B
Recommended in Reddit subreddits, larger model, moe approach, able to load subset into vram. Discussions that the qwen models are more cable for coding tasks.
I skipped this one for now as suggestions were to try Qwen 3.8 models which were just release this month.
ollama, opencode, with Qwen3.8-27B
Released this month, discussions online and benchmarks are suggesting that the 27b variant of Qwen 3.8 is the first local LLM that’s approaching frontier level capability from 6 months ago, comparable to Claude Sonnet/Opus 4.6 levels.
I think this is right too. this is the first local model I’ve tried so far where it proactively tests if generated code works, checks logs, and if there are errors continues working to resolve the issues. This is more inline with my experience using Claude Code.
ollama, opencode, with hf.co/jrell/Qwen3.8-27B-i1-IQ4_XS-GGUF-Smaller
Here’s where I feel like I’m starting to get into the weeds😀 I’ve started learning about different quantizations for models, where values in the model are stored with lower precision to reduce the model size.
The Qwen3.8-27B is slightly too large to fit in 16GB, but different quant variations of the model get closer. This one is close, but with 32k context it’s still spilling slightly, ollama ps is showing “6% CPU, 94%GPU”. It’s getting me around 20tps which is not great, but it’s somewhat usable.
The issue I’m currently running into however is opencode or the model appear to stop what they’re working on without completing, which is not great. I feel I’ve still some experimenting to do to get a local Claude replacement, but it’s close, definitely close…