Tag: local llm

Posted on

Updating ollama to store models on an NAS NFS share

Create a mount point:

sudo mkdir /mnt/[mountpoint]

To temporarily mount the share:

sudo mount -t nfs [nas-ip]:/volume1/[nfs-sharename]  /mnt/[your-mount-point]

To mount the share permanently, add to /etc/fstab and then sudo mount -a:

[nas-ip]:/volume1/[nfs-sharename]  /mnt/[your-mount-point]  nfs  defaults,_netdev  0  0

Update Ollama’s config to tell it the new location for it’s models – edit ollama.service:

sudo nano /etc/systemd/system/ollama.service

Add:

Environment="OLLAMA_MODELS=/mnt/[your-mountpoint]"

Reload service configs and then restart ollama:

  sudo systemctl daemon-reload
  sudo systemctl restart ollama

Models downloaded from this point will be stored on your NFS share.

Posted on

Connecting openweb-ui to ollama

I’m running Ollama as a service on Ubuntu 24.04, and after starting openweb-ui as a Docker container with:

docker run -d -p 3000:8080 --network=host -v open-webui:/app/backend/data -e OLLAMA_BASE_URL=http://127.0.0.1:11434 --runtime=nvidia --name open-webui --restart always ghcr.io/open-webui/open-webui:cuda

I’m unable to select a model in the ui and seeing this connect error in the container logs:

open_webui.routers.ollama:send_get_request:92 - Connection error: Cannot connect to host host.docker.internal:11434 ssl:default [Domain name not found

After reviewing the logs with Claude, it pointed out that Ollama was crash looping with an error attempting to open port 11434, and suggested that my updates in /etc/systemd/system/ollama.service to attempt to fix this issue were on the right track but attempting to add the OLLAMA_HOST env with a port to the same line defining the PATH results in an incorrect concat of the values:

Environment="OLLAMA_HOST=0.0.0.0:11434:PATH=/usr/local/sbin:..."

This needs to be split into 2 lines:

Environment="OLLAMA_HOST=0.0.0.0:11434"
Environment="PATH=/usr/local/sbin:..."

-Reload the service config and restart with:

sudo systemctl daemon-reload
sudo systemctl restart ollama

It also recommended to change the Ollama URL in the Connection settings in openweb-ui to 127.0.0.1:11434 and now open-ui is working as expected.

Posted on

Running local LLMs: ollama + opencode with gemma4:e2b

Assuming you have ollama installed and a local model already downloaded, install opencode following steps here.

Create an opencode.json with config pointing to your downloaded model:

{
  "$schema": "https://opencode.ai/config.json",
  "provider": {
    "ollama": {
      "npm": "@ai-sdk/openai-compatible",
      "name": "Ollama",
      "options": {
        "baseURL": "http://localhost:11434/v1"
      },
      "models": {
        "gemma4": {
          "name": "gemma4:e4b"
        }
      }
    }
  }
}

Start opencode from ollama with:

ollama launch opencode

Scroll down to other models and select the model you configured above.

Proudly powered by WordPress