Installing Ollama securely: a step-by-step plan for local LLMs

You want to experiment with a local LLM. Makes sense - it’s cheaper, your data stays in-house, and you learn how this technology works. But if you do it wrong, your AI server is open to the entire internet within an hour.

That is no exaggeration. Researchers at SentinelOne discovered that 175,000 Ollama servers are reachable exactly like that. Open, without a password, ready for abuse.

This is the step-by-step plan to prevent that. We run Ollama ourselves for sensitive document analysis, it’s part of our approach to becoming AI-native.

175,000 Ollama servers unsecured: what went wrong?

The researchers at SentinelOne found 175,000 Ollama installations that are publicly reachable. Spread across 130 countries, with China (30%), the US and Germany as frontrunners.

The pattern is always the same: someone installs Ollama, wants to access it from another machine, adjusts the configuration, and forgets that the machine is now reachable for the entire internet.

Nearly half of these servers support “tool calling” - the ability for the LLM to execute code and call external systems. That means attackers can not only abuse your compute power, but potentially gain access to your network.

The irony: Ollama is securely configured by default. It only listens on localhost. The problem only arises once you change that setting.

Step 1: Keep Ollama on localhost

Ollama listens by default on 127.0.0.1:11434. That means: only your own machine can reach it. Not your home network, not the internet. Only you.

Leave it that way.

The temptation is great to set OLLAMA_HOST=0.0.0.0 so you can reach it from your laptop while it runs on your server. Don’t do this. There are better solutions - more on that shortly.

What you can do

• Bind to localhost: OLLAMA_HOST=127.0.0.1:11434
• Or bind to your Tailscale IP (see step 3): OLLAMA_HOST=100.x.x.x:11434

What you never do

• OLLAMA_HOST=0.0.0.0 - this opens the port for all interfaces
• Port forwarding in your router to port 11434
• Firewall rules that open 11434 for incoming traffic

Why this matters: the same kind of misconfigurations led at Moltbook to a leak of 4.75 million records. Open ports are open ports, whether it’s a database or an LLM.

Step 2: Firewall as a safety net

A firewall is your second line of defense. Even if you accidentally change the wrong setting, the firewall stops unwanted traffic.

What does a firewall do?

A firewall determines which network traffic may enter and leave your computer. You can set rules: “block everything from outside on port 11434” or “only allow traffic from this specific IP address.”

Most operating systems have a built-in firewall. All you have to do is turn it on and add the right rule.

Per operating system

Linux (Ubuntu/Debian):

sudo ufw enable
sudo ufw deny in 11434

The first rule activates the firewall. The second blocks all incoming traffic on port 11434 - the port Ollama runs on.

macOS:

Go to System Settings → Network → Firewall → Options. Turn the firewall on. macOS blocks incoming connections by default for apps that are not explicitly allowed.

Windows:

Open Windows Firewall (search for “firewall” in the start menu). Click “Advanced settings” → “Inbound Rules” → “New Rule”. Choose “Port”, enter 11434, and select “Block the connection”.

Why this is important

Suppose six months from now you change something in your Ollama configuration. You forget that this opens the port. Without a firewall, your server is then directly exposed. With a firewall, your safety net stops it.

Step 3: Tailscale for secure remote access

This is where it gets interesting. You want to reach Ollama from another machine - your laptop while it runs on your server, or from the office while it sits at home. How do you do that without opening the port for the entire internet?

The answer: Tailscale.

What is Tailscale?

Tailscale is a tool that creates a private network between your devices. It works like a VPN, but simpler. You install it on your server and on your laptop, and they can reach each other directly via encrypted connections - without having to configure anything in your router.

Each device gets its own IP address in the 100.x.x.x range. Those addresses are only reachable for other devices in your Tailscale network. For the rest of the internet, they don’t exist.

Why Tailscale?

There are several tools that use the same principle - a mesh VPN based on WireGuard. Alternatives include ZeroTier, Netbird, and Headscale (a self-hosted variant of Tailscale).

We chose Tailscale for two reasons:

1. The free tier is generous enough. Up to 100 devices and 3 users without paying. For an organization or home user, that is more than sufficient.
2. Configuration is minimal. Install, log in, done. No manual key exchange, no port forwarding, no firewall rules. It just works.

If you already have experience with one of the alternatives, the principle works the same. The point is that you use an encrypted private network instead of opening ports.

Installation

On your server (where Ollama runs):

1. Install Tailscale via tailscale.com/download
2. Run tailscale up and log in with your account
3. Note the IP address you get (for example 100.78.42.15)

On your laptop/workstation:

1. Install Tailscale
2. Log in with the same account
3. You can now reach the server via the 100.x.x.x address

Configuring Ollama for Tailscale

Now you adjust Ollama to listen on your Tailscale address:

export OLLAMA_HOST=100.78.42.15:11434

Or add this to your .bashrc or .zshrc to make it permanent.

Ollama is now reachable from all your Tailscale devices, but invisible to the public internet. You need no port forwarding, no firewall exceptions, no hassle.

Why this is better than port forwarding

What you should not do: Tailscale Funnel

Tailscale has a feature called “Funnel” with which you can make services public over the internet. Don’t use this for Ollama. Funnel is meant for websites you want to share, not for AI servers you want to protect.

Step 4: Reverse proxy with authentication (optional)

This is an advanced step. Only relevant if you want to share Ollama with others - for example colleagues or an application you’re building.

What is a reverse proxy?

A reverse proxy is an intermediate layer between the outside world and your application. Traffic first arrives at the proxy, which checks whether it’s legitimate, and then forwards it to Ollama.

Think of a receptionist who checks visitors before they may enter the building.

When do you need this?

• You want to share Ollama with team members who are not in your Tailscale network
• You’re building an application that uses Ollama as a backend
• You want logging and rate limiting (limiting the number of requests)

If you use Ollama only for yourself via Tailscale, you can skip this step.

How does it work?

You put a web server (Nginx or Caddy) in front of Ollama that asks for a password. Without the right credentials, you don’t get in.

# Nginx example - asks for a password
location /ollama/ {
    auth_basic "Ollama";
    auth_basic_user_file /etc/nginx/.htpasswd;
    proxy_pass http://127.0.0.1:11434/;
}

Caddy makes it even simpler with automatic HTTPS. But for most home users, Tailscale is sufficient and a lot simpler.

Step 5: Keep Ollama and your system up to date

A secure setup only stays secure if you maintain it.

Installing updates

Ollama receives regular updates with bug fixes and security patches. Check monthly whether you’re running the latest version:

ollama --version

You update by running the installer again or via your package manager.

Don’t forget your operating system either. Many vulnerabilities are not in Ollama itself, but in the underlying software.

Check your own setup

Once a quarter: check whether your configuration is still correct.

• Is OLLAMA_HOST still on localhost or Tailscale?
• Is your firewall still running?
• Are there ports open that you’ve forgotten about?

A quick test: ask someone outside your network to scan your public IP on port 11434. That port should be unreachable. Also think about which tools you use alongside Ollama: 12% of AI skills turn out to be malicious, a vulnerability in a skill can undermine your entire setup. And who works with those tools: Gen Z adopts AI faster, but needs the same security guidance.

Ollama security checklist

A quick check of whether your setup is secure:

• Ollama listens on localhost (127.0.0.1) or Tailscale IP (100.x.x.x)
• Firewall is on and blocks port 11434 for incoming traffic
• No port forwarding to 11434 in your router
• Remote access goes via Tailscale, not via open ports
• Tailscale Funnel is off for Ollama
• Software is up to date (Ollama, OS, Tailscale)

Everything checked? Then you’re not part of those 175,000 exposed servers.

Working on AI infrastructure in your company yourself? Let’s talk about how to approach it securely.