Meta Releases Llama 2 — Run AI Models on Your Own Linux Server

Meta releases Llama 2 under a permissive open licence — making it the first truly capable open-source large language model that server administrators can run on their own infrastructure.

Why Self-Hosted AI Matters

• Privacy — sensitive data never leaves your server
• Cost — no per-token API fees after initial setup
• Latency — local inference can be faster than API calls
• Control — fine-tune on your own data
• Compliance — meet data residency requirements

Server Requirements for Llama 2

# Model size vs RAM requirements
Llama 2 7B  (quantised): 8GB  RAM — runs on most VPS
Llama 2 13B (quantised): 16GB RAM — mid-range server
Llama 2 70B (quantised): 48GB RAM — high-end dedicated server
Llama 2 70B (full):      140GB RAM — enterprise only

Install Ollama — Easiest Way to Run Llama 2

# Install Ollama on Linux
curl -fsSL https://ollama.com/install.sh | sh

# Pull and run Llama 2
ollama pull llama2
ollama run llama2

# API server (runs on port 11434)
curl http://localhost:11434/api/generate -d '{
  "model": "llama2",
  "prompt": "Explain nginx rate limiting"
}'

Run as a Service

# Create systemd service
sudo nano /etc/systemd/system/ollama.service

[Unit]
Description=Ollama AI Server
After=network.target

[Service]
ExecStart=/usr/bin/ollama serve
Restart=always
User=ollama
Environment=OLLAMA_HOST=0.0.0.0:11434

[Install]
WantedBy=multi-user.target

sudo systemctl enable ollama --now

Secure with Nginx Reverse Proxy

server {
    listen 443 ssl;
    server_name ai.yourdomain.com;

    auth_basic "AI Server";
    auth_basic_user_file /etc/nginx/.htpasswd;

    location / {
        proxy_pass http://127.0.0.1:11434;
        proxy_set_header Host $host;
    }
}

Conclusion

Llama 2 makes self-hosted AI viable for the first time. Our team configures AI server environments including Ollama, GPU optimisation and secure API proxying.