Deploying Ollama

Follow the steps below to easily set up a m1a.4xlarge virtual machine (VM) with the Ollama image using the exc CLI.

Prerequisites

1. Log In to exc

   • Ensure that you are logged in to the exc CLI. If you haven’t logged in yet, please refer to the Login and Registration Guide to complete the login process.

2. Verify Login Status

   • To verify if you are logged in, run the following command:

     exc compute list

   • If you receive a 401 Unauthorized error, please log in again.

3. SSH key

   • To verify if you have a key associated:

   exc compute key list

   • If there are no keys create a new key using this guide and come back.

Create a Security Group with Rules

1. Create a Security Group

   exc securitygroup create --name ollama-security-group --description "Ollama ingress and egress"

2. Create Ingress Rules to Allow HTTP, SSH, Ollama Traffic

   exc securitygroup rule ingress create --cidr 0.0.0.0/0 --protocol TCPv4 --port_range 80,22,11434 --security_group_id $(exc securitygroup list | awk '/ollama-security-group/{print $1}')
   exc securitygroup rule ingress create --cidr ::/0 --protocol TCPv6 --port_range 80,22,11434 --security_group_id $(exc securitygroup list | awk '/ollama-security-group/{print $1}')

3. Create Egress Rules to Allow All Traffic

   exc securitygroup rule egress create --cidr 0.0.0.0/0 --protocol IPv4 --port_range ANY --security_group_id $(exc securitygroup list | awk '/ollama-security-group/{print $1}')
   exc securitygroup rule egress create --cidr ::/0 --protocol IPv6 --port_range ANY --security_group_id $(exc securitygroup list | awk '/ollama-security-group/{print $1}')

Creating a New VM

Use the following one-liner command to create a new m1a.4xlarge VM with the Ollama image. This command will automatically pass all the required parameters for a hassle-free experience.

exc compute create \
  --allocate_public_ipv4 \
  --name my-ollama-vm \
  --image_id 5 \
  --instance_type m1a.4xlarge \
  --ssh_pubkey excloud \
  --security_group_ids $(exc securitygroup list | awk '/ollama-security-group/{print $1}') \
  --subnet_id 1 \
  --wait

• Command Explanation:
  • The command uses awk to extract the necessary IDs for subnet, SSH key, and security group.
  • The --allocate_public_ipv4 flag is used to allocate a public IP address to your VM.

Access the OpenWebUI

• Once the VM is up and running, navigate to port 80 of the public IP address of your VM in your web browser. You should see the OpenWebUI interface, which provides an easy way to interact with the services provided by the Ollama image.

http://PUBLIC_IP:80