Skip to content

Commit

Permalink
Correct the kubernetes terminology (ollama#3843)
Browse files Browse the repository at this point in the history
* add details on kubernetes deployment and separate the testing process

* Update examples/kubernetes/README.md

thanks for suggesting this change, I agree with you and let's make this project better together !

Co-authored-by: JonZeolla <Zeolla@gmail.com>

---------

Co-authored-by: QIN Mélony <MQN1@dsone.3ds.com>
Co-authored-by: JonZeolla <Zeolla@gmail.com>
  • Loading branch information
3 people authored May 7, 2024
1 parent 88a6712 commit 3f71ba4
Showing 1 changed file with 14 additions and 12 deletions.
26 changes: 14 additions & 12 deletions examples/kubernetes/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,12 +7,24 @@

## Steps

1. Create the Ollama namespace, daemon set, and service
1. Create the Ollama namespace, deployment, and service

```bash
kubectl apply -f cpu.yaml
```

## (Optional) Hardware Acceleration

Hardware acceleration in Kubernetes requires NVIDIA's [`k8s-device-plugin`](https://github.com/NVIDIA/k8s-device-plugin) which is deployed in Kubernetes in form of daemonset. Follow the link for more details.

Once configured, create a GPU enabled Ollama deployment.

```bash
kubectl apply -f gpu.yaml
```

## Test

1. Port forward the Ollama service to connect and use it locally

```bash
Expand All @@ -23,14 +35,4 @@

```bash
ollama run orca-mini:3b
```

## (Optional) Hardware Acceleration

Hardware acceleration in Kubernetes requires NVIDIA's [`k8s-device-plugin`](https://github.com/NVIDIA/k8s-device-plugin). Follow the link for more details.

Once configured, create a GPU enabled Ollama deployment.

```bash
kubectl apply -f gpu.yaml
```
```

0 comments on commit 3f71ba4

Please sign in to comment.