diff --git a/docs/public/sitemap-0.xml b/docs/public/sitemap-0.xml index b74cbb2372..2b4a049754 100644 --- a/docs/public/sitemap-0.xml +++ b/docs/public/sitemap-0.xml @@ -93,7 +93,7 @@ https://jan.ai/docs/desktop/windows2024-09-09T08:19:45.722Zdaily1 https://jan.ai/docs/error-codes2024-09-09T08:19:45.722Zdaily1 https://jan.ai/docs/extensions2024-09-09T08:19:45.722Zdaily1 -https://jan.ai/docs/installing-extension2024-09-09T08:19:45.722Zdaily1 +https://jan.ai/docs/install-extensions2024-09-09T08:19:45.722Zdaily1 https://jan.ai/docs/models2024-09-09T08:19:45.722Zdaily1 https://jan.ai/docs/models/manage-models2024-09-09T08:19:45.722Zdaily1 https://jan.ai/docs/models/model-parameters2024-09-09T08:19:45.722Zdaily1 diff --git a/docs/src/pages/docs/_assets/Anthropic-1.gif b/docs/src/pages/docs/_assets/Anthropic-1.gif deleted file mode 100644 index 594c53172a..0000000000 Binary files a/docs/src/pages/docs/_assets/Anthropic-1.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/Anthropic-2.gif b/docs/src/pages/docs/_assets/Anthropic-2.gif deleted file mode 100644 index 63e76cc3bc..0000000000 Binary files a/docs/src/pages/docs/_assets/Anthropic-2.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/Cohere-1.gif b/docs/src/pages/docs/_assets/Cohere-1.gif deleted file mode 100644 index 6441bd8a93..0000000000 Binary files a/docs/src/pages/docs/_assets/Cohere-1.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/Cohere-2.gif b/docs/src/pages/docs/_assets/Cohere-2.gif deleted file mode 100644 index 9488040c01..0000000000 Binary files a/docs/src/pages/docs/_assets/Cohere-2.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/Groq-1.gif b/docs/src/pages/docs/_assets/Groq-1.gif deleted file mode 100644 index d95e577b14..0000000000 Binary files a/docs/src/pages/docs/_assets/Groq-1.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/Groq-2.gif b/docs/src/pages/docs/_assets/Groq-2.gif deleted file mode 100644 index 46231696f0..0000000000 Binary files a/docs/src/pages/docs/_assets/Groq-2.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/LM-Studio-v1.gif b/docs/src/pages/docs/_assets/LM-Studio-v1.gif deleted file mode 100644 index 65dfb5db1a..0000000000 Binary files a/docs/src/pages/docs/_assets/LM-Studio-v1.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/LM-Studio-v2.gif b/docs/src/pages/docs/_assets/LM-Studio-v2.gif deleted file mode 100644 index ad012df5d1..0000000000 Binary files a/docs/src/pages/docs/_assets/LM-Studio-v2.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/LM-Studio-v3.gif b/docs/src/pages/docs/_assets/LM-Studio-v3.gif deleted file mode 100644 index d5208c2bc5..0000000000 Binary files a/docs/src/pages/docs/_assets/LM-Studio-v3.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/Martian-1.gif b/docs/src/pages/docs/_assets/Martian-1.gif deleted file mode 100644 index 3da994cc4a..0000000000 Binary files a/docs/src/pages/docs/_assets/Martian-1.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/Martian-2.gif b/docs/src/pages/docs/_assets/Martian-2.gif deleted file mode 100644 index 27fa761b77..0000000000 Binary files a/docs/src/pages/docs/_assets/Martian-2.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/Mistral-1.gif b/docs/src/pages/docs/_assets/Mistral-1.gif deleted file mode 100644 index 7d1e821084..0000000000 Binary files a/docs/src/pages/docs/_assets/Mistral-1.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/Mistral-2.gif b/docs/src/pages/docs/_assets/Mistral-2.gif deleted file mode 100644 index 25c8d2fca4..0000000000 Binary files a/docs/src/pages/docs/_assets/Mistral-2.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/Ollama-1.gif b/docs/src/pages/docs/_assets/Ollama-1.gif deleted file mode 100644 index d9c7fa135d..0000000000 Binary files a/docs/src/pages/docs/_assets/Ollama-1.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/Ollama-2.gif b/docs/src/pages/docs/_assets/Ollama-2.gif deleted file mode 100644 index eadc6afdd6..0000000000 Binary files a/docs/src/pages/docs/_assets/Ollama-2.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/Ollama-3.gif b/docs/src/pages/docs/_assets/Ollama-3.gif deleted file mode 100644 index 6b09000b95..0000000000 Binary files a/docs/src/pages/docs/_assets/Ollama-3.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/OpenAi-1.gif b/docs/src/pages/docs/_assets/OpenAi-1.gif deleted file mode 100644 index c98e2f2be2..0000000000 Binary files a/docs/src/pages/docs/_assets/OpenAi-1.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/OpenAi-2.gif b/docs/src/pages/docs/_assets/OpenAi-2.gif deleted file mode 100644 index 8cd449f7fe..0000000000 Binary files a/docs/src/pages/docs/_assets/OpenAi-2.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/OpenRouter-1.gif b/docs/src/pages/docs/_assets/OpenRouter-1.gif deleted file mode 100644 index cd6a7898fa..0000000000 Binary files a/docs/src/pages/docs/_assets/OpenRouter-1.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/OpenRouter-2.gif b/docs/src/pages/docs/_assets/OpenRouter-2.gif deleted file mode 100644 index 307b07debc..0000000000 Binary files a/docs/src/pages/docs/_assets/OpenRouter-2.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/amd.gif b/docs/src/pages/docs/_assets/amd.gif deleted file mode 100644 index cc053b80cb..0000000000 Binary files a/docs/src/pages/docs/_assets/amd.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/anthropic.png b/docs/src/pages/docs/_assets/anthropic.png new file mode 100644 index 0000000000..1085769037 Binary files /dev/null and b/docs/src/pages/docs/_assets/anthropic.png differ diff --git a/docs/src/pages/docs/_assets/assistant-01.png b/docs/src/pages/docs/_assets/assistant-01.png index 5b35460324..eabf2bba27 100644 Binary files a/docs/src/pages/docs/_assets/assistant-01.png and b/docs/src/pages/docs/_assets/assistant-01.png differ diff --git a/docs/src/pages/docs/_assets/assistant-slider.png b/docs/src/pages/docs/_assets/assistant-slider.png deleted file mode 100644 index 8289e0d51b..0000000000 Binary files a/docs/src/pages/docs/_assets/assistant-slider.png and /dev/null differ diff --git a/docs/src/pages/docs/_assets/assistant1.gif b/docs/src/pages/docs/_assets/assistant1.gif deleted file mode 100644 index 75a56ba276..0000000000 Binary files a/docs/src/pages/docs/_assets/assistant1.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/assistant2.gif b/docs/src/pages/docs/_assets/assistant2.gif deleted file mode 100644 index 44bfb2264e..0000000000 Binary files a/docs/src/pages/docs/_assets/assistant2.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/chat.gif b/docs/src/pages/docs/_assets/chat.gif deleted file mode 100644 index 94b8c27193..0000000000 Binary files a/docs/src/pages/docs/_assets/chat.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/clear.gif b/docs/src/pages/docs/_assets/clear.gif deleted file mode 100644 index 445708d223..0000000000 Binary files a/docs/src/pages/docs/_assets/clear.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/cohere.png b/docs/src/pages/docs/_assets/cohere.png new file mode 100644 index 0000000000..87d3d0fa4f Binary files /dev/null and b/docs/src/pages/docs/_assets/cohere.png differ diff --git a/docs/src/pages/docs/_assets/configure.gif b/docs/src/pages/docs/_assets/configure.gif deleted file mode 100644 index 96daaaa616..0000000000 Binary files a/docs/src/pages/docs/_assets/configure.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/data-folder.png b/docs/src/pages/docs/_assets/data-folder.png deleted file mode 100644 index def0f38c7b..0000000000 Binary files a/docs/src/pages/docs/_assets/data-folder.png and /dev/null differ diff --git a/docs/src/pages/docs/_assets/default.gif b/docs/src/pages/docs/_assets/default.gif deleted file mode 100644 index 3bc6a68f67..0000000000 Binary files a/docs/src/pages/docs/_assets/default.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/delete-tensor.gif b/docs/src/pages/docs/_assets/delete-tensor.gif deleted file mode 100644 index 86f45fc65a..0000000000 Binary files a/docs/src/pages/docs/_assets/delete-tensor.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/delete-threads.png b/docs/src/pages/docs/_assets/delete-threads.png deleted file mode 100644 index b134e8e124..0000000000 Binary files a/docs/src/pages/docs/_assets/delete-threads.png and /dev/null differ diff --git a/docs/src/pages/docs/_assets/delete.png b/docs/src/pages/docs/_assets/delete.png deleted file mode 100644 index 6b3f669c2b..0000000000 Binary files a/docs/src/pages/docs/_assets/delete.png and /dev/null differ diff --git a/docs/src/pages/docs/_assets/disable-tensor.gif b/docs/src/pages/docs/_assets/disable-tensor.gif deleted file mode 100644 index 91e9fe3d62..0000000000 Binary files a/docs/src/pages/docs/_assets/disable-tensor.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/download-button.png b/docs/src/pages/docs/_assets/download-button.png deleted file mode 100644 index 2e8088d159..0000000000 Binary files a/docs/src/pages/docs/_assets/download-button.png and /dev/null differ diff --git a/docs/src/pages/docs/_assets/download-button2.png b/docs/src/pages/docs/_assets/download-button2.png deleted file mode 100644 index 8f8c683c39..0000000000 Binary files a/docs/src/pages/docs/_assets/download-button2.png and /dev/null differ diff --git a/docs/src/pages/docs/_assets/download-button3.png b/docs/src/pages/docs/_assets/download-button3.png deleted file mode 100644 index cf05ec45aa..0000000000 Binary files a/docs/src/pages/docs/_assets/download-button3.png and /dev/null differ diff --git a/docs/src/pages/docs/_assets/download-icon.png b/docs/src/pages/docs/_assets/download-icon.png deleted file mode 100644 index f553e19f27..0000000000 Binary files a/docs/src/pages/docs/_assets/download-icon.png and /dev/null differ diff --git a/docs/src/pages/docs/_assets/download-llama2.gif b/docs/src/pages/docs/_assets/download-llama2.gif deleted file mode 100644 index 18d671209d..0000000000 Binary files a/docs/src/pages/docs/_assets/download-llama2.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/download-model.gif b/docs/src/pages/docs/_assets/download-model.gif deleted file mode 100644 index e10b293c26..0000000000 Binary files a/docs/src/pages/docs/_assets/download-model.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/download-model2.gif b/docs/src/pages/docs/_assets/download-model2.gif deleted file mode 100644 index c69819a1ec..0000000000 Binary files a/docs/src/pages/docs/_assets/download-model2.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/download-phi.gif b/docs/src/pages/docs/_assets/download-phi.gif deleted file mode 100644 index 5b2cd6034b..0000000000 Binary files a/docs/src/pages/docs/_assets/download-phi.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/download.gif b/docs/src/pages/docs/_assets/download.gif deleted file mode 100644 index 10cb4b4669..0000000000 Binary files a/docs/src/pages/docs/_assets/download.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/download.png b/docs/src/pages/docs/_assets/download.png deleted file mode 100644 index 571e44dd30..0000000000 Binary files a/docs/src/pages/docs/_assets/download.png and /dev/null differ diff --git a/docs/src/pages/docs/_assets/exp-mode.png b/docs/src/pages/docs/_assets/exp-mode.png deleted file mode 100644 index 14ecf6bfe7..0000000000 Binary files a/docs/src/pages/docs/_assets/exp-mode.png and /dev/null differ diff --git a/docs/src/pages/docs/_assets/experimental.gif b/docs/src/pages/docs/_assets/experimental.gif deleted file mode 100644 index 11c2d86225..0000000000 Binary files a/docs/src/pages/docs/_assets/experimental.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/extensions-01.png b/docs/src/pages/docs/_assets/extensions-01.png new file mode 100644 index 0000000000..912943f836 Binary files /dev/null and b/docs/src/pages/docs/_assets/extensions-01.png differ diff --git a/docs/src/pages/docs/_assets/extensions-02.png b/docs/src/pages/docs/_assets/extensions-02.png new file mode 100644 index 0000000000..e2e582b37d Binary files /dev/null and b/docs/src/pages/docs/_assets/extensions-02.png differ diff --git a/docs/src/pages/docs/_assets/extensions-03.png b/docs/src/pages/docs/_assets/extensions-03.png new file mode 100644 index 0000000000..bbdd2c3712 Binary files /dev/null and b/docs/src/pages/docs/_assets/extensions-03.png differ diff --git a/docs/src/pages/docs/_assets/extensions-page.png b/docs/src/pages/docs/_assets/extensions-page.png deleted file mode 100644 index 74a336769c..0000000000 Binary files a/docs/src/pages/docs/_assets/extensions-page.png and /dev/null differ diff --git a/docs/src/pages/docs/_assets/extensions-page2.png b/docs/src/pages/docs/_assets/extensions-page2.png deleted file mode 100644 index 51863aa99b..0000000000 Binary files a/docs/src/pages/docs/_assets/extensions-page2.png and /dev/null differ diff --git a/docs/src/pages/docs/_assets/gpt.gif b/docs/src/pages/docs/_assets/gpt.gif deleted file mode 100644 index 3972e812f4..0000000000 Binary files a/docs/src/pages/docs/_assets/gpt.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/gpu-accel.png b/docs/src/pages/docs/_assets/gpu-accel.png deleted file mode 100644 index 294013fcf1..0000000000 Binary files a/docs/src/pages/docs/_assets/gpu-accel.png and /dev/null differ diff --git a/docs/src/pages/docs/_assets/gpu.gif b/docs/src/pages/docs/_assets/gpu.gif deleted file mode 100644 index 58d1da9704..0000000000 Binary files a/docs/src/pages/docs/_assets/gpu.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/gpu2.gif b/docs/src/pages/docs/_assets/gpu2.gif deleted file mode 100644 index fcacc5e99c..0000000000 Binary files a/docs/src/pages/docs/_assets/gpu2.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/groq.png b/docs/src/pages/docs/_assets/groq.png new file mode 100644 index 0000000000..d9927fe7a9 Binary files /dev/null and b/docs/src/pages/docs/_assets/groq.png differ diff --git a/docs/src/pages/docs/_assets/history.png b/docs/src/pages/docs/_assets/history.png deleted file mode 100644 index 7112a0d242..0000000000 Binary files a/docs/src/pages/docs/_assets/history.png and /dev/null differ diff --git a/docs/src/pages/docs/_assets/http.png b/docs/src/pages/docs/_assets/http.png deleted file mode 100644 index afa0aeae02..0000000000 Binary files a/docs/src/pages/docs/_assets/http.png and /dev/null differ diff --git a/docs/src/pages/docs/_assets/https.gif b/docs/src/pages/docs/_assets/https.gif deleted file mode 100644 index e9ed05b855..0000000000 Binary files a/docs/src/pages/docs/_assets/https.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/hub.gif b/docs/src/pages/docs/_assets/hub.gif deleted file mode 100644 index fad29d5db6..0000000000 Binary files a/docs/src/pages/docs/_assets/hub.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/hub.png b/docs/src/pages/docs/_assets/hub.png deleted file mode 100644 index 75b0f575d1..0000000000 Binary files a/docs/src/pages/docs/_assets/hub.png and /dev/null differ diff --git a/docs/src/pages/docs/_assets/import-folder.gif b/docs/src/pages/docs/_assets/import-folder.gif deleted file mode 100644 index fc78222070..0000000000 Binary files a/docs/src/pages/docs/_assets/import-folder.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/import-hf.gif b/docs/src/pages/docs/_assets/import-hf.gif deleted file mode 100644 index 4ca1a8b2aa..0000000000 Binary files a/docs/src/pages/docs/_assets/import-hf.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/import.png b/docs/src/pages/docs/_assets/import.png deleted file mode 100644 index 5ca7fff424..0000000000 Binary files a/docs/src/pages/docs/_assets/import.png and /dev/null differ diff --git a/docs/src/pages/docs/_assets/import2.png b/docs/src/pages/docs/_assets/import2.png deleted file mode 100644 index caf2c022db..0000000000 Binary files a/docs/src/pages/docs/_assets/import2.png and /dev/null differ diff --git a/docs/src/pages/docs/_assets/inf.gif b/docs/src/pages/docs/_assets/inf.gif deleted file mode 100644 index aaef44380f..0000000000 Binary files a/docs/src/pages/docs/_assets/inf.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/install-engines-01.png b/docs/src/pages/docs/_assets/install-engines-01.png new file mode 100644 index 0000000000..85db02b980 Binary files /dev/null and b/docs/src/pages/docs/_assets/install-engines-01.png differ diff --git a/docs/src/pages/docs/_assets/install-engines-02.png b/docs/src/pages/docs/_assets/install-engines-02.png new file mode 100644 index 0000000000..dcdd7e4dfd Binary files /dev/null and b/docs/src/pages/docs/_assets/install-engines-02.png differ diff --git a/docs/src/pages/docs/_assets/install-engines-03.png b/docs/src/pages/docs/_assets/install-engines-03.png new file mode 100644 index 0000000000..cb2ecde56b Binary files /dev/null and b/docs/src/pages/docs/_assets/install-engines-03.png differ diff --git a/docs/src/pages/docs/_assets/install-ext.gif b/docs/src/pages/docs/_assets/install-ext.gif deleted file mode 100644 index 4357ed2365..0000000000 Binary files a/docs/src/pages/docs/_assets/install-ext.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/install-ext.png b/docs/src/pages/docs/_assets/install-ext.png deleted file mode 100644 index ee2ff8453c..0000000000 Binary files a/docs/src/pages/docs/_assets/install-ext.png and /dev/null differ diff --git a/docs/src/pages/docs/_assets/install-tensor.gif b/docs/src/pages/docs/_assets/install-tensor.gif deleted file mode 100644 index 927ee0b9b9..0000000000 Binary files a/docs/src/pages/docs/_assets/install-tensor.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/install-tensor.png b/docs/src/pages/docs/_assets/install-tensor.png deleted file mode 100644 index 050075d844..0000000000 Binary files a/docs/src/pages/docs/_assets/install-tensor.png and /dev/null differ diff --git a/docs/src/pages/docs/_assets/linux.png b/docs/src/pages/docs/_assets/linux.png deleted file mode 100644 index 72230191ca..0000000000 Binary files a/docs/src/pages/docs/_assets/linux.png and /dev/null differ diff --git a/docs/src/pages/docs/_assets/llama.cpp-01.png b/docs/src/pages/docs/_assets/llama.cpp-01.png new file mode 100644 index 0000000000..b794fe1df0 Binary files /dev/null and b/docs/src/pages/docs/_assets/llama.cpp-01.png differ diff --git a/docs/src/pages/docs/_assets/llama2.gif b/docs/src/pages/docs/_assets/llama2.gif deleted file mode 100644 index 0c9502aa6e..0000000000 Binary files a/docs/src/pages/docs/_assets/llama2.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/local-api1.png b/docs/src/pages/docs/_assets/local-api1.png deleted file mode 100644 index d636d37ad9..0000000000 Binary files a/docs/src/pages/docs/_assets/local-api1.png and /dev/null differ diff --git a/docs/src/pages/docs/_assets/local-api2.png b/docs/src/pages/docs/_assets/local-api2.png deleted file mode 100644 index 58652775c9..0000000000 Binary files a/docs/src/pages/docs/_assets/local-api2.png and /dev/null differ diff --git a/docs/src/pages/docs/_assets/local-api3.png b/docs/src/pages/docs/_assets/local-api3.png deleted file mode 100644 index aa83726400..0000000000 Binary files a/docs/src/pages/docs/_assets/local-api3.png and /dev/null differ diff --git a/docs/src/pages/docs/_assets/local-api4.png b/docs/src/pages/docs/_assets/local-api4.png deleted file mode 100644 index 2a66c9c1f5..0000000000 Binary files a/docs/src/pages/docs/_assets/local-api4.png and /dev/null differ diff --git a/docs/src/pages/docs/_assets/local-api5.png b/docs/src/pages/docs/_assets/local-api5.png deleted file mode 100644 index 2dc80c6c70..0000000000 Binary files a/docs/src/pages/docs/_assets/local-api5.png and /dev/null differ diff --git a/docs/src/pages/docs/_assets/mac2.png b/docs/src/pages/docs/_assets/mac2.png deleted file mode 100644 index aeb4b6bbcc..0000000000 Binary files a/docs/src/pages/docs/_assets/mac2.png and /dev/null differ diff --git a/docs/src/pages/docs/_assets/martian.png b/docs/src/pages/docs/_assets/martian.png new file mode 100644 index 0000000000..e00349b015 Binary files /dev/null and b/docs/src/pages/docs/_assets/martian.png differ diff --git a/docs/src/pages/docs/_assets/mistralai.png b/docs/src/pages/docs/_assets/mistralai.png new file mode 100644 index 0000000000..70139c6aef Binary files /dev/null and b/docs/src/pages/docs/_assets/mistralai.png differ diff --git a/docs/src/pages/docs/_assets/model-anthropic.gif b/docs/src/pages/docs/_assets/model-anthropic.gif deleted file mode 100644 index 8de07e6258..0000000000 Binary files a/docs/src/pages/docs/_assets/model-anthropic.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/model-cohere.gif b/docs/src/pages/docs/_assets/model-cohere.gif deleted file mode 100644 index 2bf5007ff7..0000000000 Binary files a/docs/src/pages/docs/_assets/model-cohere.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/model-groq.gif b/docs/src/pages/docs/_assets/model-groq.gif deleted file mode 100644 index 7c6d202807..0000000000 Binary files a/docs/src/pages/docs/_assets/model-groq.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/model-management-01.png b/docs/src/pages/docs/_assets/model-management-01.png index ea6c363e1b..9020c66b29 100644 Binary files a/docs/src/pages/docs/_assets/model-management-01.png and b/docs/src/pages/docs/_assets/model-management-01.png differ diff --git a/docs/src/pages/docs/_assets/model-management-02.png b/docs/src/pages/docs/_assets/model-management-02.png index a0fe086469..bb446d0569 100644 Binary files a/docs/src/pages/docs/_assets/model-management-02.png and b/docs/src/pages/docs/_assets/model-management-02.png differ diff --git a/docs/src/pages/docs/_assets/model-management-04.png b/docs/src/pages/docs/_assets/model-management-04.png index 6b3f824640..71d72dff26 100644 Binary files a/docs/src/pages/docs/_assets/model-management-04.png and b/docs/src/pages/docs/_assets/model-management-04.png differ diff --git a/docs/src/pages/docs/_assets/model-management-05.png b/docs/src/pages/docs/_assets/model-management-05.png index e7d4ea38d7..891fa943ae 100644 Binary files a/docs/src/pages/docs/_assets/model-management-05.png and b/docs/src/pages/docs/_assets/model-management-05.png differ diff --git a/docs/src/pages/docs/_assets/model-management-06.png b/docs/src/pages/docs/_assets/model-management-06.png index 5d5449c8be..ebd2de3d14 100644 Binary files a/docs/src/pages/docs/_assets/model-management-06.png and b/docs/src/pages/docs/_assets/model-management-06.png differ diff --git a/docs/src/pages/docs/_assets/model-management-07.png b/docs/src/pages/docs/_assets/model-management-07.png index 83a562ddb6..612904ac72 100644 Binary files a/docs/src/pages/docs/_assets/model-management-07.png and b/docs/src/pages/docs/_assets/model-management-07.png differ diff --git a/docs/src/pages/docs/_assets/model-martian.gif b/docs/src/pages/docs/_assets/model-martian.gif deleted file mode 100644 index 8bf4dbbf5d..0000000000 Binary files a/docs/src/pages/docs/_assets/model-martian.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/model-mistral.gif b/docs/src/pages/docs/_assets/model-mistral.gif deleted file mode 100644 index a266619032..0000000000 Binary files a/docs/src/pages/docs/_assets/model-mistral.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/model-openai.gif b/docs/src/pages/docs/_assets/model-openai.gif deleted file mode 100644 index b20aa9c0e9..0000000000 Binary files a/docs/src/pages/docs/_assets/model-openai.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/model-parameters.png b/docs/src/pages/docs/_assets/model-parameters.png index beb26972bd..a99deb75e0 100644 Binary files a/docs/src/pages/docs/_assets/model-parameters.png and b/docs/src/pages/docs/_assets/model-parameters.png differ diff --git a/docs/src/pages/docs/_assets/model-tab.png b/docs/src/pages/docs/_assets/model-tab.png deleted file mode 100644 index ea595bf197..0000000000 Binary files a/docs/src/pages/docs/_assets/model-tab.png and /dev/null differ diff --git a/docs/src/pages/docs/_assets/model.gif b/docs/src/pages/docs/_assets/model.gif deleted file mode 100644 index e5abdf757e..0000000000 Binary files a/docs/src/pages/docs/_assets/model.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/modelparam.gif b/docs/src/pages/docs/_assets/modelparam.gif deleted file mode 100644 index f33cf5ab5b..0000000000 Binary files a/docs/src/pages/docs/_assets/modelparam.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/models.gif b/docs/src/pages/docs/_assets/models.gif deleted file mode 100644 index 8dc4d7925f..0000000000 Binary files a/docs/src/pages/docs/_assets/models.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/mymodels.png b/docs/src/pages/docs/_assets/mymodels.png deleted file mode 100644 index d12664b86e..0000000000 Binary files a/docs/src/pages/docs/_assets/mymodels.png and /dev/null differ diff --git a/docs/src/pages/docs/_assets/nitro-network.png b/docs/src/pages/docs/_assets/nitro-network.png deleted file mode 100644 index 5f1f7104eb..0000000000 Binary files a/docs/src/pages/docs/_assets/nitro-network.png and /dev/null differ diff --git a/docs/src/pages/docs/_assets/nvidia-1.gif b/docs/src/pages/docs/_assets/nvidia-1.gif deleted file mode 100644 index df28abc1fd..0000000000 Binary files a/docs/src/pages/docs/_assets/nvidia-1.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/nvidia-2.gif b/docs/src/pages/docs/_assets/nvidia-2.gif deleted file mode 100644 index 4c628884af..0000000000 Binary files a/docs/src/pages/docs/_assets/nvidia-2.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/nvidia-nim.png b/docs/src/pages/docs/_assets/nvidia-nim.png new file mode 100644 index 0000000000..8eb637c126 Binary files /dev/null and b/docs/src/pages/docs/_assets/nvidia-nim.png differ diff --git a/docs/src/pages/docs/_assets/openai.png b/docs/src/pages/docs/_assets/openai.png new file mode 100644 index 0000000000..e6407d9eda Binary files /dev/null and b/docs/src/pages/docs/_assets/openai.png differ diff --git a/docs/src/pages/docs/_assets/openrouter.png b/docs/src/pages/docs/_assets/openrouter.png new file mode 100644 index 0000000000..d9e0327004 Binary files /dev/null and b/docs/src/pages/docs/_assets/openrouter.png differ diff --git a/docs/src/pages/docs/_assets/phi.gif b/docs/src/pages/docs/_assets/phi.gif deleted file mode 100644 index e8cd587d28..0000000000 Binary files a/docs/src/pages/docs/_assets/phi.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/preserve.png b/docs/src/pages/docs/_assets/preserve.png deleted file mode 100644 index 459c940e26..0000000000 Binary files a/docs/src/pages/docs/_assets/preserve.png and /dev/null differ diff --git a/docs/src/pages/docs/_assets/quick-ask.gif b/docs/src/pages/docs/_assets/quick-ask.gif deleted file mode 100644 index f4081ca3c3..0000000000 Binary files a/docs/src/pages/docs/_assets/quick-ask.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/quick-ask.png b/docs/src/pages/docs/_assets/quick-ask.png deleted file mode 100644 index 34620569bb..0000000000 Binary files a/docs/src/pages/docs/_assets/quick-ask.png and /dev/null differ diff --git a/docs/src/pages/docs/_assets/quick-start-01.png b/docs/src/pages/docs/_assets/quick-start-01.png index e5aaeb88be..430b01fb0a 100644 Binary files a/docs/src/pages/docs/_assets/quick-start-01.png and b/docs/src/pages/docs/_assets/quick-start-01.png differ diff --git a/docs/src/pages/docs/_assets/quick-start-02.png b/docs/src/pages/docs/_assets/quick-start-02.png index f1ecd7ee57..998e04dc8a 100644 Binary files a/docs/src/pages/docs/_assets/quick-start-02.png and b/docs/src/pages/docs/_assets/quick-start-02.png differ diff --git a/docs/src/pages/docs/_assets/quick-start-03.png b/docs/src/pages/docs/_assets/quick-start-03.png index fe249f1945..51d6de0d2c 100644 Binary files a/docs/src/pages/docs/_assets/quick-start-03.png and b/docs/src/pages/docs/_assets/quick-start-03.png differ diff --git a/docs/src/pages/docs/_assets/reset-jan.png b/docs/src/pages/docs/_assets/reset-jan.png deleted file mode 100644 index 2c20ca6674..0000000000 Binary files a/docs/src/pages/docs/_assets/reset-jan.png and /dev/null differ diff --git a/docs/src/pages/docs/_assets/reset.gif b/docs/src/pages/docs/_assets/reset.gif deleted file mode 100644 index 6ff171f259..0000000000 Binary files a/docs/src/pages/docs/_assets/reset.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/retrieval-01.png b/docs/src/pages/docs/_assets/retrieval-01.png index 12c8d832bd..b29b964b5d 100644 Binary files a/docs/src/pages/docs/_assets/retrieval-01.png and b/docs/src/pages/docs/_assets/retrieval-01.png differ diff --git a/docs/src/pages/docs/_assets/retrieval-02.png b/docs/src/pages/docs/_assets/retrieval-02.png index 6c57e55a22..37fc8bf990 100644 Binary files a/docs/src/pages/docs/_assets/retrieval-02.png and b/docs/src/pages/docs/_assets/retrieval-02.png differ diff --git a/docs/src/pages/docs/_assets/retrieval.gif b/docs/src/pages/docs/_assets/retrieval.gif deleted file mode 100644 index 38d3fbb581..0000000000 Binary files a/docs/src/pages/docs/_assets/retrieval.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/retrieval1.png b/docs/src/pages/docs/_assets/retrieval1.png deleted file mode 100644 index 8b5d63bb89..0000000000 Binary files a/docs/src/pages/docs/_assets/retrieval1.png and /dev/null differ diff --git a/docs/src/pages/docs/_assets/retrieval2.png b/docs/src/pages/docs/_assets/retrieval2.png deleted file mode 100644 index 2c9443c444..0000000000 Binary files a/docs/src/pages/docs/_assets/retrieval2.png and /dev/null differ diff --git a/docs/src/pages/docs/_assets/scheme.png b/docs/src/pages/docs/_assets/scheme.png deleted file mode 100644 index fe50d4b6a2..0000000000 Binary files a/docs/src/pages/docs/_assets/scheme.png and /dev/null differ diff --git a/docs/src/pages/docs/_assets/search-bar.png b/docs/src/pages/docs/_assets/search-bar.png deleted file mode 100644 index b6e5597734..0000000000 Binary files a/docs/src/pages/docs/_assets/search-bar.png and /dev/null differ diff --git a/docs/src/pages/docs/_assets/select-openai.gif b/docs/src/pages/docs/_assets/select-openai.gif deleted file mode 100644 index 3196c4320d..0000000000 Binary files a/docs/src/pages/docs/_assets/select-openai.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/server-anthropic.gif b/docs/src/pages/docs/_assets/server-anthropic.gif deleted file mode 100644 index da684dbd46..0000000000 Binary files a/docs/src/pages/docs/_assets/server-anthropic.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/server-cohere.gif b/docs/src/pages/docs/_assets/server-cohere.gif deleted file mode 100644 index 72c9bd883e..0000000000 Binary files a/docs/src/pages/docs/_assets/server-cohere.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/server-groq.gif b/docs/src/pages/docs/_assets/server-groq.gif deleted file mode 100644 index dcb3e03f7f..0000000000 Binary files a/docs/src/pages/docs/_assets/server-groq.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/server-llama2.gif b/docs/src/pages/docs/_assets/server-llama2.gif deleted file mode 100644 index b1ebafc849..0000000000 Binary files a/docs/src/pages/docs/_assets/server-llama2.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/server-martian.gif b/docs/src/pages/docs/_assets/server-martian.gif deleted file mode 100644 index a33dd7f198..0000000000 Binary files a/docs/src/pages/docs/_assets/server-martian.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/server-mistral.gif b/docs/src/pages/docs/_assets/server-mistral.gif deleted file mode 100644 index 8fdb6ef0f9..0000000000 Binary files a/docs/src/pages/docs/_assets/server-mistral.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/server-openai.gif b/docs/src/pages/docs/_assets/server-openai.gif deleted file mode 100644 index fd10aa75b2..0000000000 Binary files a/docs/src/pages/docs/_assets/server-openai.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/server-openai2.gif b/docs/src/pages/docs/_assets/server-openai2.gif deleted file mode 100644 index b4338d1221..0000000000 Binary files a/docs/src/pages/docs/_assets/server-openai2.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/server-phi.gif b/docs/src/pages/docs/_assets/server-phi.gif deleted file mode 100644 index 4abc916c12..0000000000 Binary files a/docs/src/pages/docs/_assets/server-phi.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/set-model.gif b/docs/src/pages/docs/_assets/set-model.gif deleted file mode 100644 index 87de857322..0000000000 Binary files a/docs/src/pages/docs/_assets/set-model.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/set-tensor.gif b/docs/src/pages/docs/_assets/set-tensor.gif deleted file mode 100644 index 22e8248825..0000000000 Binary files a/docs/src/pages/docs/_assets/set-tensor.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/set-url.gif b/docs/src/pages/docs/_assets/set-url.gif deleted file mode 100644 index b675b3ea45..0000000000 Binary files a/docs/src/pages/docs/_assets/set-url.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/settings-01.png b/docs/src/pages/docs/_assets/settings-01.png index 8a2b734d74..7a8539d655 100644 Binary files a/docs/src/pages/docs/_assets/settings-01.png and b/docs/src/pages/docs/_assets/settings-01.png differ diff --git a/docs/src/pages/docs/_assets/settings-02.png b/docs/src/pages/docs/_assets/settings-02.png index f54d6bd44d..b00924d5aa 100644 Binary files a/docs/src/pages/docs/_assets/settings-02.png and b/docs/src/pages/docs/_assets/settings-02.png differ diff --git a/docs/src/pages/docs/_assets/settings-03.png b/docs/src/pages/docs/_assets/settings-03.png index f149ebb01c..aced464bed 100644 Binary files a/docs/src/pages/docs/_assets/settings-03.png and b/docs/src/pages/docs/_assets/settings-03.png differ diff --git a/docs/src/pages/docs/_assets/settings-04.png b/docs/src/pages/docs/_assets/settings-04.png index cb55f4cd0a..36dc64fce5 100644 Binary files a/docs/src/pages/docs/_assets/settings-04.png and b/docs/src/pages/docs/_assets/settings-04.png differ diff --git a/docs/src/pages/docs/_assets/settings-05.png b/docs/src/pages/docs/_assets/settings-05.png index 37aea156ab..8fb76e0fc5 100644 Binary files a/docs/src/pages/docs/_assets/settings-05.png and b/docs/src/pages/docs/_assets/settings-05.png differ diff --git a/docs/src/pages/docs/_assets/settings-06.png b/docs/src/pages/docs/_assets/settings-06.png index a25688f1e3..3a582b42a3 100644 Binary files a/docs/src/pages/docs/_assets/settings-06.png and b/docs/src/pages/docs/_assets/settings-06.png differ diff --git a/docs/src/pages/docs/_assets/settings-07.png b/docs/src/pages/docs/_assets/settings-07.png index 620ca20781..878c4ea669 100644 Binary files a/docs/src/pages/docs/_assets/settings-07.png and b/docs/src/pages/docs/_assets/settings-07.png differ diff --git a/docs/src/pages/docs/_assets/settings-08.png b/docs/src/pages/docs/_assets/settings-08.png index 9597831e03..a1c7d7dd1d 100644 Binary files a/docs/src/pages/docs/_assets/settings-08.png and b/docs/src/pages/docs/_assets/settings-08.png differ diff --git a/docs/src/pages/docs/_assets/settings-09.png b/docs/src/pages/docs/_assets/settings-09.png index 58ce63cba5..8d5e577e1e 100644 Binary files a/docs/src/pages/docs/_assets/settings-09.png and b/docs/src/pages/docs/_assets/settings-09.png differ diff --git a/docs/src/pages/docs/_assets/settings-10.png b/docs/src/pages/docs/_assets/settings-10.png index c72aa7c0bf..b89c9dacef 100644 Binary files a/docs/src/pages/docs/_assets/settings-10.png and b/docs/src/pages/docs/_assets/settings-10.png differ diff --git a/docs/src/pages/docs/_assets/settings-11.png b/docs/src/pages/docs/_assets/settings-11.png index 833386ef4e..b304f9f3b9 100644 Binary files a/docs/src/pages/docs/_assets/settings-11.png and b/docs/src/pages/docs/_assets/settings-11.png differ diff --git a/docs/src/pages/docs/_assets/settings-12.png b/docs/src/pages/docs/_assets/settings-12.png index a41744fa1e..a4d45dedd0 100644 Binary files a/docs/src/pages/docs/_assets/settings-12.png and b/docs/src/pages/docs/_assets/settings-12.png differ diff --git a/docs/src/pages/docs/_assets/settings-13.png b/docs/src/pages/docs/_assets/settings-13.png index a1111d8e28..f2b41ba3f2 100644 Binary files a/docs/src/pages/docs/_assets/settings-13.png and b/docs/src/pages/docs/_assets/settings-13.png differ diff --git a/docs/src/pages/docs/_assets/settings-14.png b/docs/src/pages/docs/_assets/settings-14.png index 049f0070a5..b253cb0dc2 100644 Binary files a/docs/src/pages/docs/_assets/settings-14.png and b/docs/src/pages/docs/_assets/settings-14.png differ diff --git a/docs/src/pages/docs/_assets/settings-15.png b/docs/src/pages/docs/_assets/settings-15.png index 6292523fa1..f739f85130 100644 Binary files a/docs/src/pages/docs/_assets/settings-15.png and b/docs/src/pages/docs/_assets/settings-15.png differ diff --git a/docs/src/pages/docs/_assets/settings-17.png b/docs/src/pages/docs/_assets/settings-17.png index c8e74183bb..83c10fa952 100644 Binary files a/docs/src/pages/docs/_assets/settings-17.png and b/docs/src/pages/docs/_assets/settings-17.png differ diff --git a/docs/src/pages/docs/_assets/settings-18.png b/docs/src/pages/docs/_assets/settings-18.png index fbc081e85a..63718d0efc 100644 Binary files a/docs/src/pages/docs/_assets/settings-18.png and b/docs/src/pages/docs/_assets/settings-18.png differ diff --git a/docs/src/pages/docs/_assets/settings.png b/docs/src/pages/docs/_assets/settings.png deleted file mode 100644 index e53f71d1ec..0000000000 Binary files a/docs/src/pages/docs/_assets/settings.png and /dev/null differ diff --git a/docs/src/pages/docs/_assets/shortcut.png b/docs/src/pages/docs/_assets/shortcut.png deleted file mode 100644 index b7912f8387..0000000000 Binary files a/docs/src/pages/docs/_assets/shortcut.png and /dev/null differ diff --git a/docs/src/pages/docs/_assets/specific-model.gif b/docs/src/pages/docs/_assets/specific-model.gif deleted file mode 100644 index 2664bcc479..0000000000 Binary files a/docs/src/pages/docs/_assets/specific-model.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/spell.png b/docs/src/pages/docs/_assets/spell.png deleted file mode 100644 index 92e9a6d8c8..0000000000 Binary files a/docs/src/pages/docs/_assets/spell.png and /dev/null differ diff --git a/docs/src/pages/docs/_assets/ssl.gif b/docs/src/pages/docs/_assets/ssl.gif deleted file mode 100644 index b18f075c1d..0000000000 Binary files a/docs/src/pages/docs/_assets/ssl.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/ssl.png b/docs/src/pages/docs/_assets/ssl.png deleted file mode 100644 index 970051744b..0000000000 Binary files a/docs/src/pages/docs/_assets/ssl.png and /dev/null differ diff --git a/docs/src/pages/docs/_assets/start-anthropic.gif b/docs/src/pages/docs/_assets/start-anthropic.gif deleted file mode 100644 index 9ca9386553..0000000000 Binary files a/docs/src/pages/docs/_assets/start-anthropic.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/start-cohere.gif b/docs/src/pages/docs/_assets/start-cohere.gif deleted file mode 100644 index b28fde4984..0000000000 Binary files a/docs/src/pages/docs/_assets/start-cohere.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/start-groq.gif b/docs/src/pages/docs/_assets/start-groq.gif deleted file mode 100644 index 8e5ced12a1..0000000000 Binary files a/docs/src/pages/docs/_assets/start-groq.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/start-martian.gif b/docs/src/pages/docs/_assets/start-martian.gif deleted file mode 100644 index 31a79d16dd..0000000000 Binary files a/docs/src/pages/docs/_assets/start-martian.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/start-mistral.gif b/docs/src/pages/docs/_assets/start-mistral.gif deleted file mode 100644 index 1012a651a8..0000000000 Binary files a/docs/src/pages/docs/_assets/start-mistral.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/start-router.gif b/docs/src/pages/docs/_assets/start-router.gif deleted file mode 100644 index ca1737de03..0000000000 Binary files a/docs/src/pages/docs/_assets/start-router.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/system-mili.png b/docs/src/pages/docs/_assets/system-mili.png deleted file mode 100644 index 0a0f28592d..0000000000 Binary files a/docs/src/pages/docs/_assets/system-mili.png and /dev/null differ diff --git a/docs/src/pages/docs/_assets/system-mili2.png b/docs/src/pages/docs/_assets/system-mili2.png deleted file mode 100644 index 6135c5a499..0000000000 Binary files a/docs/src/pages/docs/_assets/system-mili2.png and /dev/null differ diff --git a/docs/src/pages/docs/_assets/system-monitor.png b/docs/src/pages/docs/_assets/system-monitor.png deleted file mode 100644 index 04d6749cb5..0000000000 Binary files a/docs/src/pages/docs/_assets/system-monitor.png and /dev/null differ diff --git a/docs/src/pages/docs/_assets/system-monitor2.png b/docs/src/pages/docs/_assets/system-monitor2.png deleted file mode 100644 index e03dfbf63b..0000000000 Binary files a/docs/src/pages/docs/_assets/system-monitor2.png and /dev/null differ diff --git a/docs/src/pages/docs/_assets/system-slider.png b/docs/src/pages/docs/_assets/system-slider.png deleted file mode 100644 index 409338f703..0000000000 Binary files a/docs/src/pages/docs/_assets/system-slider.png and /dev/null differ diff --git a/docs/src/pages/docs/_assets/system-slider2.png b/docs/src/pages/docs/_assets/system-slider2.png deleted file mode 100644 index 9114d32c32..0000000000 Binary files a/docs/src/pages/docs/_assets/system-slider2.png and /dev/null differ diff --git a/docs/src/pages/docs/_assets/tensor.png b/docs/src/pages/docs/_assets/tensor.png deleted file mode 100644 index 812640de34..0000000000 Binary files a/docs/src/pages/docs/_assets/tensor.png and /dev/null differ diff --git a/docs/src/pages/docs/_assets/theme.png b/docs/src/pages/docs/_assets/theme.png deleted file mode 100644 index ae14f20546..0000000000 Binary files a/docs/src/pages/docs/_assets/theme.png and /dev/null differ diff --git a/docs/src/pages/docs/_assets/thread-settings.png b/docs/src/pages/docs/_assets/thread-settings.png deleted file mode 100644 index a7cc60304c..0000000000 Binary files a/docs/src/pages/docs/_assets/thread-settings.png and /dev/null differ diff --git a/docs/src/pages/docs/_assets/thread-settings2.png b/docs/src/pages/docs/_assets/thread-settings2.png deleted file mode 100644 index b0d728d43c..0000000000 Binary files a/docs/src/pages/docs/_assets/thread-settings2.png and /dev/null differ diff --git a/docs/src/pages/docs/_assets/threads-01.png b/docs/src/pages/docs/_assets/threads-01.png index 0781c8347c..564c753c2e 100644 Binary files a/docs/src/pages/docs/_assets/threads-01.png and b/docs/src/pages/docs/_assets/threads-01.png differ diff --git a/docs/src/pages/docs/_assets/threads-02.png b/docs/src/pages/docs/_assets/threads-02.png index c30ce7a15e..f519165710 100644 Binary files a/docs/src/pages/docs/_assets/threads-02.png and b/docs/src/pages/docs/_assets/threads-02.png differ diff --git a/docs/src/pages/docs/_assets/threads-03.png b/docs/src/pages/docs/_assets/threads-03.png index 5193bdc46e..5a5fd32380 100644 Binary files a/docs/src/pages/docs/_assets/threads-03.png and b/docs/src/pages/docs/_assets/threads-03.png differ diff --git a/docs/src/pages/docs/_assets/threads-04.png b/docs/src/pages/docs/_assets/threads-04.png index 352133941a..748282868b 100644 Binary files a/docs/src/pages/docs/_assets/threads-04.png and b/docs/src/pages/docs/_assets/threads-04.png differ diff --git a/docs/src/pages/docs/_assets/threads-05.png b/docs/src/pages/docs/_assets/threads-05.png index c232c773f1..4c11667d83 100644 Binary files a/docs/src/pages/docs/_assets/threads-05.png and b/docs/src/pages/docs/_assets/threads-05.png differ diff --git a/docs/src/pages/docs/_assets/tools.png b/docs/src/pages/docs/_assets/tools.png deleted file mode 100644 index 8104f6336b..0000000000 Binary files a/docs/src/pages/docs/_assets/tools.png and /dev/null differ diff --git a/docs/src/pages/docs/_assets/trouble-shooting-01.png b/docs/src/pages/docs/_assets/trouble-shooting-01.png index fc965988ba..a12ea97b0b 100644 Binary files a/docs/src/pages/docs/_assets/trouble-shooting-01.png and b/docs/src/pages/docs/_assets/trouble-shooting-01.png differ diff --git a/docs/src/pages/docs/_assets/trouble-shooting-02.png b/docs/src/pages/docs/_assets/trouble-shooting-02.png index e8f7ee4a67..290c4a8ceb 100644 Binary files a/docs/src/pages/docs/_assets/trouble-shooting-02.png and b/docs/src/pages/docs/_assets/trouble-shooting-02.png differ diff --git a/docs/src/pages/docs/_assets/turn-off.png b/docs/src/pages/docs/_assets/turn-off.png deleted file mode 100644 index a16a26a5b9..0000000000 Binary files a/docs/src/pages/docs/_assets/turn-off.png and /dev/null differ diff --git a/docs/src/pages/docs/_assets/ui.gif b/docs/src/pages/docs/_assets/ui.gif deleted file mode 100644 index 946ad8fb9e..0000000000 Binary files a/docs/src/pages/docs/_assets/ui.gif and /dev/null differ diff --git a/docs/src/pages/docs/_assets/vulkan.png b/docs/src/pages/docs/_assets/vulkan.png deleted file mode 100644 index e8104bcc79..0000000000 Binary files a/docs/src/pages/docs/_assets/vulkan.png and /dev/null differ diff --git a/docs/src/pages/docs/_assets/windows.png b/docs/src/pages/docs/_assets/windows.png deleted file mode 100644 index 69ef6f9a89..0000000000 Binary files a/docs/src/pages/docs/_assets/windows.png and /dev/null differ diff --git a/docs/src/pages/docs/_meta.json b/docs/src/pages/docs/_meta.json index 86cba0de69..bdd9be1595 100644 --- a/docs/src/pages/docs/_meta.json +++ b/docs/src/pages/docs/_meta.json @@ -24,17 +24,20 @@ "threads": "Threads", "settings": "Settings", "inference-engines": { - "title": "MODEL PROVIDER", + "title": "ENGINES", "type": "separator" }, - "built-in": "Built-in Models", - "remote-models": "Remote APIs", + "local-engines": "Local Engines", + "remote-models": "Remote Engines", + "install-engines": "Install Engines", "extensions-separator": { "title": "EXTENSIONS", "type": "separator" }, "extensions": "Overview", - "installing-extension": "Installing an Extension", + "extensions-settings": "Extension Settings", + "configure-extensions": "Configure Extensions", + "install-extensions": "Install Extensions", "troubleshooting-separator": { "title": "TROUBLESHOOTING", "type": "separator" diff --git a/docs/src/pages/docs/built-in/_meta.json b/docs/src/pages/docs/built-in/_meta.json deleted file mode 100644 index 0b15c47f2b..0000000000 --- a/docs/src/pages/docs/built-in/_meta.json +++ /dev/null @@ -1,10 +0,0 @@ -{ - "llama-cpp": { - "title": "llama.cpp", - "href": "/docs/built-in/llama-cpp" - }, - "tensorrt-llm": { - "title": "TensorRT-LLM", - "href": "/docs/built-in/tensorrt-llm" - } -} diff --git a/docs/src/pages/docs/built-in/llama-cpp.mdx b/docs/src/pages/docs/built-in/llama-cpp.mdx deleted file mode 100644 index 8d71ff2ae7..0000000000 --- a/docs/src/pages/docs/built-in/llama-cpp.mdx +++ /dev/null @@ -1,133 +0,0 @@ ---- -title: llama.cpp -description: A step-by-step guide on how to customize the llama.cpp engine. -keywords: - [ - Jan, - Customizable Intelligence, LLM, - local AI, - privacy focus, - free and open source, - private and offline, - conversational AI, - no-subscription fee, - large language models, - Llama CPP integration, - llama.cpp Engine, - Intel CPU, - AMD CPU, - NVIDIA GPU, - AMD GPU Radeon, - Apple Silicon, - Intel Arc GPU, - ] ---- - -import { Tabs } from 'nextra/components' -import { Callout, Steps } from 'nextra/components' - -# llama.cpp (Cortex) - -## Overview - -Jan has [**Cortex**](https://github.com/janhq/cortex) - a default C++ inference server built on top of [llama.cpp](https://github.com/ggerganov/llama.cpp). This server provides an OpenAI-compatible API, queues, scaling, and additional features on top of the wide capabilities of `llama.cpp`. - -This guide shows you how to initialize the `llama.cpp` to download and install the required dependencies to start chatting with a model using the `llama.cpp` engine. - -## Prerequisites -- Mac Intel: - - Make sure you're using an Intel-based Mac. For a complete list of supported Intel CPUs, please see [here](https://en.wikipedia.org/wiki/MacBook_Pro_(Intel-based)). - - For Mac Intel, it is recommended to utilize smaller models. -- Mac Sillicon: - - Make sure you're using a Mac Silicon. For a complete list of supported Apple Silicon CPUs, please see [here](https://en.wikipedia.org/wiki/Apple_Silicon). - - Using an adequate model size based on your hardware is recommended for Mac Silicon. - - This can use Apple GPU with Metal by default for acceleration. Apple ANE is not supported yet. - -- Windows: - - Ensure that you have **Windows with x86_64** architecture. -- Linux: - - Ensure that you have **Linux with x86_64** architecture. - -#### GPU Acceleration Options -Enable the GPU acceleration option within the Jan application by following the [Installation Setup](/docs/desktop-installation) guide. -## Step-by-step Guide - -### Step 1: Open the `model.json` -1. Open [Jan Data Folder](/docs/data-folder#open-jan-data-folder) - -
-![Jan Data Folder](../_assets/settings-11.png) -
- -2. Select **models** folder > Click **model folder** that you want to modify > click `model.json` -3. Once open, `model.json` file looks like below, use model "TinyLlama Chat 1.1B Q4" as an example: -```json -{ - "sources": [ - { - "filename": "tinyllama-1.1b-chat-v1.0.Q4_K_M.gguf", - "url": "https://huggingface.co/TheBloke/TinyLlama-1.1B-Chat-v1.0-GGUF/resolve/main/tinyllama-1.1b-chat-v1.0.Q4_K_M.gguf" - } - ], - "id": "tinyllama-1.1b", - "object": "model", - "name": "TinyLlama Chat 1.1B Q4", - "version": "1.0", - "description": "TinyLlama is a tiny model with only 1.1B. It's a good model for less powerful computers.", - "format": "gguf", - "settings": { - "ctx_len": 4096, - "prompt_template": "<|system|>\n{system_message}<|user|>\n{prompt}<|assistant|>", - "llama_model_path": "tinyllama-1.1b-chat-v1.0.Q4_K_M.gguf" - }, - "parameters": { - "temperature": 0.7, - "top_p": 0.95, - "stream": true, - "max_tokens": 2048, - "stop": [], - "frequency_penalty": 0, - "presence_penalty": 0 - }, - "metadata": { - "author": "TinyLlama", - "tags": [ - "Tiny", - "Foundation Model" - ], - "size": 669000000 - }, - "engine": "nitro" -} -``` -### Step 2: Modify the `model.json` -1. Modify the model's engine settings under the settings array. You can modify the settings with the following parameters: - -| Parameter | Type | Description | -| --------------- | ----------- | ---------------------------------------------------------------------------------------------------------------------------------- | -| `ctx_len` | **Integer** | Provides ample context for model operations like `GPT-3.5`. The default value is `2048` (_Maximum_: `4096`, _Minimum_: `1`). | -| `prompt_template` | **String** | Defines the template used to format prompts | -| `model_path` | **String** | Specifies the path to the model `.GGUF` file. | -| `ngl` | **Integer** | Determines GPU layer usage. The default value is `100`. | -| `cpu_threads` | **Integer** | Determines CPU inference threads, limited by hardware and OS. (_Maximum_ determined by system) | -| `cont_batching` | **Integer** | Controls continuous batching, enhancing throughput for LLM inference. | -| `embedding` | **Integer** | Enables embedding utilization for tasks like document-enhanced chat in RAG-based applications. | -2. Save the `model.json` file. - - If you use a different model, you must set it up again. As this only affects the selected model. - -### Step 3: Start the Model -1. Restart the Jan application to apply your settings. -2. Navigate to the **Threads**. -3. Chat with your model. - - - - To utilize the embedding feature, include the JSON parameter `"embedding": true`. It will enable Nitro to process inferences with embedding capabilities. Please refer to the [Embedding in the Nitro documentation](https://nitro.jan.ai/features/embed) for a more detailed explanation. - - To utilize the continuous batching feature for boosting throughput and minimizing latency in large language model (LLM) inference, include `cont_batching: true`. For details, please refer to the [Continuous Batching in the Nitro documentation](https://nitro.jan.ai/features/cont-batch). - - - - - If you have questions, please join our [Discord community](https://discord.gg/Dt7MxDyNNZ) for support, updates, and discussions. - diff --git a/docs/src/pages/docs/built-in/tensorrt-llm.mdx b/docs/src/pages/docs/built-in/tensorrt-llm.mdx deleted file mode 100644 index b165dedc9b..0000000000 --- a/docs/src/pages/docs/built-in/tensorrt-llm.mdx +++ /dev/null @@ -1,91 +0,0 @@ ---- -title: TensorRT-LLM -description: A step-by-step guide on customizing the TensorRT-LLM engine. -keywords: - [ - Jan, - Customizable Intelligence, LLM, - local AI, - privacy focus, - free and open source, - private and offline, - conversational AI, - no-subscription fee, - large language models, - TensorRT-LLM Engine, - TensorRT, - tensorRT, - engine, - ] ---- - -import { Callout, Steps } from 'nextra/components' - -# TensorRT-LLM - -## Overview - -This guide walks you through installing Jan's official [TensorRT-LLM Engine](https://github.com/janhq/nitro-tensorrt-llm). This engine uses [Cortex-TensorRT-LLM](https://github.com/janhq/cortex.tensorrt-llm) as the AI engine instead of the default [Cortex-Llama-CPP](https://github.com/janhq/cortex). It includes an efficient C++ server that executes the [TRT-LLM C++ runtime](https://nvidia.github.io/TensorRT-LLM/gpt_runtime.html) natively. It also includes features and performance improvements like OpenAI compatibility, tokenizer improvements, and queues. - - - This feature is only available for Windows users. Linux is coming soon. - - -### Pre-requisites - -- A **Windows** PC. -- **Nvidia GPU(s)**: Ada or Ampere series (i.e. RTX 4000s & 3000s). More will be supported soon. -- Sufficient disk space for the TensorRT-LLM models and data files (space requirements vary depending on the model size). - - - -### Step 1: Install TensorRT-Extension - -1. Click the **Gear Icon (⚙️)** on the bottom left of your screen. -
-![Settings](../_assets/settings.png) -
-2. Select the **TensorRT-LLM** under the **Model Provider** section. -
-![Click Tensor](../_assets/tensor.png) -
-3. Click **Install** to install the required dependencies to use TensorRT-LLM. -
-![Install Extension](../_assets/install-tensor.png) -
-3. Check that files are correctly downloaded. - -```bash -ls ~/jan/data/extensions/@janhq/tensorrt-llm-extension/dist/bin -# Your Extension Folder should now include `nitro.exe`, among other artifacts needed to run TRT-LLM -``` - -### Step 2: Download a Compatible Model - -TensorRT-LLM can only run models in `TensorRT` format. These models, aka "TensorRT Engines", are prebuilt for each target OS+GPU architecture. - -We offer a handful of precompiled models for Ampere and Ada cards that you can immediately download and play with: - -1. Restart the application and go to the Hub. -2. Look for models with the `TensorRT-LLM` label in the recommended models list > Click **Download**. - - - This step might take some time. 🙏 - - -![image](https://hackmd.io/_uploads/rJewrEgRp.png) - -3. Click **Download** to download the model. - -### Step 3: Configure Settings - -1. Navigate to the Thread section. -2. Select the model that you have downloaded. -3. Customize the default parameters of the model for how Jan runs TensorRT-LLM. - - Please see [here](/docs/models/model-parameters) for more detailed model parameters. - -
-![Specific Conversation](../_assets/model-parameters.png) - - \ No newline at end of file diff --git a/docs/src/pages/docs/configure-extensions.mdx b/docs/src/pages/docs/configure-extensions.mdx new file mode 100644 index 0000000000..c72d8c6a5a --- /dev/null +++ b/docs/src/pages/docs/configure-extensions.mdx @@ -0,0 +1,301 @@ +--- +title: Configure Extensions +description: Learn about Jan's default extensions and explore how to configure them. + [ + Jan, + Customizable Intelligence, LLM, + local AI, + privacy focus, + free and open source, + private and offline, + conversational AI, + no-subscription fee, + large language models, + Jan Extensions, + Extensions, + ] +--- + +# Configure Extensions +Extensions in Jan are configured through a JSON file that defines their behavior and metadata. While most users won't need to modify this file directly, advanced users can customize extension settings for specific needs. + +## Access extensions.json + +1. Navigate to [Jan Data Folder](/docs/data-folder): +2. Open the `extensions.json` file in any text editor + +## Configuration Options + +| Option | Description | Example | +|--------|-------------|---------| +| `_active` | Enable/disable the extension | `true` or `false` | +| `listeners` | Event listeners configuration | `{}` for default | +| `origin` | Installation path of the extension | `"C:\\Users\\...\\jan\\resources\\app.asar.unpacked\\..."` | +| `installOptions` | Installation configuration | `{"version": false, "fullMetadata": true}` | +| `name` | Internal extension identifier | `"@janhq/conversational-extension"` | +| `productName` | Display name shown in UI | `"Conversational"` | +| `version` | Extension version number | `"1.0.0"` | +| `main` | Entry point file path | `"dist/index.js"` | +| `description` | Extension description | `"This extension enables conversations..."` | +| `url` | Extension repository URL | `"extension://@janhq/conversational-extension/dist/index.js"` | + +## Full Example Configuration + +```json title="~/jan/data/extensions/extensions.json" +{ + "@janhq/conversational-extension": { + "_active": true, + "listeners": {}, + "origin": "C:\\Users\\ACER\\AppData\\Local\\Programs\\jan\\resources\\app.asar.unpacked\\pre-install\\janhq-conversational-extension-1.0.0.tgz", + "installOptions": { + "version": false, + "fullMetadata": true + }, + "name": "@janhq/conversational-extension", + "productName": "Conversational", + "version": "1.0.0", + "main": "dist/index.js", + "description": "This extension enables conversations and state persistence via your filesystem", + "url": "extension://@janhq/conversational-extension/dist/index.js" + }, + "@janhq/inference-anthropic-extension": { + "_active": true, + "listeners": {}, + "origin": "C:\\Users\\ACER\\AppData\\Local\\Programs\\jan\\resources\\app.asar.unpacked\\pre-install\\janhq-inference-anthropic-extension-1.0.2.tgz", + "installOptions": { + "version": false, + "fullMetadata": true + }, + "name": "@janhq/inference-anthropic-extension", + "productName": "Anthropic Inference Engine", + "version": "1.0.2", + "main": "dist/index.js", + "description": "This extension enables Anthropic chat completion API calls", + "url": "extension://@janhq/inference-anthropic-extension/dist/index.js" + }, + "@janhq/inference-triton-trt-llm-extension": { + "_active": true, + "listeners": {}, + "origin": "C:\\Users\\ACER\\AppData\\Local\\Programs\\jan\\resources\\app.asar.unpacked\\pre-install\\janhq-inference-triton-trt-llm-extension-1.0.0.tgz", + "installOptions": { + "version": false, + "fullMetadata": true + }, + "name": "@janhq/inference-triton-trt-llm-extension", + "productName": "Triton-TRT-LLM Inference Engine", + "version": "1.0.0", + "main": "dist/index.js", + "description": "This extension enables Nvidia's TensorRT-LLM as an inference engine option", + "url": "extension://@janhq/inference-triton-trt-llm-extension/dist/index.js" + }, + "@janhq/inference-mistral-extension": { + "_active": true, + "listeners": {}, + "origin": "C:\\Users\\ACER\\AppData\\Local\\Programs\\jan\\resources\\app.asar.unpacked\\pre-install\\janhq-inference-mistral-extension-1.0.1.tgz", + "installOptions": { + "version": false, + "fullMetadata": true + }, + "name": "@janhq/inference-mistral-extension", + "productName": "MistralAI Inference Engine", + "version": "1.0.1", + "main": "dist/index.js", + "description": "This extension enables Mistral chat completion API calls", + "url": "extension://@janhq/inference-mistral-extension/dist/index.js" + }, + "@janhq/inference-martian-extension": { + "_active": true, + "listeners": {}, + "origin": "C:\\Users\\ACER\\AppData\\Local\\Programs\\jan\\resources\\app.asar.unpacked\\pre-install\\janhq-inference-martian-extension-1.0.1.tgz", + "installOptions": { + "version": false, + "fullMetadata": true + }, + "name": "@janhq/inference-martian-extension", + "productName": "Martian Inference Engine", + "version": "1.0.1", + "main": "dist/index.js", + "description": "This extension enables Martian chat completion API calls", + "url": "extension://@janhq/inference-martian-extension/dist/index.js" + }, + "@janhq/inference-openrouter-extension": { + "_active": true, + "listeners": {}, + "origin": "C:\\Users\\ACER\\AppData\\Local\\Programs\\jan\\resources\\app.asar.unpacked\\pre-install\\janhq-inference-openrouter-extension-1.0.0.tgz", + "installOptions": { + "version": false, + "fullMetadata": true + }, + "name": "@janhq/inference-openrouter-extension", + "productName": "OpenRouter Inference Engine", + "version": "1.0.0", + "main": "dist/index.js", + "description": "This extension enables Open Router chat completion API calls", + "url": "extension://@janhq/inference-openrouter-extension/dist/index.js" + }, + "@janhq/inference-nvidia-extension": { + "_active": true, + "listeners": {}, + "origin": "C:\\Users\\ACER\\AppData\\Local\\Programs\\jan\\resources\\app.asar.unpacked\\pre-install\\janhq-inference-nvidia-extension-1.0.1.tgz", + "installOptions": { + "version": false, + "fullMetadata": true + }, + "name": "@janhq/inference-nvidia-extension", + "productName": "NVIDIA NIM Inference Engine", + "version": "1.0.1", + "main": "dist/index.js", + "description": "This extension enables NVIDIA chat completion API calls", + "url": "extension://@janhq/inference-nvidia-extension/dist/index.js" + }, + "@janhq/inference-groq-extension": { + "_active": true, + "listeners": {}, + "origin": "C:\\Users\\ACER\\AppData\\Local\\Programs\\jan\\resources\\app.asar.unpacked\\pre-install\\janhq-inference-groq-extension-1.0.1.tgz", + "installOptions": { + "version": false, + "fullMetadata": true + }, + "name": "@janhq/inference-groq-extension", + "productName": "Groq Inference Engine", + "version": "1.0.1", + "main": "dist/index.js", + "description": "This extension enables fast Groq chat completion API calls", + "url": "extension://@janhq/inference-groq-extension/dist/index.js" + }, + "@janhq/inference-openai-extension": { + "_active": true, + "listeners": {}, + "origin": "C:\\Users\\ACER\\AppData\\Local\\Programs\\jan\\resources\\app.asar.unpacked\\pre-install\\janhq-inference-openai-extension-1.0.2.tgz", + "installOptions": { + "version": false, + "fullMetadata": true + }, + "name": "@janhq/inference-openai-extension", + "productName": "OpenAI Inference Engine", + "version": "1.0.2", + "main": "dist/index.js", + "description": "This extension enables OpenAI chat completion API calls", + "url": "extension://@janhq/inference-openai-extension/dist/index.js" + }, + "@janhq/inference-cohere-extension": { + "_active": true, + "listeners": {}, + "origin": "C:\\Users\\ACER\\AppData\\Local\\Programs\\jan\\resources\\app.asar.unpacked\\pre-install\\janhq-inference-cohere-extension-1.0.0.tgz", + "installOptions": { + "version": false, + "fullMetadata": true + }, + "name": "@janhq/inference-cohere-extension", + "productName": "Cohere Inference Engine", + "version": "1.0.0", + "main": "dist/index.js", + "description": "This extension enables Cohere chat completion API calls", + "url": "extension://@janhq/inference-cohere-extension/dist/index.js" + }, + "@janhq/model-extension": { + "_active": true, + "listeners": {}, + "origin": "C:\\Users\\ACER\\AppData\\Local\\Programs\\jan\\resources\\app.asar.unpacked\\pre-install\\janhq-model-extension-1.0.33.tgz", + "installOptions": { + "version": false, + "fullMetadata": true + }, + "name": "@janhq/model-extension", + "productName": "Model Management", + "version": "1.0.33", + "main": "dist/index.js", + "description": "Model Management Extension provides model exploration and seamless downloads", + "url": "extension://@janhq/model-extension/dist/index.js" + }, + "@janhq/monitoring-extension": { + "_active": true, + "listeners": {}, + "origin": "C:\\Users\\ACER\\AppData\\Local\\Programs\\jan\\resources\\app.asar.unpacked\\pre-install\\janhq-monitoring-extension-1.0.10.tgz", + "installOptions": { + "version": false, + "fullMetadata": true + }, + "name": "@janhq/monitoring-extension", + "productName": "System Monitoring", + "version": "1.0.10", + "main": "dist/index.js", + "description": "This extension provides system health and OS level data", + "url": "extension://@janhq/monitoring-extension/dist/index.js" + }, + "@janhq/assistant-extension": { + "_active": true, + "listeners": {}, + "origin": "C:\\Users\\ACER\\AppData\\Local\\Programs\\jan\\resources\\app.asar.unpacked\\pre-install\\janhq-assistant-extension-1.0.1.tgz", + "installOptions": { + "version": false, + "fullMetadata": true + }, + "name": "@janhq/assistant-extension", + "productName": "Jan Assistant", + "version": "1.0.1", + "main": "dist/index.js", + "description": "This extension enables assistants, including Jan, a default assistant that can call all downloaded models", + "url": "extension://@janhq/assistant-extension/dist/index.js" + }, + "@janhq/tensorrt-llm-extension": { + "_active": true, + "listeners": {}, + "origin": "C:\\Users\\ACER\\AppData\\Local\\Programs\\jan\\resources\\app.asar.unpacked\\pre-install\\janhq-tensorrt-llm-extension-0.0.3.tgz", + "installOptions": { + "version": false, + "fullMetadata": true + }, + "name": "@janhq/tensorrt-llm-extension", + "productName": "TensorRT-LLM Inference Engine", + "version": "0.0.3", + "main": "dist/index.js", + "description": "This extension enables Nvidia's TensorRT-LLM for the fastest GPU acceleration. See the [setup guide](https://jan.ai/guides/providers/tensorrt-llm/) for next steps.", + "url": "extension://@janhq/tensorrt-llm-extension/dist/index.js" + }, + "@janhq/inference-cortex-extension": { + "_active": true, + "listeners": {}, + "origin": "C:\\Users\\ACER\\AppData\\Local\\Programs\\jan\\resources\\app.asar.unpacked\\pre-install\\janhq-inference-cortex-extension-1.0.15.tgz", + "installOptions": { + "version": false, + "fullMetadata": true + }, + "name": "@janhq/inference-cortex-extension", + "productName": "Cortex Inference Engine", + "version": "1.0.15", + "main": "dist/index.js", + "description": "This extension embeds cortex.cpp, a lightweight inference engine written in C++. See https://nitro.jan.ai.\nAdditional dependencies could be installed to run without Cuda Toolkit installation.", + "url": "extension://@janhq/inference-cortex-extension/dist/index.js" + } +} +``` + +## Common Use Cases + +1. **Disable an Extension** + ```json + { + "@janhq/example-extension": { + "_active": false + // other settings... + } + } + ``` + +2. **Update Entry Point** + ```json + { + "@janhq/example-extension": { + "main": "new/path/index.js" + // other settings... + } + } + ``` + +## Important Notes + +- Always backup the `extensions.json` file before making changes +- Invalid JSON syntax can prevent Jan from starting properly +- Most users should manage extensions through Jan's Settings UI +- Changes require restarting Jan to take effect diff --git a/docs/src/pages/docs/extensions-settings/_meta.json b/docs/src/pages/docs/extensions-settings/_meta.json new file mode 100644 index 0000000000..100d720ddf --- /dev/null +++ b/docs/src/pages/docs/extensions-settings/_meta.json @@ -0,0 +1,11 @@ +{ + "model-management": { + "title": "Model Management", + "href": "/docs/extensions-settings/model-management" + }, + "system-monitoring": { + "title": "System Monitoring", + "href": "/docs/extensions-settings/system-monitoring" + } +} + \ No newline at end of file diff --git a/docs/src/pages/docs/extensions-settings/model-management.mdx b/docs/src/pages/docs/extensions-settings/model-management.mdx new file mode 100644 index 0000000000..fb7ae69129 --- /dev/null +++ b/docs/src/pages/docs/extensions-settings/model-management.mdx @@ -0,0 +1,43 @@ +--- +title: Model Management +description: Learn about Jan's default extensions and explore how to configure them. +keywords: + [ + Jan, + Customizable Intelligence, LLM, + local AI, + privacy focus, + free and open source, + private and offline, + conversational AI, + no-subscription fee, + large language models, + Jan Extensions, + Extensions, + ] +--- + +import { Callout } from 'nextra/components' +import { Settings, EllipsisVertical, Plus, FolderOpen, Pencil } from 'lucide-react' + +# Model Management + +This extension configures how Jan handles model downloads and management: +- Model discovery and browsing +- Version control & configuration handling +- Download management + +## Hugging Face Access Token + +Access tokens authenticate your identity to Hugging Face Hub for model downloads. +1. Get your token from [Hugging Face Tokens](https://huggingface.co/docs/hub/en/security-tokens) +2. Navigate to **Settings** () > **Core Extensions** > **Model Management** +2. Enter your token in Jan: `hf_************************` + + +Keep your access tokens secure and never share them. + + +
+![Model Management](../_assets/extensions-02.png) +
\ No newline at end of file diff --git a/docs/src/pages/docs/extensions-settings/system-monitoring.mdx b/docs/src/pages/docs/extensions-settings/system-monitoring.mdx new file mode 100644 index 0000000000..f506842e66 --- /dev/null +++ b/docs/src/pages/docs/extensions-settings/system-monitoring.mdx @@ -0,0 +1,54 @@ +--- +title: System Monitoring +description: Learn about Jan's default extensions and explore how to configure them. +keywords: + [ + Jan, + Customizable Intelligence, LLM, + local AI, + privacy focus, + free and open source, + private and offline, + conversational AI, + no-subscription fee, + large language models, + Jan Extensions, + Extensions, + ] +--- +import { Callout } from 'nextra/components' +import { Settings, EllipsisVertical } from 'lucide-react' + + +# System Monitoring +Provides system health and OS level data: +- Hardware utilization tracking +- Performance monitoring +- Error logging + +You can configure your logs in Jan in **Settings** () > **Core Extensions** > **System Monitoring**: + + +## Enable App Logs +Jan can save logs locally on your computer for: +- Debugging model issues +- Crash reports +- Download troubleshooting + +To enable, toggle on **Enable App Logs**: + +
+![Model Management](../_assets/extensions-03.png) +
+ + +## Log Cleaning Interval +Set automatic log deletion interval in milliseconds: +- Default: 120000 (2 minutes); however, there's no minimum or maximum intervals +- Controls disk space usage +- Prevents log accumulation + + +You can clear Jan logs manually with [Clear logs](/docs/settings#log-management) in **Privacy**. + + diff --git a/docs/src/pages/docs/extensions.mdx b/docs/src/pages/docs/extensions.mdx index e248556cfc..aa612a940e 100644 --- a/docs/src/pages/docs/extensions.mdx +++ b/docs/src/pages/docs/extensions.mdx @@ -17,346 +17,59 @@ description: Learn about Jan's default extensions and explore how to configure t --- import { Callout } from 'nextra/components' +import { Settings, EllipsisVertical } from 'lucide-react' -# Extensions Overview +# Extensions -The current Jan Desktop Client has some default extensions built on this framework to enhance the user experience. There are two types of extensions: -- Model Providers: These are extensions that enable Jan to support cloud-based models and provide the engine to support additional local models. -- Core Extensions: These are built-in tools for managing and monitoring the Jan's system. +## Overview +Extensions are modular components that add functionality to Jan. Each extension is designed to handle specific features. -## List of Model Provider Extensions -| Extension Name | Version | Description | -|----------------|---------|-------------| -| Anthropic | v1.0.2 | This extension enables Anthropic chat completion API calls | -| Cohere | v1.0.0 | This extension enables Cohere chat completion API calls | -| Groq | v1.0.1 | This extension enables fast Groq chat completion API calls | -| Martian | v1.0.1 | This extension enables Martian chat completion API calls | -| MistralAI | v1.0.1 | This extension enables Mistral chat completion API calls | -| TensorRT-LLM | v0.0.3 | This extension enables Nvidia's TensorRT-LLM for the fastest GPU acceleration. See the setup guide for next steps | -| NVIDIA NIM | v1.0.1 | This extension enables NVIDIA chat completion API calls | -| OpenAI | v1.0.2 | This extension enables OpenAI chat completion API calls | -| OpenRouter | v1.0.0 | This extension enables Open Router chat completion API calls | -| Triton-TRT-LLM | v1.0.0 | This extension enables Nvidia's Triton-TRT-LLM as an inference engine option | - -## List of Core Extensions -| Extension Name | Version | Description | -|---------------------|----------|-----------------------------------------------------------------------------| -| Jan Assistant | v1.0.1 | This extension enables assistants, including Jan, a default assistant that can call all downloaded models | -| Conversational | v1.0.0 | This extension enables conversations and state persistence via your filesystem | -| Model Management | v1.0.33 | Model Management Extension provides model exploration and seamless downloads | -| System Monitoring | v1.0.10 | This extension provides system health and OS level data | +Extensions can be managed through **Settings** () > **Extensions**: +
+![Remove Model](./_assets/extensions-01.png) +
-## Configure an Extension Settings +## Core Extensions -To configure an extension settings: +### Cortex +The primary extension that manages both **local** and **remote engines** capabilities: -1. Navigate to the `~/jan/data/extensions`. -2. Open the `extensions.json` file -3. Edit the file with options including: +#### Local Engines +[llama.cpp](/docs/local-engines/llama-cpp): Fast, efficient local inference engine that runs GGUF models directly on your device. Powers Jan's default local AI capabilities with support for multiple hardware configurations. -| Option | Description | -| ---------------- | ----------------------------------- | -| `_active` | Enable/disable the extension. | -| `listeners` | Default listener setting. | -| `origin` | Extension file path. | -| `installOptions` | Version and metadata configuration. | -| `name` | Extension name. | -| `productName` | Extension display name. | -| `version` | Extension version. | -| `main` | Main file path. | -| `description` | Extension description. | -| `url` | Extension URL. | +#### Remote Engines +- [Anthropic](/docs/remote-models/anthropic): Access Claude models +- [Cohere](/docs/remote-models/cohere): Access Cohere's models +- [Groq](/docs/remote-models/groq): High-performance inference +- [Martian](/docs/remote-models/martian): Specialized model access +- [MistralAI](/docs/remote-models/mistralai): Access Mistral models +- [NVIDIA NIM](/docs/remote-models/nvidia-nim) (NVIDIA Inference Microservices): Platform for deploying and serving GPU-accelerated AI models, providing enterprise-grade reliability and scalability. +- [OpenAI](/docs/remote-models/openai): Access GPT models +- [OpenRouter](/docs/remote-models/openrouter): Multi-provider model access +- [Triton-TRT-LLM](/docs/remote-models/triton): High-performance inference backend using NVIDIA Triton Inference Server with TensorRT-LLM optimization, designed for large-scale model deployment. -```json title="~/jan/data/extensions/extensions.json" -{ - "@janhq/conversational-extension": { - "_active": true, - "listeners": {}, - "origin": "C:\\Users\\ACER\\AppData\\Local\\Programs\\jan\\resources\\app.asar.unpacked\\pre-install\\janhq-conversational-extension-1.0.0.tgz", - "installOptions": { - "version": false, - "fullMetadata": true - }, - "name": "@janhq/conversational-extension", - "productName": "Conversational", - "version": "1.0.0", - "main": "dist/index.js", - "description": "This extension enables conversations and state persistence via your filesystem", - "url": "extension://@janhq/conversational-extension/dist/index.js" - }, - "@janhq/inference-anthropic-extension": { - "_active": true, - "listeners": {}, - "origin": "C:\\Users\\ACER\\AppData\\Local\\Programs\\jan\\resources\\app.asar.unpacked\\pre-install\\janhq-inference-anthropic-extension-1.0.2.tgz", - "installOptions": { - "version": false, - "fullMetadata": true - }, - "name": "@janhq/inference-anthropic-extension", - "productName": "Anthropic Inference Engine", - "version": "1.0.2", - "main": "dist/index.js", - "description": "This extension enables Anthropic chat completion API calls", - "url": "extension://@janhq/inference-anthropic-extension/dist/index.js" - }, - "@janhq/inference-triton-trt-llm-extension": { - "_active": true, - "listeners": {}, - "origin": "C:\\Users\\ACER\\AppData\\Local\\Programs\\jan\\resources\\app.asar.unpacked\\pre-install\\janhq-inference-triton-trt-llm-extension-1.0.0.tgz", - "installOptions": { - "version": false, - "fullMetadata": true - }, - "name": "@janhq/inference-triton-trt-llm-extension", - "productName": "Triton-TRT-LLM Inference Engine", - "version": "1.0.0", - "main": "dist/index.js", - "description": "This extension enables Nvidia's TensorRT-LLM as an inference engine option", - "url": "extension://@janhq/inference-triton-trt-llm-extension/dist/index.js" - }, - "@janhq/inference-mistral-extension": { - "_active": true, - "listeners": {}, - "origin": "C:\\Users\\ACER\\AppData\\Local\\Programs\\jan\\resources\\app.asar.unpacked\\pre-install\\janhq-inference-mistral-extension-1.0.1.tgz", - "installOptions": { - "version": false, - "fullMetadata": true - }, - "name": "@janhq/inference-mistral-extension", - "productName": "MistralAI Inference Engine", - "version": "1.0.1", - "main": "dist/index.js", - "description": "This extension enables Mistral chat completion API calls", - "url": "extension://@janhq/inference-mistral-extension/dist/index.js" - }, - "@janhq/inference-martian-extension": { - "_active": true, - "listeners": {}, - "origin": "C:\\Users\\ACER\\AppData\\Local\\Programs\\jan\\resources\\app.asar.unpacked\\pre-install\\janhq-inference-martian-extension-1.0.1.tgz", - "installOptions": { - "version": false, - "fullMetadata": true - }, - "name": "@janhq/inference-martian-extension", - "productName": "Martian Inference Engine", - "version": "1.0.1", - "main": "dist/index.js", - "description": "This extension enables Martian chat completion API calls", - "url": "extension://@janhq/inference-martian-extension/dist/index.js" - }, - "@janhq/inference-openrouter-extension": { - "_active": true, - "listeners": {}, - "origin": "C:\\Users\\ACER\\AppData\\Local\\Programs\\jan\\resources\\app.asar.unpacked\\pre-install\\janhq-inference-openrouter-extension-1.0.0.tgz", - "installOptions": { - "version": false, - "fullMetadata": true - }, - "name": "@janhq/inference-openrouter-extension", - "productName": "OpenRouter Inference Engine", - "version": "1.0.0", - "main": "dist/index.js", - "description": "This extension enables Open Router chat completion API calls", - "url": "extension://@janhq/inference-openrouter-extension/dist/index.js" - }, - "@janhq/inference-nvidia-extension": { - "_active": true, - "listeners": {}, - "origin": "C:\\Users\\ACER\\AppData\\Local\\Programs\\jan\\resources\\app.asar.unpacked\\pre-install\\janhq-inference-nvidia-extension-1.0.1.tgz", - "installOptions": { - "version": false, - "fullMetadata": true - }, - "name": "@janhq/inference-nvidia-extension", - "productName": "NVIDIA NIM Inference Engine", - "version": "1.0.1", - "main": "dist/index.js", - "description": "This extension enables NVIDIA chat completion API calls", - "url": "extension://@janhq/inference-nvidia-extension/dist/index.js" - }, - "@janhq/inference-groq-extension": { - "_active": true, - "listeners": {}, - "origin": "C:\\Users\\ACER\\AppData\\Local\\Programs\\jan\\resources\\app.asar.unpacked\\pre-install\\janhq-inference-groq-extension-1.0.1.tgz", - "installOptions": { - "version": false, - "fullMetadata": true - }, - "name": "@janhq/inference-groq-extension", - "productName": "Groq Inference Engine", - "version": "1.0.1", - "main": "dist/index.js", - "description": "This extension enables fast Groq chat completion API calls", - "url": "extension://@janhq/inference-groq-extension/dist/index.js" - }, - "@janhq/inference-openai-extension": { - "_active": true, - "listeners": {}, - "origin": "C:\\Users\\ACER\\AppData\\Local\\Programs\\jan\\resources\\app.asar.unpacked\\pre-install\\janhq-inference-openai-extension-1.0.2.tgz", - "installOptions": { - "version": false, - "fullMetadata": true - }, - "name": "@janhq/inference-openai-extension", - "productName": "OpenAI Inference Engine", - "version": "1.0.2", - "main": "dist/index.js", - "description": "This extension enables OpenAI chat completion API calls", - "url": "extension://@janhq/inference-openai-extension/dist/index.js" - }, - "@janhq/inference-cohere-extension": { - "_active": true, - "listeners": {}, - "origin": "C:\\Users\\ACER\\AppData\\Local\\Programs\\jan\\resources\\app.asar.unpacked\\pre-install\\janhq-inference-cohere-extension-1.0.0.tgz", - "installOptions": { - "version": false, - "fullMetadata": true - }, - "name": "@janhq/inference-cohere-extension", - "productName": "Cohere Inference Engine", - "version": "1.0.0", - "main": "dist/index.js", - "description": "This extension enables Cohere chat completion API calls", - "url": "extension://@janhq/inference-cohere-extension/dist/index.js" - }, - "@janhq/model-extension": { - "_active": true, - "listeners": {}, - "origin": "C:\\Users\\ACER\\AppData\\Local\\Programs\\jan\\resources\\app.asar.unpacked\\pre-install\\janhq-model-extension-1.0.33.tgz", - "installOptions": { - "version": false, - "fullMetadata": true - }, - "name": "@janhq/model-extension", - "productName": "Model Management", - "version": "1.0.33", - "main": "dist/index.js", - "description": "Model Management Extension provides model exploration and seamless downloads", - "url": "extension://@janhq/model-extension/dist/index.js" - }, - "@janhq/monitoring-extension": { - "_active": true, - "listeners": {}, - "origin": "C:\\Users\\ACER\\AppData\\Local\\Programs\\jan\\resources\\app.asar.unpacked\\pre-install\\janhq-monitoring-extension-1.0.10.tgz", - "installOptions": { - "version": false, - "fullMetadata": true - }, - "name": "@janhq/monitoring-extension", - "productName": "System Monitoring", - "version": "1.0.10", - "main": "dist/index.js", - "description": "This extension provides system health and OS level data", - "url": "extension://@janhq/monitoring-extension/dist/index.js" - }, - "@janhq/assistant-extension": { - "_active": true, - "listeners": {}, - "origin": "C:\\Users\\ACER\\AppData\\Local\\Programs\\jan\\resources\\app.asar.unpacked\\pre-install\\janhq-assistant-extension-1.0.1.tgz", - "installOptions": { - "version": false, - "fullMetadata": true - }, - "name": "@janhq/assistant-extension", - "productName": "Jan Assistant", - "version": "1.0.1", - "main": "dist/index.js", - "description": "This extension enables assistants, including Jan, a default assistant that can call all downloaded models", - "url": "extension://@janhq/assistant-extension/dist/index.js" - }, - "@janhq/tensorrt-llm-extension": { - "_active": true, - "listeners": {}, - "origin": "C:\\Users\\ACER\\AppData\\Local\\Programs\\jan\\resources\\app.asar.unpacked\\pre-install\\janhq-tensorrt-llm-extension-0.0.3.tgz", - "installOptions": { - "version": false, - "fullMetadata": true - }, - "name": "@janhq/tensorrt-llm-extension", - "productName": "TensorRT-LLM Inference Engine", - "version": "0.0.3", - "main": "dist/index.js", - "description": "This extension enables Nvidia's TensorRT-LLM for the fastest GPU acceleration. See the [setup guide](https://jan.ai/guides/providers/tensorrt-llm/) for next steps.", - "url": "extension://@janhq/tensorrt-llm-extension/dist/index.js" - }, - "@janhq/inference-cortex-extension": { - "_active": true, - "listeners": {}, - "origin": "C:\\Users\\ACER\\AppData\\Local\\Programs\\jan\\resources\\app.asar.unpacked\\pre-install\\janhq-inference-cortex-extension-1.0.15.tgz", - "installOptions": { - "version": false, - "fullMetadata": true - }, - "name": "@janhq/inference-cortex-extension", - "productName": "Cortex Inference Engine", - "version": "1.0.15", - "main": "dist/index.js", - "description": "This extension embeds cortex.cpp, a lightweight inference engine written in C++. See https://nitro.jan.ai.\nAdditional dependencies could be installed to run without Cuda Toolkit installation.", - "url": "extension://@janhq/inference-cortex-extension/dist/index.js" - } -} -``` -## Specific Extension Settings -Jan offers an Extensions settings menu for configuring extensions that have registered their settings within the application. Here, you can directly integrate Remote Inference Engines with Jan without inserting the URL and API Key directly in the `JSON` file. Additionally, you can turn the Logging extensions available on or off in Jan. To access the Extension settings, follow the steps below: -1. Navigate to the main dashboard. -2. Click the **gear icon (⚙️)** on the bottom left of your screen. -
-![Settings](./_assets/settings.png) -
-3. Click **Extensions**. -
-![Extensions](./_assets/extensions-page2.png) -## Turn Off an Extension - -To turn off the extension, follow the steps below: - -1. Click the **Gear Icon (⚙️)** on the bottom left of your screen. -
-![Settings](./_assets/settings.png) -
-2. Click the **Extensions** button. -
-![Extensions](./_assets/extensions-page2.png) -
-3. Click the slider button to turn off the extension. +### Jan Assistant +Enables assistants functionality, including Jan - the default assistant that can utilize all downloaded models. This extension manages: +- Default assistant configurations +- Model selection +- Conversation management -4. Restart the app to see that the extension has been disabled. +### Conversational +Manages all chat-related functionality and data persistence: -## Model Management -The Model Management extension allows Jan's app to download specific models from Hugging Face repositories, which require an access token for access. -1. Navigate to the main dashboard. -2. Click the **Gear Icon (⚙️)** on the bottom left of your screen. -
-![Settings](./_assets/settings.png) -
-3. Under the **Core Extensions** section, select the **Model Management** extension. -4. Enter the HuggingFace access token. +### [Model Management](/docs/extensions-settings/model-management) +Provides model exploration and seamless downloads: +- Model discovery and browsing +- Version control & configuration handling +- Download management -## System Monitor -The System Monitor extension now offers enhanced customization for app logging. Users can toggle the application logging feature on or off and set a custom interval for clearing the app logs. To configure the app log feature, follow these steps: -1. Navigate to the main dashboard. -2. Click the **Gear Icon (⚙️)** on the bottom left of your screen. -
-![Settings](./_assets/settings.png) -
-3. Under the **Core Extensions** section, select the **System Monitoring** extension. -
-![System Monitoring extension](./_assets/system-monitor2.png) -
-4. Use the **slider** to turn the app logging feature on or off. -
-![System Monitoring Enable](./_assets/system-slider2.png) -
-5. Specify the log cleaning interval in milliseconds. -
-![System Monitoring Interval](./_assets/system-mili2.png) -
- - - You can clear the app logs manually by clicking the **Clear logs** button in the advanced settings. - - There are no minimum or maximum intervals for setting the time. However, invalid inputs will default to `120000ms (2 minutes)`. - \ No newline at end of file +### [System Monitoring](/docs/extensions-settings/system-monitoring) +Provides system health and OS level data: +- Hardware utilization tracking +- Performance monitoring +- Error logging diff --git a/docs/src/pages/docs/install-engines.mdx b/docs/src/pages/docs/install-engines.mdx new file mode 100644 index 0000000000..4e99d337d1 --- /dev/null +++ b/docs/src/pages/docs/install-engines.mdx @@ -0,0 +1,193 @@ +--- +title: Install Engines +description: Learn about Jan's default extensions and explore how to configure them. + [ + Jan, + Customizable Intelligence, LLM, + local AI, + privacy focus, + free and open source, + private and offline, + conversational AI, + no-subscription fee, + large language models, + Jan Extensions, + Extensions, + ] +--- + +import { Callout } from 'nextra/components' +import { Settings, EllipsisVertical } from 'lucide-react' + +# Install Engines + +## Install Local Engines +Jan currently doesn't support installing a local engine yet. + +## Install Remote Engines + +### Step-by-step Guide +You can add any OpenAI API-compatible providers like OpenAI, Anthropic, or others. +To add a new remote engine: + +1. Navigate to **Settings** () > **Engines** +1. At **Remote Engine** category, click **+ Install Engine** + +
+![Install Remote Engines](./_assets/install-engines-01.png) +
+ +2. Fill in the following required information: + +
+![Install Remote Engines](./_assets/install-engines-02.png) +
+ +| Field | Description | Required | +|-------|-------------|----------| +| Engine Name | Name for your engine (e.g., "OpenAI", "Claude") | ✓ | +| API URL | The base URL of the provider's API | ✓ | +| API Key | Your authentication key from the provider | ✓ | +| Model List URL | URL for fetching available models | | +| API Key Template | Custom authorization header format | | +| Request Format Conversion | Function to convert Jan's request format to provider's format | | +| Response Format Conversion | Function to convert provider's response format to Jan's format | | + + +> - The conversion functions are only needed for providers that don't follow the OpenAI API format. For OpenAI-compatible APIs, you can leave these empty. +> - For OpenAI-compatible APIs like OpenAI, Anthropic, or Groq, you only need to fill in the required fields. Leave optional fields empty. + +4. Click **Install** +5. Once completed, you should see your engine in **Engines** page: + - You can rename or uninstall your engine + - You can navigate to its own settings page + +
+![Install Remote Engines](./_assets/install-engines-03.png) +
+ +### Examples +#### OpenAI-Compatible Setup +Here's how to set up OpenAI as a remote engine: + +1. Engine Name: `OpenAI` +2. API URL: `https://api.openai.com` +3. Model List URL: `https://api.openai.com/v1/models` +4. API Key: Your OpenAI API key +5. Leave other fields as default + + +#### Custom APIs Setup +If you're integrating an API that doesn't follow OpenAI's format, you'll need to use the conversion functions. +Let's say you have a custom API with this format: + +```javascript +// Custom API Request Format +{ + "prompt": "What is AI?", + "max_length": 100, + "temperature": 0.7 +} + +// Custom API Response Format +{ + "generated_text": "AI is...", + "tokens_used": 50, + "status": "success" +} +``` + +**Here's how to set it up in Jan:** +``` +Engine Name: Custom LLM +API Key: your_api_key_here +Header template: your header template here +Transform request template: your transform request template here +Transform response template: your transform response template here +``` + +1. Header template +``` +"Authorization: Bearer {{api_key}}" +``` +2. Transform request template: +Convert from Jan's OpenAI-style format to your API's format +``` +"chat_completions": { + "url": "https://api.custom_endpoint.com/v1/messages", + "template": "{ +{% for key, value in input_request %} + {% if key == "messages" %} + "prompt": "{{ last(input_request.messages).content }}" + {% else if key == "max_tokens" or key == "temperature" %} + "{{ key }}": {{ tojson(value) }} + {% endif %} +{% endfor %} +}" +} +``` + + +3. Transform response template +Convert from your API's format back to OpenAI-style format +``` +"chat_completions": { +"template": "{ +{ + "choices": [{ + "message": { + "role": "assistant", + "content": "{{ input_request.generated_text }}" + } + }], + "usage": { + "total_tokens": {{ input_request.tokens_used }} + } +} +}" +} +``` + + + +**Expected Formats:** + +1. Jan's Request Format +``` +{ + "messages": [ + {"role": "user", "content": "What is AI?"} + ], + "max_tokens": 100, + "temperature": 0.7 +} +``` + +2. Jan's Expected Response Format +``` +{ + "choices": [{ + "message": { + "role": "assistant", + "content": "AI is..." + } + }], + "usage": { + "total_tokens": 50 + } +} +``` + + +Make sure to test your conversion functions thoroughly. Incorrect conversions may cause errors or unexpected behavior. + + + + + + + + + + + diff --git a/docs/src/pages/docs/install-extensions.mdx b/docs/src/pages/docs/install-extensions.mdx new file mode 100644 index 0000000000..1e8078034e --- /dev/null +++ b/docs/src/pages/docs/install-extensions.mdx @@ -0,0 +1,89 @@ +--- +title: Install Extensions +description: A step-by-step guide on installing an extension. +keywords: + [ + Jan, + Customizable Intelligence, LLM, + local AI, + privacy focus, + free and open source, + private and offline, + conversational AI, + no-subscription fee, + large language models, + Jan Extensions, + Extensions, + ] +--- + +import { Callout } from 'nextra/components' + + +# Install Extensions + +Jan comes with several [pre-installed extensions](/docs/extensions#core-extensions) that provide core functionalities. You can manually add custom third-party extensions at your own risk. + +## Create Extensions + + +Jan currently only accepts `.tgz` file format for extensions. + + +> **Heads Up:** +> - Please use the following structure and setup as a **reference** only. +> - You're free to develop extensions using any approach or structure that works for your needs. As long as your extension can be packaged as a `.tgz` file, it can be installed in Jan. Feel free to experiment and innovate! +> - If you already have your own `.tgz` extension file, please move forward to [install extension](/docs/install-extensions#install-extensions) step. + +#### Extension Structure +Your extension should follow this basic structure: + +``` +my-extension/ +├── package.json # Extension metadata and dependencies +├── dist/ # Compiled JavaScript files +│ └── index.js # Main extension entry point +├── src/ # Source code +│ └── index.ts # TypeScript source +└── README.md # Extension documentation +``` + +#### Required package.json Fields +```json +{ + "name": "@your-org/extension-name", + "version": "1.0.0", + "main": "dist/index.js", + "types": "dist/index.d.ts", + "jan": { + "type": "extension", + "displayName": "Your Extension Name", + "description": "Description of what your extension does" + }, + "scripts": { + "build": "tsc", + "package": "npm pack" + }, + "dependencies": { + // List your dependencies + }, + "devDependencies": { + "typescript": "^5.0.0" + } +} +``` + +#### Example Extension Template + +You can find a template for creating Jan extensions in our [example repository](https://github.com/janhq/extension-template). +## Install Extensions + +To install a custom extension in Jan: + +1. Navigate to [Jan Data Folder](/docs/data-folder): +2. Copy `.tgz` extension file into the extensions directory +3. Extract the `.tgz` file into its own folder under the extensions directory +4. Restart Jan + +After restart, the `~/jan/data/extensions/extensions.json` file will be updated automatically to include your new extension. + diff --git a/docs/src/pages/docs/installing-extension.mdx b/docs/src/pages/docs/installing-extension.mdx deleted file mode 100644 index 6471e9135b..0000000000 --- a/docs/src/pages/docs/installing-extension.mdx +++ /dev/null @@ -1,46 +0,0 @@ ---- -title: Installing an Extension -description: A step-by-step guide on installing an extension. -keywords: - [ - Jan, - Customizable Intelligence, LLM, - local AI, - privacy focus, - free and open source, - private and offline, - conversational AI, - no-subscription fee, - large language models, - Jan Extensions, - Extensions, - ] ---- - -import { Callout } from 'nextra/components' - - -## Installing an Extension - -Jan automatically installs 11 default extensions when you set up the application. You can also add a custom third-party extension at your own risk. - -Here are the steps to install a custom extension: - -Jan only accepts the `.tgz` file format for installing a custom extension. - - -1. Click the **Gear Icon (⚙️)** on the bottom left of your screen. -
-![Settings](./_assets/settings.png) -
-2. Click the **Extensions** button. -
-![Extensions](./_assets/extensions-page2.png) -
-2. Select **Install Extension** on top right corner. -
-![Install Extension](./_assets/install-ext.png) -
-3. Select a `.tgz` extension file. -4. Restart the Jan application. -5. Then, the `~/jan/data/extensions/extensions.json` file will be updated automatically. \ No newline at end of file diff --git a/docs/src/pages/docs/local-engines/_meta.json b/docs/src/pages/docs/local-engines/_meta.json new file mode 100644 index 0000000000..2e323af69e --- /dev/null +++ b/docs/src/pages/docs/local-engines/_meta.json @@ -0,0 +1,6 @@ +{ + "llama-cpp": { + "title": "llama.cpp", + "href": "/docs/local-engines/llama-cpp" + } +} diff --git a/docs/src/pages/docs/local-engines/llama-cpp.mdx b/docs/src/pages/docs/local-engines/llama-cpp.mdx new file mode 100644 index 0000000000..df11c48ca0 --- /dev/null +++ b/docs/src/pages/docs/local-engines/llama-cpp.mdx @@ -0,0 +1,171 @@ +--- +title: llama.cpp +description: A step-by-step guide on how to customize the llama.cpp engine. +keywords: + [ + Jan, + Customizable Intelligence, LLM, + local AI, + privacy focus, + free and open source, + private and offline, + conversational AI, + no-subscription fee, + large language models, + Llama CPP integration, + llama.cpp Engine, + Intel CPU, + AMD CPU, + NVIDIA GPU, + AMD GPU Radeon, + Apple Silicon, + Intel Arc GPU, + ] +--- + +import { Tabs } from 'nextra/components' +import { Callout, Steps } from 'nextra/components' +import { Settings, EllipsisVertical, Plus, FolderOpen, Pencil } from 'lucide-react' + +# llama.cpp (Cortex) + +## Overview +Jan uses **llama.cpp** for running local AI models. You can find its settings in **Settings** () > **Local Engine** > **llama.cpp**: + +
+![llama.cpp](../_assets/llama.cpp-01.png) +
+ +These settings are for advanced users, you would want to check these settings when: +- Your AI models are running slowly or not working +- You've installed new hardware (like a graphics card) +- You want to tinker & test performance with different [backends](/docs/local-engines/llama-cpp#available-backends) + +## Engine Version and Updates +- **Engine Version**: View current version of llama.cpp engine +- **Check Updates**: Verify if a newer version is available & install available updates when it's available + + +## Available Backends + +Jan offers different backend variants for **llama.cpp** based on your operating system, you can: +- Download different backends as needed +- Switch between backends for different hardware configurations +- View currently installed backends in the list + + +Choose the backend that matches your hardware. Using the wrong variant may cause performance issues or prevent models from loading. + + + + + +### CUDA Support (NVIDIA GPUs) +- `llama.cpp-avx-cuda-11-7` +- `llama.cpp-avx-cuda-12-0` +- `llama.cpp-avx2-cuda-11-7` +- `llama.cpp-avx2-cuda-12-0` +- `llama.cpp-avx512-cuda-11-7` +- `llama.cpp-avx512-cuda-12-0` +- `llama.cpp-noavx-cuda-11-7` +- `llama.cpp-noavx-cuda-12-0` + +### CPU Only +- `llama.cpp-avx` +- `llama.cpp-avx2` +- `llama.cpp-avx512` +- `llama.cpp-noavx` + +### Other Accelerators +- `llama.cpp-vulkan` + + +- For detailed hardware compatibility, please visit our guide for [Windows](/docs/desktop/windows#compatibility). +- AVX, AVX2, and AVX-512 are CPU instruction sets. For best performance, use the most advanced instruction set your CPU supports. +- CUDA versions should match your installed NVIDIA drivers. + + + + + +### CUDA Support (NVIDIA GPUs) +- `llama.cpp-avx-cuda-11-7` +- `llama.cpp-avx-cuda-12-0` +- `llama.cpp-avx2-cuda-11-7` +- `llama.cpp-avx2-cuda-12-0` +- `llama.cpp-avx512-cuda-11-7` +- `llama.cpp-avx512-cuda-12-0` +- `llama.cpp-noavx-cuda-11-7` +- `llama.cpp-noavx-cuda-12-0` + +### CPU Only +- `llama.cpp-avx` +- `llama.cpp-avx2` +- `llama.cpp-avx512` +- `llama.cpp-noavx` + +### Other Accelerators +- `llama.cpp-vulkan` +- `llama.cpp-arm64` + + +- For detailed hardware compatibility, please visit our guide for [Linux](docs/desktop/linux). +- AVX, AVX2, and AVX-512 are CPU instruction sets. For best performance, use the most advanced instruction set your CPU supports. +- CUDA versions should match your installed NVIDIA drivers. + + + + + +### Apple Silicon +- `llama.cpp-mac-arm64`: For M1/M2/M3 Macs + +### Intel +- `llama.cpp-mac-amd64`: For Intel-based Macs + + +For detailed hardware compatibility, please visit our guide for [Mac](/docs/desktop/mac#compatibility). + + + + + + + +## Performance Settings + + +| Setting | Description | Default | +|---------|-------------|---------| +| **Continuous Batching** | - Enables processing multiple requests simultaneously

- Improves throughput for concurrent operations | Enabled | +| **Parallel Operations** | - Number of prompts to run in parallel

- Affects model inference speed | 4 | +| **CPU Threads** | - Number of CPU cores to use when running without GPU

- Higher thread counts may improve performance but increase CPU usage | Auto-detected based on your system's capabilities | + + + +## Memory Settings + +| Setting | Description | Default | +|---------|-------------|---------| +| **Flash Attention** | - Optimizes attention computation

- Reduces memory usage

- Recommended for most cases | Enabled | +| **Caching** | - Enable to store recent prompts and responses

- Improves response time for repeated prompts | Enabled | +| **KV Cache Type** | - KV cache implementation type; controls memory usage and precision trade-off

- Options:

• f16 (most stable)

• q8_0 (balanced)

• q4_0 (lowest memory) | f16 | +| **mmap** | - Enables memory-mapped model loading

- Reduces memory usage

- Recommended for large models | Enabled | + + +## Best Practices +- Start with default settings +- Adjust based on your hardware capabilities +- Monitor system performance +- Test changes with your specific use case + + +Performance impact varies by hardware, model size, and usage patterns. + + + + + + + + diff --git a/docs/src/pages/docs/models/manage-models.mdx b/docs/src/pages/docs/models/manage-models.mdx index dbbfa5d2d1..01f0fc06f9 100644 --- a/docs/src/pages/docs/models/manage-models.mdx +++ b/docs/src/pages/docs/models/manage-models.mdx @@ -17,7 +17,7 @@ keywords: thread history, ] --- -import { Callout, Steps } from 'nextra/components' +import { Callout, Steps } from 'nextra/components' import { Settings, EllipsisVertical, Plus, FolderOpen, Pencil } from 'lucide-react' @@ -108,10 +108,13 @@ You need to own your **model configurations**, use at your own risk. Misconfigur #### 4. Manual Setup For advanced users who add a specific model that is not available within Jan **Hub**: -1. Navigate to `~/jan/data/models/` -2. Create a new **Folder** for your model -3. Add a `model.json` file with your configuration: -``` + +##### Step 1: Create Model File +1. Navigate to [Jan Data Folder]() +2. Open `models` folder +3. Create a new **Folder** for your model +4. Add a `model.json` file with your configuration: +```bash "id": "", "object": "", "name": "", @@ -130,9 +133,50 @@ For advanced users who add a specific model that is not available within Jan **H "engine": "", "source": "" ``` -Key fields to configure: +Here's model "TinyLlama Chat 1.1B Q4" as an example: +```json +{ + "sources": [ + { + "filename": "tinyllama-1.1b-chat-v1.0.Q4_K_M.gguf", + "url": "https://huggingface.co/TheBloke/TinyLlama-1.1B-Chat-v1.0-GGUF/resolve/main/tinyllama-1.1b-chat-v1.0.Q4_K_M.gguf" + } + ], + "id": "tinyllama-1.1b", + "object": "model", + "name": "TinyLlama Chat 1.1B Q4", + "version": "1.0", + "description": "TinyLlama is a tiny model with only 1.1B. It's a good model for less powerful computers.", + "format": "gguf", + "settings": { + "ctx_len": 4096, + "prompt_template": "<|system|>\n{system_message}<|user|>\n{prompt}<|assistant|>", + "llama_model_path": "tinyllama-1.1b-chat-v1.0.Q4_K_M.gguf" + }, + "parameters": { + "temperature": 0.7, + "top_p": 0.95, + "stream": true, + "max_tokens": 2048, + "stop": [], + "frequency_penalty": 0, + "presence_penalty": 0 + }, + "metadata": { + "author": "TinyLlama", + "tags": [ + "Tiny", + "Foundation Model" + ], + "size": 669000000 + }, + "engine": "nitro" +} +``` +##### Step 2: Modify Model Parameters +Modify model parameters under the settings array. Key fields to configure: 1. **Settings** is where you can set your engine configurations. -2. [**Parameters**](/docs/models#model-parameters) are the adjustable settings that affect how your model operates or processes the data. The fields in parameters are typically general and can be the same across models. Here is an example of model parameters: +2. [**Parameters**](/docs/models/model-parameters) are the adjustable settings that affect how your model operates or processes the data. The fields in parameters are typically general and can be the same across models. Here is an example of model parameters: ``` "parameters":{ "temperature": 0.7, @@ -140,9 +184,10 @@ Key fields to configure: "stream": true, "max_tokens": 4096, "frequency_penalty": 0, - "presence_penalty": 0 + "presence_penalty": 0, +} ``` - + ### Delete Models 1. Go to **Settings** () > **My Models** @@ -160,8 +205,8 @@ When using cloud models, be aware of any associated costs and rate limits from t Jan supports connecting to various AI cloud providers that are OpenAI API-compatible, including: OpenAI (GPT-4, o1,...), Anthropic (Claude), Groq, Mistral, and more. -1. Open **Settings** -2. Under **Model Provider** section in left sidebar (OpenAI, Anthropic, etc.), choose a provider +1. Navigate to **Settings** () +2. Under **Remote Engines** section in the left sidebar, choose your preferred engines (OpenAI, Anthropic, etc.) 3. Enter your API key 4. The activated cloud models will be available in your model selector in **Threads** diff --git a/docs/src/pages/docs/quickstart.mdx b/docs/src/pages/docs/quickstart.mdx index 44ee2df922..62a923b518 100644 --- a/docs/src/pages/docs/quickstart.mdx +++ b/docs/src/pages/docs/quickstart.mdx @@ -111,10 +111,9 @@ Jan supports both local and cloud AI models. You can connect to cloud AI service See [Remote APIs](/docs/remote-models/openai) for detailed configuration.
- ![Connect Remote APIs](./_assets/quick-start-03.png) -
+ ## What's Next? diff --git a/docs/src/pages/docs/remote-models/_meta.json b/docs/src/pages/docs/remote-models/_meta.json index 97ad17bad4..529047774f 100644 --- a/docs/src/pages/docs/remote-models/_meta.json +++ b/docs/src/pages/docs/remote-models/_meta.json @@ -1,38 +1,39 @@ { - "openai": { - "title": "OpenAI API", - "href": "/docs/remote-models/openai" + "anthropic": { + "title": "Anthropic", + "href": "/docs/remote-models/anthropic" }, - "azure": { - "title": "Azure OpenAI API", - "href": "/docs/remote-models/azure", - "display": "hidden" + "cohere": { + "title": "Cohere", + "href": "/docs/remote-models/cohere" }, - "groq": { "title": "Groq API", "href": "/docs/remote-models/groq" }, - "mistralai": { - "title": "Mistral AI API", - "href": "/docs/remote-models/mistralai" + "groq": { + "title": "Groq", + "href": "/docs/remote-models/groq" }, - "openrouter": { "title": "OpenRouter", "href": "/docs/remote-models/openrouter" }, - "generic-openai": { "title": "Any OpenAI Compatible API", "href": "/docs/remote-models/generic-openai", "display": "hidden"}, "martian": { - "title": "Martian API", + "title": "Martian", "href": "/docs/remote-models/martian" }, - "cohere": { - "title": "Cohere API", - "href": "/docs/remote-models/cohere" - }, - "anthropic": { - "title": "Anthropic API", - "href": "/docs/remote-models/anthropic" + "mistralai": { + "title": "Mistral AI", + "href": "/docs/remote-models/mistralai" }, "nvidia-nim": { - "title": "NVIDIA NIM API", + "title": "Nvidia NIM", "href": "/docs/remote-models/nvidia-nim" }, + "openai": { + "title": "OpenAI", + "href": "/docs/remote-models/openai" + }, + "openrouter": { + "title": "OpenRouter", + "href": "/docs/remote-models/openrouter" + }, "triton": { "title": "Triton-TRT-LLM", - "href": "/docs/remote-models/triton" + "href": "/docs/remote-models/triton", + "display": "hidden" } -} +} \ No newline at end of file diff --git a/docs/src/pages/docs/remote-models/anthropic.mdx b/docs/src/pages/docs/remote-models/anthropic.mdx index 33259bb3b3..8b7fdbccd8 100644 --- a/docs/src/pages/docs/remote-models/anthropic.mdx +++ b/docs/src/pages/docs/remote-models/anthropic.mdx @@ -16,46 +16,71 @@ keywords: --- import { Callout, Steps } from 'nextra/components' +import { Settings, Plus } from 'lucide-react' # Anthropic -## How to Integrate Anthropic with Jan +Jan supports [Anthropic](https://anthropic.com/) API integration, allowing you to use Claude models (Claude 3, Claude 2.1, and more) through Jan's interface. -This guide provides step-by-step instructions on integrating Anthropic with Jan, enabling users to chat with Claude's LLMs within Jan's conversational interface. +## Integrate Anthropic API with Jan -Before proceeding, ensure you have the following: -- Access to the Jan application -- Anthropic API credentials + +### Step 1: Get Your API Key +1. Visit [Anthropic Console](https://console.anthropic.com/settings/keys) and sign in +2. Create & copy a new API key or copy your existing one -## Integration Steps + +Ensure your API key has sufficient credits + - +### Step 2: Configure Jan +There are two ways to add your Anthropic API keys in Jan: -### Step 1: Configure Anthropic API Key -1. Obtain Anthropic API Keys from your [Anthropic Console](https://console.anthropic.com/). -2. Copy your **Anthropic API Key**. -3. There are three ways to configure your API Key in Jan app: - - Navigate to the **Jan app** > **Gear Icon (⚙️)** > **My Models** tab > **Add Icon (➕)** next to **Anthropic**. - - Navigate to the **Jan app** > **Thread** > **Model** tab > **Add Icon (➕)** next to **Anthropic**. - - Navigate to the **Jan app** > **Gear Icon (⚙️)** > **Anthropic** section under Model Providers. -4. Insert your **Anthropic API Key**. +**Through Threads:** +1. In Threads, click **Model** tab in the **right sidebar** or **model selector** in input field +2. Once the selector is poped up, choose the **Cloud** tab +3. Click **Add** () icon next to **Anthropic** +4. Once you are directed to Anthropic settings, insert your **API Key** -### Step 2: Start Chatting with the Model +**Through Settings:** +1. Navigate to **Settings** () +2. Under **Remote Engines**, select **Anthropic** +3. Insert your **API Key** -1. Select the Anthropic model you want to use. - -Anthropic is the default extension for the Jan application. All the Anthropic models are automatically installed when you install the Jan application. - -2. Specify the model's parameters. -3. Start the conversation with the Anthropic model. +
+![Anthropic](../_assets/anthropic.png) +
+### Step 3: Start Using Anthropic's Models +1. In any existing **Threads** or create a new one +2. Select an Anthropic model from **model selector** +3. Start chatting +## Available Anthropic Models + +Jan automatically includes Anthropic's available models. In case you want to use a specific Anthropic model that you cannot find in **Jan**, follow instructions in [Manual Setup](/docs/models/manage-models#4-manual-setup) to add custom models: + - See list of available models in [Anthropic Models](https://docs.anthropic.com/claude/docs/models-overview). + - The `id` property must match the model name in the list. For example, `claude-3-opus-20240229`, `claude-3-sonnet-20240229`, or `claude-2.1`. + ## Troubleshooting -If you encounter any issues during the integration process or while using Anthropic with Jan, consider the following troubleshooting steps: +Common issues and solutions: + +**1. API Key Issues** +- Verify your API key is correct and not expired +- Check if you have billing set up on your Anthropic account +- Ensure you have access to the model you're trying to use + +**2. Connection Problems** +- Check your internet connection +- Verify Anthropic's system status +- Look for error messages in [Jan's logs](/docs/troubleshooting#how-to-get-error-logs) + +**3. Model Unavailable** +- Confirm your API key has access to the model +- Check if you're using the correct model ID +- Verify your Anthropic account has the necessary permissions -- Double-check your API credentials to ensure they are correct. -- Check for error messages or logs that may provide insight into the issue. -- Reach out to Anthropic API support for assistance if needed. \ No newline at end of file +Need more help? Join our [Discord community](https://discord.gg/FTk2MvZwJH) or check the [Anthropic documentation](https://docs.anthropic.com/claude/docs). \ No newline at end of file diff --git a/docs/src/pages/docs/remote-models/azure.mdx b/docs/src/pages/docs/remote-models/azure.mdx deleted file mode 100644 index af49fe89b6..0000000000 --- a/docs/src/pages/docs/remote-models/azure.mdx +++ /dev/null @@ -1,58 +0,0 @@ ---- -title: Azure OpenAI -description: A step-by-step guide on integrating Jan with Azure OpenAI. -keywords: - [ - Jan, - Customizable Intelligence, LLM, - local AI, - privacy focus, - free and open source, - private and offline, - conversational AI, - no-subscription fee, - large language models, - integration, - Azure OpenAI Service, - ] ---- - -import { Callout, Steps } from 'nextra/components' - - - -# Azure OpenAI API - -## How to Integrate Azure OpenAI API with Jan -This guide provides step-by-step instructions for integrating the Azure OpenAI API with Jan, allowing users to utilize Azure's capabilities within Jan's conversational interface. - -## Integration Steps - -### Step 1: Configure OpenAI API Key -1. Obtain OpenAI API Key from your [OpenAI Platform](https://platform.openai.com/api-keys) dashboard. -2. Copy your **OpenAI API Key**. -3. There are three ways to configure your API Key in Jan app: - - Navigate to the **Jan app** > **Gear Icon (⚙️)** > **My Models** tab > **Add Icon (➕)** next to **OpenAI**. - - Navigate to the **Jan app** > **Thread** > **Model** tab > **Add Icon (➕)** next to **OpenAI**. - - Navigate to the **Jan app** > **Gear Icon (⚙️)** > **OpenAI** section under Model Providers. -4. Insert your **OpenAI API Key**. - - -The **OpenAI** fields can be used for any OpenAI-compatible API. - - -### Step 2: Start Chatting with the Model - -1. Select the OpenAI model you want to use. -2. Specify the model's parameters. -3. Start the conversation with the OpenAI model. - - - -## Troubleshooting - -If you encounter any issues during the integration process or while using OpenAI with Jan, consider the following troubleshooting steps: - -- Double-check your API credentials to ensure they are correct. -- Check for error messages or logs that may provide insight into the issue. -- Reach out to Azure OpenAI API support for assistance if needed. \ No newline at end of file diff --git a/docs/src/pages/docs/remote-models/cohere.mdx b/docs/src/pages/docs/remote-models/cohere.mdx index 8f774a0de7..41e60431d3 100644 --- a/docs/src/pages/docs/remote-models/cohere.mdx +++ b/docs/src/pages/docs/remote-models/cohere.mdx @@ -16,46 +16,71 @@ keywords: --- import { Callout, Steps } from 'nextra/components' +import { Settings, Plus } from 'lucide-react' # Cohere -## How to Integrate Cohere with Jan +Jan supports [Cohere](https://cohere.com/) API integration, allowing you to use Cohere's models (Command, Command-R and more) through Jan's interface. -This guide provides step-by-step instructions on integrating Cohere with Jan, enabling users to chat with Cohere's LLMs within Jan's conversational interface. +## Integrate Cohere API with Jan -Before proceeding, ensure you have the following: -- Access to the Jan application -- Cohere API credentials + +### Step 1: Get Your API Key +1. Visit [Cohere Dashboard](https://dashboard.cohere.com/api-keys) and sign in +2. Create & copy a new API key or copy your existing one -## Integration Steps + +Ensure your API key has sufficient credits + - +### Step 2: Configure Jan +There are two ways to add your Cohere API keys in Jan: -### Step 1: Configure Cohere API Key -1. Obtain Cohere API Keys from your [Cohere Dashboard](https://dashboard.cohere.com/). -2. Copy your **Cohere API Key**. -3. There are three ways to configure your API Key in Jan app: - - Navigate to the **Jan app** > **Gear Icon (⚙️)** > **My Models** tab > **Add Icon (➕)** next to **Cohere**. - - Navigate to the **Jan app** > **Thread** > **Model** tab > **Add Icon (➕)** next to **Cohere**. - - Navigate to the **Jan app** > **Gear Icon (⚙️)** > **Cohere** section under Model Providers. -4. Insert your **Cohere API Key**. +**Through Threads:** +1. In Threads, click **Model** tab in the **right sidebar** or **model selector** in input field +2. Once the selector is poped up, choose the **Cloud** tab +3. Click **Add** () icon next to **Cohere** +4. Once you are directed to Cohere settings, insert your **API Key** +**Through Settings:** +1. Navigate to **Settings** () +2. Under **Remote Engines**, select **Cohere** +3. Insert your **API Key** -### Step 2: Start Chatting with the Model +
+![Cohere](../_assets/cohere.png) +
-1. Select the Cohere model you want to use. - -Cohere is the default extension for the Jan application. All the Cohere models are automatically installed when you install the Jan application. - -2. Specify the model's parameters. -3. Start the conversation with the Cohere model. +### Step 3: Start Using Cohere's Models +1. In any existing **Threads** or create a new one +2. Select a Cohere model from **model selector** +3. Start chatting +## Available Cohere Models + +Jan automatically includes Cohere's available models. In case you want to use a specific Cohere model that you cannot find in **Jan**, follow instructions in [Manual Setup](/docs/models/manage-models#4-manual-setup) to add custom models: + - See list of available models in [Cohere Documentation](https://docs.cohere.com/v2/docs/models). + - The `id` property must match the model name in the list. For example, `command-nightly` or `command-light`. + ## Troubleshooting -If you encounter any issues during the integration process or while using Cohere with Jan, consider the following troubleshooting steps: +Common issues and solutions: + +**1. API Key Issues** +- Verify your API key is correct and not expired +- Check if you have billing set up on your Cohere account +- Ensure you have access to the model you're trying to use + +**2. Connection Problems** +- Check your internet connection +- Verify Cohere's [system status](https://status.cohere.com/) +- Look for error messages in [Jan's logs](/docs/troubleshooting#how-to-get-error-logs) + +**3. Model Unavailable** +- Confirm your API key has access to the model +- Check if you're using the correct model ID +- Verify your Cohere account has the necessary permissions -- Double-check your API credentials to ensure they are correct. -- Check for error messages or logs that may provide insight into the issue. -- Reach out to Cohere API support for assistance if needed. \ No newline at end of file +Need more help? Join our [Discord community](https://discord.gg/FTk2MvZwJH) or check the [Cohere documentation](https://docs.cohere.com). \ No newline at end of file diff --git a/docs/src/pages/docs/remote-models/generic-openai.mdx b/docs/src/pages/docs/remote-models/generic-openai.mdx deleted file mode 100644 index 22f4833729..0000000000 --- a/docs/src/pages/docs/remote-models/generic-openai.mdx +++ /dev/null @@ -1,69 +0,0 @@ ---- -title: Any OpenAI Compatible API -description: A step-by-step guide on how to set up Jan to connect with any remote or local API server. -keywords: - [ - Jan, - Customizable Intelligence, LLM, - local AI, - privacy focus, - free and open source, - private and offline, - conversational AI, - no-subscription fee, - large language models, - import-models-manually, - remote server, - OAI compatible, - ] ---- - -import { Callout, Steps } from 'nextra/components' - -# Any OpenAI-compatible API -This guide outlines the process for configuring Jan as a client for both remote and local API servers, using the `mistral-ins-7b-q4` model for illustration. We'll show how to connect to Jan's API-hosting servers. - - - Currently, you can only connect to one OpenAI-compatible endpoint at a time. - - - -### Step 1: Configure a Client Connection - -1. Navigate to the **Jan app** > **Settings**. -2. Select the **OpenAI**. - - -The **OpenAI** fields can be used for any OpenAI-compatible API. - - -3. Insert the **API Key** and the **endpoint URL** into their respective fields. For example, if you're going to communicate to Jan's API server, you can configure it as follows: -```json -"full_url": "https://:1337/v1/chat/completions" -``` - - Please note that currently, the code that supports any OpenAI-compatible endpoint only reads the `~/jan/data/extensions/@janhq/inference-openai-extension/settings.json` file, which is OpenAI Inference Engines in the extensions page. Thus, it will not search any other files in this directory. - -
-![Server Setup](../_assets/OpenAi-1.gif) - -### Step 2: Start Chatting with the Model - -1. Navigate to the **Hub** section. -2. Select the model you want to use. -3. Specify the model's parameters. -4. Start the conversation with the model. -
-![Start Model](../_assets/OpenAi-2.gif) - - - If you have questions or want more preconfigured GGUF models, please join our [Discord community](https://discord.gg/Dt7MxDyNNZ) for support, updates, and discussions. - - -## Troubleshooting - -If you encounter any issues during the integration process or while using OpenAI with Jan, consider the following troubleshooting steps: - -- Double-check your API credentials to ensure they are correct. -- Check for error messages or logs that may provide insight into the issue. -- Reach out to their API support for assistance if needed. \ No newline at end of file diff --git a/docs/src/pages/docs/remote-models/groq.mdx b/docs/src/pages/docs/remote-models/groq.mdx index 32c192fc50..c0ccaf6270 100644 --- a/docs/src/pages/docs/remote-models/groq.mdx +++ b/docs/src/pages/docs/remote-models/groq.mdx @@ -16,47 +16,72 @@ keywords: --- import { Callout, Steps } from 'nextra/components' +import { Settings, Plus } from 'lucide-react' -# Groq API +# Groq -## How to Integrate Groq API with Jan +Jan supports [Groq](https://groq.com/) API integration, allowing you to use Groq's high-performance LLM models (LLaMA 2, Mixtral and more) through Jan's interface. -This guide provides step-by-step instructions on integrating the Groq API with Jan, enabling users to leverage Groq's capabilities within Jan's conversational interface. +## Integrate Groq API with Jan -Before proceeding, ensure you have the following: -- Access to the Jan application -- Groq API credentials + +### Step 1: Get Your API Key +1. Visit [Groq Console](https://console.groq.com/keys) and sign in +2. Create & copy a new API key or copy your existing one -## Integration Steps + +Ensure your API key has sufficient credits + - +### Step 2: Configure Jan +There are two ways to add your Groq API keys in Jan: -### Step 1: Configure Groq API Key -1. Obtain Groq API Keys from your [Groq Console](https://console.groq.com/keys) dashboard. -2. Copy your **Groq API Key**. -3. There are three ways to configure your API Key in Jan app: - - Navigate to the **Jan app** > **Gear Icon (⚙️)** > **My Models** tab > **Add Icon (➕)** next to **Groq**. - - Navigate to the **Jan app** > **Thread** > **Model** tab > **Add Icon (➕)** next to **Groq**. - - Navigate to the **Jan app** > **Gear Icon (⚙️)** > **Groq** section under Model Providers. -4. Insert your **Groq API Key**. +**Through Threads:** +1. In Threads, click **Model** tab in the **right sidebar** or **model selector** in input field +2. Once the selector is poped up, choose the **Cloud** tab +3. Click **Add** () icon next to **Groq** +4. Once you are directed to Groq settings, insert your **API Key** -### Step 2: Start Chatting with the Model +**Through Settings:** +1. Navigate to **Settings** () +2. Under **Remote Engines**, select **Groq** +3. Insert your **API Key** -1. Select the Groq model you want to use. +
+![Groq](../_assets/groq.png) +
- - The Groq Inference Engine is the default extension for the Jan application. All the Groq models are automatically installed when you install the Jan application. - -2. Specify the model's parameters. -3. Start the conversation with the Groq model. +### Step 3: Start Using Groq's Models +1. In any existing **Threads** or create a new one +2. Select a Groq model from **model selector** +3. Start chatting +## Available Models Through Groq + +Jan automatically includes Groq's available models. In case you want to use a specific Groq model that you cannot find in **Jan**, follow instructions in [Manual Setup](/docs/models/manage-models#4-manual-setup) to add custom models: + - See list of available models in [Groq Documentation](https://console.groq.com/docs/models). + - The `id` property must match the model name in the list. For example, if you want to use Llama3.3 70B, you must set the `id` property to `llama-3.3-70b-versatile`. + ## Troubleshooting -If you encounter any issues during the integration process or while using Groq with Jan, consider the following troubleshooting steps: +Common issues and solutions: + +**1. API Key Issues** +- Verify your API key is correct and not expired +- Check if you have billing set up on your Groq account +- Ensure you have access to the model you're trying to use + +**2. Connection Problems** +- Check your internet connection +- Verify Groq's system status +- Look for error messages in [Jan's logs](/docs/troubleshooting#how-to-get-error-logs) + +**3. Model Unavailable** +- Confirm your API key has access to the model +- Check if you're using the correct model ID +- Verify your Groq account has the necessary permissions -- Double-check your API credentials to ensure they are correct. -- Check for error messages or logs that may provide insight into the issue. -- Reach out to Groq API support for assistance if needed. \ No newline at end of file +Need more help? Join our [Discord community](https://discord.gg/FTk2MvZwJH) or check the [Groq documentation](https://console.groq.com/docs). \ No newline at end of file diff --git a/docs/src/pages/docs/remote-models/martian.mdx b/docs/src/pages/docs/remote-models/martian.mdx index 03278709a2..9b494419c6 100644 --- a/docs/src/pages/docs/remote-models/martian.mdx +++ b/docs/src/pages/docs/remote-models/martian.mdx @@ -14,48 +14,71 @@ keywords: API integration ] --- - import { Callout, Steps } from 'nextra/components' +import { Settings, Plus } from 'lucide-react' # Martian -## How to Integrate Martian with Jan +Jan supports [Martian](https://withmartian.com/) API integration, allowing you to use Martian's models through Jan's interface. -This guide provides step-by-step instructions on integrating Martian with Jan, enabling users to leverage Martian's capabilities within Jan's conversational interface. +## Integrate Martian with Jan -Before proceeding, ensure you have the following: -- Access to the Jan application -- Martian API credentials + +### Step 1: Get Your API Key +1. Visit [Martian API Keys](https://www.withmartian.com/dashboard/undefined/api-keys) and sign in +2. Create & copy a new API key or copy your existing one -## Integration Steps + +Ensure your API key has sufficient credits + - +### Step 2: Configure Jan +There are two ways to add your Martian key in Jan: -### Step 1: Configure Martian API Key -1. Obtain Martian API Keys from your [Martian Dashboard](https://auth.withmartian.com/). -2. Copy your **Martian API Key**. -3. There are three ways to configure your API Key in Jan app: - - Navigate to the **Jan app** > **Gear Icon (⚙️)** > **My Models** tab > **Add Icon (➕)** next to **Martian**. - - Navigate to the **Jan app** > **Thread** > **Model** tab > **Add Icon (➕)** next to **Martian**. - - Navigate to the **Jan app** > **Gear Icon (⚙️)** > **Martian** section under Model Providers. -4. Insert your **Martian API Key**. +**Through Threads:** +1. In Threads, click **Model** tab in the **right sidebar** or **model selector** in input field +2. Once the selector is poped up, choose the **Cloud** tab +3. Click **Add** () icon next to **Martian** +4. Once you are directed to Martian settings, insert your **API Key** +**Through Settings:** +1. Navigate to **Settings** () +2. Under **Remote Engines**, select **Martian** +3. Insert your **API Key** -### Step 2: Start Chatting with the Model +
+![Martian](../_assets/martian.png) +
-1. Select the Martian model you want to use. - -Martian is the default extension for the Jan application. All the Martian models are automatically installed when you install the Jan application. - -2. Specify the model's parameters. -3. Start the conversation with the Martian model. +### Step 3: Start Using Martian Models +1. In any existing **Threads** or create a new one +2. Select a Martian model from **model selector** +3. Start chatting +## Available Models + +Jan includes the Martian Model Router which automatically selects the best model for your use case. You can start using it right away after configuring your API key. See list of available models in [Martian Documentation](https://docs.withmartian.com/martian-model-router/getting-started/supported-models-gateway). + + ## Troubleshooting -If you encounter any issues during the integration process or while using Martian with Jan, consider the following troubleshooting steps: +Common issues and solutions: + +**1. API Key Issues** +- Verify your API key is correct and not expired +- Check if you have billing set up on your Martian account +- Ensure you have access to the model you're trying to use + +**2. Connection Problems** +- Check your internet connection +- Verify Martian's system status +- Look for error messages in [Jan's logs](/docs/troubleshooting#how-to-get-error-logs) + +**3. Model Unavailable** +- Confirm your API key has access to the model +- Check if you're using the correct model ID +- Verify your Martian account has the necessary permissions -- Double-check your API credentials to ensure they are correct. -- Check for error messages or logs that may provide insight into the issue. -- Reach out to Martian API support for assistance if needed. \ No newline at end of file +Need more help? Join our [Discord community](https://discord.gg/FTk2MvZwJH) or check the [Martian documentation](https://docs.withmartian.com/martian-model-router). \ No newline at end of file diff --git a/docs/src/pages/docs/remote-models/mistralai.mdx b/docs/src/pages/docs/remote-models/mistralai.mdx index 5e153dbeca..c1051a7037 100644 --- a/docs/src/pages/docs/remote-models/mistralai.mdx +++ b/docs/src/pages/docs/remote-models/mistralai.mdx @@ -17,49 +17,71 @@ keywords: --- import { Callout, Steps } from 'nextra/components' +import { Settings, Plus } from 'lucide-react' -# Mistral AI API +# Mistral AI -## How to Integrate Mistral AI with Jan -This guide provides step-by-step instructions for integrating the Mistral API with Jan, enabling users to utilize Mistral's capabilities within Jan's conversational interface. +Jan supports [Mistral AI](https://mistral.ai/) API integration, allowing you to use Mistral's powerful models (Mistral Large, Mistral Medium, Mistral Small and more) through Jan's interface. -Before proceeding, ensure you have the following: -- Access to the Jan Application -- Mistral API credentials +## Integrate Mistral AI with Jan -## Integration Steps - -### Step 1: Configure Mistral API Key - -1. Obtain the Mistral API Key from your [Mistral](https://console.mistral.ai/user/api-keys/) dashboard. -2. Copy your **Mistral API Key**. -3. There are three ways to configure your API Key in Jan app: - - Navigate to the **Jan app** > **Gear Icon (⚙️)** > **My Models** tab > **Add Icon (➕)** next to **Mistral**. - - Navigate to the **Jan app** > **Thread** > **Model** tab > **Add Icon (➕)** next to **Mistral**. - - Navigate to the **Jan app** > **Gear Icon (⚙️)** > **Mistral** section under Model Providers. -4. Insert your **Mistral API Key**. +### Step 1: Get Your API Key +1. Visit [Mistral AI Platform](https://console.mistral.ai/api-keys/) and sign in +2. Create & copy a new API key or copy your existing one - - Mistral AI offers various endpoints. Refer to their [endpoint documentation](https://docs.mistral.ai/platform/endpoints/) to select the one that fits your requirements. +Ensure your API key has sufficient credits -### Step 2: Start Chatting with the Model +### Step 2: Configure Jan +There are two ways to add your Mistral AI keys in Jan: -1. Select the Mistral model you want to use. +**Through Threads:** +1. In Threads, click **Model** tab in the **right sidebar** or **model selector** in input field +2. Once the selector is poped up, choose the **Cloud** tab +3. Click **Add** () icon next to **Mistral AI** +4. Once you are directed to Mistral AI settings, insert your **API Key** - - The MistralAI is the default extension for the Jan application. All the Mistral models are automatically installed when you install the Jan application. - -2. Specify the model's parameters. -3. Start the conversation with the Mistral model. +**Through Settings:** +1. Navigate to **Settings** () +2. Under **Remote Engines**, select **Mistral AI** +3. Insert your **API Key** + +
+![Mistral](../_assets/mistralai.png) +
+ +### Step 3: Start Using Mistral's Models +1. In any existing **Threads** or create a new one +2. Select a Mistral model from **model selector** +3. Start chatting +## Available Mistral Models + +Jan automatically includes Mistral's available models. In case you want to use a specific Mistral model that you cannot find in **Jan**, follow instructions in [Manual Setup](/docs/models/manage-models#4-manual-setup) to add custom models: + - See list of available models in [Mistral AI Documentation](https://docs.mistral.ai/platform/endpoints). + - The `id` property must match the model name in the list. For example, if you want to use Mistral Large, you must set the `id` property to `mistral-large-latest` + ## Troubleshooting -If you encounter any issues during the integration process or while using Mistral with Jan, consider the following troubleshooting steps: +Common issues and solutions: + +**1. API Key Issues** +- Verify your API key is correct and not expired +- Check if you have billing set up on your Mistral AI account +- Ensure you have access to the model you're trying to use + +**2. Connection Problems** +- Check your internet connection +- Verify Mistral AI's system status +- Look for error messages in [Jan's logs](/docs/troubleshooting#how-to-get-error-logs) + +**3. Model Unavailable** +- Confirm your API key has access to the model +- Check if you're using the correct model ID +- Verify your Mistral AI account has the necessary permissions -- Double-check your API credentials to ensure they are correct. -- Check for error messages or logs that may provide insight into the issue. -- Reach out to Mistral API support for assistance if needed. \ No newline at end of file +Need more help? Join our [Discord community](https://discord.gg/FTk2MvZwJH) or check the [Mistral AI documentation](https://docs.mistral.ai/). \ No newline at end of file diff --git a/docs/src/pages/docs/remote-models/nvidia-nim.mdx b/docs/src/pages/docs/remote-models/nvidia-nim.mdx index 2fafa96cda..f2e3b0d06a 100644 --- a/docs/src/pages/docs/remote-models/nvidia-nim.mdx +++ b/docs/src/pages/docs/remote-models/nvidia-nim.mdx @@ -16,45 +16,76 @@ keywords: --- import { Callout, Steps } from 'nextra/components' +import { Settings, Plus } from 'lucide-react' # NVIDIA NIM -## How to Integrate NVIDIA NIM with Jan - -This guide provides step-by-step instructions on integrating NVIDIA NIM with Jan, enabling users to leverage Nvidia NIM's capabilities within Jan's conversational interface. +Jan supports [NVIDIA NIM](https://www.nvidia.com/en-us/ai/) API integration, allowing you to use NVIDIA's Large Language Models through Jan's interface. - Nvidia NIM extension is only supported on Jan version 0.5.1 or later. +NVIDIA NIM extension is only supported on Jan version 0.5.1 or later. -Before proceeding, ensure you have the following: -- Access to the Jan application -- NVIDIA NIM API credentials +## Integrate Nvidia NIM API with Jan -## Integration Steps -### Step 1: Configure Nvidia API Key -1. Obtain Nvidia API Keys from your [Nvidia dashboard](https://org.ngc.nvidia.com/setup/personal-keys). -2. Copy your **Nvidia API Key**. -3. There are three ways to configure your API Key in Jan app: - - Navigate to the **Jan app** > **Gear Icon (⚙️)** > **My Models** tab > **Add Icon (➕)** next to **Nvidia**. - - Navigate to the **Jan app** > **Thread** > **Model** tab > **Add Icon (➕)** next to **Nvidia**. - - Navigate to the **Jan app** > **Gear Icon (⚙️)** > **NVIDIA NIM** section under Model Providers. -4. Insert your **Nvidia API Key**. +### Step 1: Get Your API Key +1. Visit [NVIDIA Docs](https://docs.nvidia.com/nim/nemo-retriever/text-reranking/latest/getting-started.html#generate-an-api-key) and generate an API key +2. Copy your API key + + +Ensure your API key has sufficient credits + + +### Step 2: Configure Jan +There are two ways to add your Nvidia NIM API keys in Jan: + +**Through Threads:** +1. In Threads, click **Model** tab in the **right sidebar** or **model selector** in input field +2. Once the selector is poped up, choose the **Cloud** tab +3. Click **Add** () icon next to **Nvidia NIM** +4. Once you are directed to Nvidia NIM settings, insert your **API Key** +**Through Settings:** +1. Navigate to **Settings** () +2. Under **Remote Engines**, select **Nvidia NIM** +3. Insert your **API Key** -### Step 2: Start Chatting with the Model +
+![NVIDIA NIM](../_assets/nvidia-nim.png) +
-1. Select the model you want to use. -2. Specify the model's parameters. -3. Start the conversation with the model. +### Step 3: Start Using Nvidia NIM Models +1. In any existing **Threads** or create a new one +2. Select a NVIDIA NIM model from **model selector** +3. Start chatting +## Available Nvidia NIM Models + +Jan automatically includes NVIDIA NIM's available models. In case you want to use a specific model that you cannot find in **Jan**, follow instructions in [Manual Setup](/docs/models/manage-models#4-manual-setup) to add custom models: + - See list of available models in [Nvidia NIM Documentation](https://build.nvidia.com/models). + - The `id` property must match the model name in the list. + ## Troubleshooting -If you encounter any issues during the integration process or while using Nvidia with Jan, consider the following troubleshooting steps: +Common issues and solutions: + +**1. API Key Issues** +- Verify your API key is correct and not expired +- Check if you have billing set up on your NVIDIA account +- Ensure you have access to the model you're trying to use + +**2. Connection Problems** +- Check your internet connection +- Verify NVIDIA's system status +- Look for error messages in [Jan's logs](/docs/troubleshooting#how-to-get-error-logs) + +**3. Model Unavailable** +- Confirm your API key has access to the model +- Check if you're using the correct model ID +- Verify your NVIDIA account has the necessary permissions +- Make sure you're using Jan version 0.5.1 or later -- Double-check your API credentials to ensure they are correct. -- Check for error messages or logs that may provide insight into the issue. -- Reach out to Nvidia API support for assistance if needed. \ No newline at end of file +Need more help? Join our [Discord community](https://discord.gg/FTk2MvZwJH) or check the [Nvidia documentation](https://docs.nvidia.com/nim/large-language-models/latest/getting-started.html). \ No newline at end of file diff --git a/docs/src/pages/docs/remote-models/openai.mdx b/docs/src/pages/docs/remote-models/openai.mdx index cd7d0454ef..502891f3d7 100644 --- a/docs/src/pages/docs/remote-models/openai.mdx +++ b/docs/src/pages/docs/remote-models/openai.mdx @@ -16,57 +16,73 @@ keywords: Azure OpenAI Service, ] --- - import { Callout, Steps } from 'nextra/components' +import { Settings, Plus } from 'lucide-react' + +# OpenAI +Jan supports [OpenAI](https://openai.com/) and OpenAI-compatible APIs, allowing you to use all models from OpenAI (GPT-4, GPT o1 and more) through Jan's interface. +## Integrate OpenAI API with Jan -# OpenAI API + +### Step 1: Get Your API Key +1. Visit [OpenAI Platform](https://platform.openai.com/api-keys) and sign in +2. Create & copy a new API key or copy your existing one -The OpenAI Extension can be used for any OpenAI-compatible API endpoints. +Ensure your API key has sufficient credits -## How to Integrate OpenAI API with Jan -This guide provides step-by-step instructions for integrating the OpenAI API with Jan, allowing users to utilize OpenAI's capabilities within Jan's conversational interface. +### Step 2: Configure Jan +There are two ways to add your OpenAI API keys in Jan: -## Integration Steps - -### Step 1: Configure OpenAI API Key -1. Obtain OpenAI API Key from your [OpenAI Platform](https://platform.openai.com/api-keys) dashboard. -2. Copy your **OpenAI API Key**. -3. There are three ways to configure your API Key in Jan app: - - Navigate to the **Jan app** > **Gear Icon (⚙️)** > **My Models** tab > **Add Icon (➕)** next to **OpenAI**. - - Navigate to the **Jan app** > **Thread** > **Model** tab > **Add Icon (➕)** next to **OpenAI**. - - Navigate to the **Jan app** > **Gear Icon (⚙️)** > **OpenAI** section under Model Providers. -4. Insert your **OpenAI API Key**. +Through Threads: +1. In Threads, click Model tab in the right sidebar or model selector in input field +2. Once the selector is poped up, choose the Cloud tab +3. Click Add () icon next to OpenAI +4. Once you are directed to OpenAI settings, insert your API Key -### Step 2: Start Chatting with the Model +Through Settings: +1. Navigate to Settings () +2. Under Remote Engines, select OpenAI +3. Insert your API Key -1. Select the OpenAI model you want to use. +
+![OpenAI](../_assets/openai.png) +
- -The OpenAI is the default extension for the Jan application. All the OpenAI models are automatically installed when you install the Jan application. - +### Step 3: Start Using OpenAI's Models -2. Specify the model's parameters. -3. Start the conversation with the OpenAI model. - +In any existing Threads or create a new one +Select an OpenAI model from model selector +Start chatting -### OpenAI Models + -You can also use specific OpenAI models you cannot find in the **Hub** section by customizing the `model.yaml` file, which you can see in the `~/jan/data/models/`. Follow the steps in the [Manage Models](/docs/models/manage-models) to manually add a model. +## Available OpenAI Models - -- You can find the list of available models in the [OpenAI Platform](https://platform.openai.com/docs/models/overview). -- The `id` property must match the model name in the list. - - For example, if you want to use the [GPT-4 Turbo](https://platform.openai.com/docs/models/gpt-4-and-gpt-4-turbo), you must set the `id` property to `gpt-4-1106-preview`. - +Jan automatically includes popular OpenAI models. In case you want to use a specific OpenAI model that you cannot find in Jan, follow instructions in [Manual Setup](/docs/models/manage-models#4-manual-setup) to add custom models: + - See list of available models in [OpenAI Platform](https://platform.openai.com/docs/models/overview). + - The id property must match the model name in the list. For example, if you want to use the [GPT-4 Turbo](https://platform.openai.com/docs/models/gpt-4-and-gpt-4-turbo), you must set the id property to gpt-4-1106-preview. ## Troubleshooting -If you encounter any issues during the integration process or while using OpenAI with Jan, consider the following troubleshooting steps: +Common issues and solutions: + +1. API Key Issues +- Verify your API key is correct and not expired +- Check if you have billing set up on your OpenAI account +- Ensure you have access to the model you're trying to use + +2. Connection Problems +- Check your internet connection +- Verify OpenAI's [system status](https://status.openai.com) +- Look for error messages in [Jan's logs](/docs/troubleshooting#how-to-get-error-logs) + +3. Model Unavailable +- Confirm your API key has access to the model +- Check if you're using the correct model ID +- Verify your OpenAI account has the necessary permissions -- Double-check your API credentials to ensure they are correct. -- Check for error messages or logs that may provide insight into the issue. -- Reach out to OpenAI API support for assistance if needed. \ No newline at end of file +Need more help? Join our [Discord community](https://discord.gg/FTk2MvZwJH) or check the [OpenAI documentation](https://platform.openai.com/docs). \ No newline at end of file diff --git a/docs/src/pages/docs/remote-models/openrouter.mdx b/docs/src/pages/docs/remote-models/openrouter.mdx index 8efcd932a0..a32de24236 100644 --- a/docs/src/pages/docs/remote-models/openrouter.mdx +++ b/docs/src/pages/docs/remote-models/openrouter.mdx @@ -18,38 +18,83 @@ keywords: --- import { Callout, Steps } from 'nextra/components' +import { Settings, Plus } from 'lucide-react' # OpenRouter ## Integrate OpenRouter with Jan -[OpenRouter](https://openrouter.ai/docs#quick-start) is a tool that gathers AI models. Developers can utilize its API to engage with diverse large language models, generative image models, and generative 3D object models. +[OpenRouter](https://openrouter.ai/) is a tool that gathers AI models. Developers can utilize its API to engage with diverse large language models, generative image models, and generative 3D object models with a competitive pricing. -To connect Jan with OpenRouter for accessing remote Large Language Models (LLMs) through OpenRouter, you can follow the steps below: +Jan supports OpenRouter API integration, allowing you to use models from various providers (Anthropic, Google, Meta and more) through a single API. + +## Integrate OpenRouter with Jan -### Step 1: Configure OpenRouter API Key - -1. Find your API Key in the [OpenRouter API Key](https://openrouter.ai/keys). -2. Copy your **OpenRouter API Key**. -3. There are three ways to configure your API Key in Jan app: - - Navigate to the **Jan app** > **Gear Icon (⚙️)** > **My Models** tab > **Add Icon (➕)** next to **OpenRouter**. - - Navigate to the **Jan app** > **Thread** > **Model** tab > **Add Icon (➕)** next to **OpenRouter**. - - Navigate to the **Jan app** > **Gear Icon (⚙️)** > **OpenRouter** section under Model Providers. -4. Insert your **OpenRouter API Key**. -5. For **OpenRouter**, specify the model you want to use, or the system will default to the preset model linked to your **OpenRouter API Key**. - -### Step 2: Start Chatting with the Model - -1. Select the OpenRouter model you want to use. -2. Specify the model's parameters. -3. Start the conversation with the OpenRouter model. +### Step 1: Get Your API Key +1. Visit [OpenRouter](https://openrouter.ai/keys) and sign in +2. Create & copy a new API key or copy your existing one + + +Ensure your API key has sufficient credits. OpenRouter credits work across all available models. + + +### Step 2: Configure Jan +There are two ways to add your OpenRouter key in Jan: + +**Through Threads:** +1. In Threads, click **Model** tab in the **right sidebar** or **model selector** in input field +2. Once the selector is poped up, choose the **Cloud** tab +3. Click **Add** () icon next to **OpenRouter** +4. Once you are directed to OpenRouter settings, insert your **API Key** + +**Through Settings:** +1. Navigate to **Settings** () +2. Under **Remote Engines**, select **OpenRouter** +3. Insert your **API Key** + +
+![OpenRouter](../_assets/openrouter.png) +
+ +### Step 3: Start Using OpenRouter Models + +1. In any existing **Threads** or create a new one +2. Select any model from **model selector** under OpenRouter +3. Start chatting +## Available Models Through OpenRouter + +Jan automatically use your default OpenRouter's available models. For custom configurations: + +**Model Field Settings:** +- Leave empty to use your account's default model +- Specify a model using the format: `organization/model-name` +- Available options can be found in [OpenRouter's Model Reference](https://openrouter.ai/models) + +**Examples of Model IDs:** +- Claude 3 Opus: `anthropic/claude-3-opus-20240229` +- Google Gemini Pro: `google/gemini-pro` +- Mistral Large: `mistralai/mistral-large` + ## Troubleshooting -If you encounter any issues during the integration process or while using OpenAI with Jan, consider the following troubleshooting steps: +Common issues and solutions: + +**1. API Key Issues** +- Verify your API key is correct and not expired +- Check if you have sufficient credits in your OpenRouter account +- Ensure you have access to the model you're trying to use + +**2. Connection Problems** +- Check your internet connection +- Verify OpenRouter's [system status](https://status.openrouter.ai) +- Look for error messages in [Jan's logs](/docs/troubleshooting#how-to-get-error-logs) + +**3. Model Unavailable** +- Confirm the model is currently available on OpenRouter +- Check if you're using the correct model ID format +- Verify the model provider is currently operational -- Double-check your API credentials to ensure they are correct. -- Check for error messages or logs that may provide insight into the issue. -- Reach out to OpenRouter API support for assistance if needed. \ No newline at end of file +Need more help? Join our [Discord community](https://discord.gg/FTk2MvZwJH) or check the [OpenRouter documentation](https://openrouter.ai/docs). \ No newline at end of file diff --git a/docs/src/pages/docs/settings.mdx b/docs/src/pages/docs/settings.mdx index fb99df5186..9b5f3fe4d6 100644 --- a/docs/src/pages/docs/settings.mdx +++ b/docs/src/pages/docs/settings.mdx @@ -29,10 +29,6 @@ import { Settings, EllipsisVertical, Plus, FolderOpen, Pencil } from 'lucide-rea This guide explains how to customize your Jan application settings. To access **Settings**, click icon in the bottom left corner of Jan. - -Settings are stored in a `cortex.db` file in [Jan Data Folder](/docs/data-folder), ensuring they persist across sessions. - - ## My models Here's at **Settings** () > **My Models** you can manage all your installed AI models: