More Model Providers support 更多模型服务商支持 #1284
Replies: 86 comments 168 replies
This comment was marked as spam.
This comment was marked as spam.
This comment has been hidden.
This comment has been hidden.
This comment has been hidden.
This comment has been hidden.
This comment has been hidden.
This comment has been hidden.
This comment has been hidden.
This comment has been hidden.
This comment has been hidden.
This comment has been hidden.
This comment has been hidden.
This comment has been hidden.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
-
申请增加GCP的Vertex AI大模型服务,此类服务用服务账号身份认证。可以满足企业级数据合规要求。🙏 |
Beta Was this translation helpful? Give feedback.
This comment was marked as off-topic.
This comment was marked as off-topic.
This comment has been hidden.
This comment has been hidden.
This comment has been hidden.
This comment has been hidden.
-
It would be great if you could add the Hugging Face Inference API, available at Huggingface. |
Beta Was this translation helpful? Give feedback.
-
希望增加文心一言:https://yiyan.baidu.com/ |
Beta Was this translation helpful? Give feedback.
This comment has been hidden.
This comment has been hidden.
This comment has been hidden.
This comment has been hidden.
This comment has been minimized.
This comment has been minimized.
-
cloudflare ai啥时候完事,等不及了😊 |
Beta Was this translation helpful? Give feedback.
-
octoai 可以支持吗 |
Beta Was this translation helpful? Give feedback.
-
Please add replicate https://replicate.com |
Beta Was this translation helpful? Give feedback.
This comment has been hidden.
This comment has been hidden.
-
请问是否可以兼容POE的api,这里面的模型种类也很丰富 |
Beta Was this translation helpful? Give feedback.
-
Support Azure ML Studio EndpointCan you please support Azure ML Studio endpoints like An exemple on how to use this endpoint (from Azure code snippet helper) // Request data goes here
// The example below assumes JSON formatting which may be updated
// depending on the format your endpoint expects.
// More information can be found here:
// https://docs.microsoft.com/azure/machine-learning/how-to-deploy-advanced-entry-script
const requestBody = `{}`;
const requestHeaders = new Headers({"Content-Type" : "application/json"});
// Replace this with the primary/secondary key, AMLToken, or Microsoft Entra ID token for the endpoint
const apiKey = "";
if (!apiKey)
{
throw new Error("A key should be provided to invoke the endpoint");
}
requestHeaders.append("Authorization", "Bearer " + apiKey)
// This header will force the request to go to a specific deployment.
// Remove this line to have the request observe the endpoint traffic rules
requestHeaders.append("azureml-model-deployment", "Phi-3-mini-4k-instruct");
const url = "https://model_workspace.westeurope.inference.ml.azure.com/score";
fetch(url, {
method: "POST",
body: requestBody,
headers: requestHeaders
})
.then((response) => {
if (response.ok) {
return response.json();
} else {
// Print the headers - they include the request ID and the timestamp, which are useful for debugging the failure
console.debug(...response.headers);
console.debug(response.body)
throw new Error("Request failed with status code" + response.status);
}
})
.then((json) => console.log(json))
.catch((error) => {
console.error(error)
}); And an example to use my model ( // Request body
{
data: "<|system|>the system prompt<|end|><|user|>the user prompt<|end|><|assistant|>"
} The input data structure use input format of the Phi3 model Thanks ! |
Beta Was this translation helpful? Give feedback.
-
期待增加 midjourney或者SD绘画 |
Beta Was this translation helpful? Give feedback.
-
SambaNova would make for a nice addition |
Beta Was this translation helpful? Give feedback.
-
Beside Azure ML Studio, there is also an Azure AI Studio at https://ai.azure.com/ Here I deployed the LLaMa 3.1 405B and this is the sample code from Azure code snippet: # pip install azure-ai-inference
import os
from azure.ai.inference import ChatCompletionsClient
from azure.core.credentials import AzureKeyCredential
api_key = os.getenv("AZURE_INFERENCE_CREDENTIAL", '')
if not api_key:
raise Exception("A key should be provided to invoke the endpoint")
client = ChatCompletionsClient(
endpoint='https://Meta-Llama-3-1-405B-Instruct-bft.eastus.models.ai.azure.com',
credential=AzureKeyCredential(api_key)
)
model_info = client.get_model_info()
print("Model name:", model_info.model_name)
print("Model type:", model_info.model_type)
print("Model provider name:", model_info.model_provider_name)
payload = {
"messages": [
{
"role": "user",
"content": "I am going to Paris, what should I see?"
},
{
"role": "assistant",
"content": "Paris, the capital of France, is known for its stunning architecture, art museums, historical landmarks, and romantic atmosphere. Here are some of the top attractions to see in Paris:\n\n1. The Eiffel Tower: The iconic Eiffel Tower is one of the most recognizable landmarks in the world and offers breathtaking views of the city.\n2. The Louvre Museum: The Louvre is one of the world's largest and most famous museums, housing an impressive collection of art and artifacts, including the Mona Lisa.\n3. Notre-Dame Cathedral: This beautiful cathedral is one of the most famous landmarks in Paris and is known for its Gothic architecture and stunning stained glass windows.\n\nThese are just a few of the many attractions that Paris has to offer. With so much to see and do, it's no wonder that Paris is one of the most popular tourist destinations in the world."
},
{
"role": "user",
"content": "What is so great about #1?"
}
],
"max_tokens": 4096,
"temperature": 0.8,
"top_p": 0.1,
"presence_penalty": 0
}
response = client.complete(payload)
print("Response:", response.choices[0].message.content)
print("Model:", response.model)
print("Usage:")
print(" Prompt tokens:", response.usage.prompt_tokens)
print(" Total tokens:", response.usage.total_tokens)
print(" Completion tokens:", response.usage.completion_tokens) |
Beta Was this translation helpful? Give feedback.
-
Cerebras AI |
Beta Was this translation helpful? Give feedback.
-
LobeChat currently supports many model providers, and considering the development of the AI ecosystem, we have specifically opened this discussion to record everyone's needs for new model service providers.
LobeChat 目前已经支持了诸多模型供应商,考虑到 AI 生态的发展,我们专门开放此贴用于记录大家对于新模型服务商的需求。
Currently, the service providers currently supported by LobeChat are as follows:
目前 LobeChat 已支持的服务商如下:
WIP:
Plan:
Others:
Important
If you have any requests for new model suppliers, feel free to comment below. When the number of upvotes exceeds 15, we will consider implementing it.
如果大家对于新模型供应商有诉求,欢迎在下方跟帖,当up数超过 15 时,我们会考虑将其实现
PRs Welcome 💞
Tip
We also welcome PRs. We have received many contributions regarding model Providers, so we have maintained a library of icons for LLMs and Provider vendors, facilitating everyone's future development. https://github.com/lobehub/lobe-icons
同时欢迎 PR,我们已经收获到许多关于模型 Provider 的贡献, 因此我们维护了一个 LLMs 和 Provider 供应商的图标库,方便大家后续开发 https://github.com/lobehub/lobe-icons
Beta Was this translation helpful? Give feedback.
All reactions