Description
Description
Jupyternaut fails when trying to use NVIDIA::playground_llama2_70b (and other models from NVIDIA) with:
ValueError: Model meta/llama2-70b is incompatible with client ChatNVIDIAProvider. Please check `ChatNVIDIAProvider.get_available_models()`.
When suggested command run it fails with other error:
AssertionError: model_id was not specified. Please specify it as a keyword argument.
Reproduce
To reproduce:
python -m venv /tmp/venv
source /tmp/venv/bin/activate
pip install jupyter-ai[all]==2.28.2 langchain==0.2.17 langchain-core==0.2.43 langchain-nvidia-ai-endpoints==0.2.2
export NVIDIA_API_KEY="key"
python -c 'from jupyter_ai_magics.partner_providers.nvidia import ChatNVIDIAProvider; ChatNVIDIAProvider(model_id="playground_llama2_70b")'
Output will be
/tmp/venv/lib/python3.12/site-packages/langchain_nvidia_ai_endpoints/_statics.py:659: UserWarning: Model playground_llama2_70b is deprecated. Using meta/llama2-70b instead.
warnings.warn(
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/tmp/venv/lib/python3.12/site-packages/jupyter_ai_magics/providers.py", line 354, in __init__
super().__init__(*args, **kwargs, **model_kwargs)
File "/tmp/venv/lib/python3.12/site-packages/langchain_nvidia_ai_endpoints/chat_models.py", line 243, in __init__
self._client = _NVIDIAClient(
^^^^^^^^^^^^^^
File "/tmp/venv/lib/python3.12/site-packages/langchain_nvidia_ai_endpoints/_common.py", line 175, in __init__
raise ValueError(
ValueError: Model meta/llama2-70b is incompatible with client ChatNVIDIAProvider. Please check `ChatNVIDIAProvider.get_available_models()`.
Context
Troubleshoot Output
PATH: /tmp/venv/bin /usr/sbin /usr/bin /bin /sbinsys.path:
/tmp/venv/bin
/usr/lib/python312.zip
/usr/lib/python3.12
/usr/lib/python3.12/lib-dynload
/usr/python3.12/site-packagessys.executable:
/tmp/venv/bin/pythonsys.version:
3.12.7 (main, Nov 6 2024, 21:09:52) [GCC 14.1.0]platform.platform():
Linux-5.14.0-427.13.1.el9_4.x86_64-x86_64-with-glibc2.34which -a jupyter:
/tmp/venv/bin/jupyterpip list:
Package Version
----------------------------- ---------------
ai21 3.0.0
ai21-tokenizer 0.12.0
aiohappyeyeballs 2.4.3
aiohttp 3.11.7
aiolimiter 1.1.0
aiosignal 1.3.1
aiosqlite 0.20.0
annotated-types 0.7.0
anthropic 0.39.0
anyio 4.6.2.post1
argon2-cffi 23.1.0
argon2-cffi-bindings 21.2.0
arrow 1.3.0
arxiv 2.1.3
asttokens 2.4.1
async-lru 2.0.4
attrs 24.2.0
babel 2.16.0
bce-python-sdk 0.9.23
beautifulsoup4 4.12.3
bleach 6.2.0
boto3 1.34.162
botocore 1.34.162
cachetools 5.5.0
certifi 2024.8.30
cffi 1.17.1
charset-normalizer 3.4.0
click 8.1.7
cloudpickle 3.1.0
cohere 5.11.4
comm 0.2.2
dask 2024.11.2
dataclasses-json 0.6.7
debugpy 1.8.9
decorator 5.1.1
deepmerge 2.0
defusedxml 0.7.1
dill 0.3.9
diskcache 5.6.3
distributed 2024.11.2
distro 1.9.0
eval_type_backport 0.2.0
executing 2.1.0
faiss-cpu 1.9.0.post1
fastavro 1.9.7
fastjsonschema 2.20.0
feedparser 6.0.11
filelock 3.16.1
fqdn 1.5.1
frozenlist 1.5.0
fsspec 2024.10.0
future 1.0.0
google-ai-generativelanguage 0.6.6
google-api-core 2.23.0
google-api-python-client 2.154.0
google-auth 2.36.0
google-auth-httplib2 0.2.0
google-generativeai 0.7.2
googleapis-common-protos 1.66.0
gpt4all 2.8.2
greenlet 3.1.1
grpcio 1.68.0
grpcio-status 1.62.3
h11 0.14.0
httpcore 1.0.7
httplib2 0.22.0
httpx 0.27.2
httpx-sse 0.4.0
huggingface-hub 0.26.2
idna 3.10
importlib_metadata 8.5.0
ipykernel 6.29.5
ipython 8.29.0
ipywidgets 8.1.5
isoduration 20.11.0
jedi 0.19.2
Jinja2 3.1.4
jiter 0.7.1
jmespath 1.0.1
json5 0.9.28
jsonpatch 1.33
jsonpath-ng 1.7.0
jsonpointer 3.0.0
jsonschema 4.23.0
jsonschema-specifications 2024.10.1
jupyter_ai 2.28.2
jupyter_ai_magics 2.28.2
jupyter_client 8.6.3
jupyter_core 5.7.2
jupyter-events 0.10.0
jupyter-lsp 2.2.5
jupyter_server 2.14.2
jupyter_server_terminals 0.5.3
jupyterlab 4.3.1
jupyterlab_pygments 0.3.0
jupyterlab_server 2.27.3
jupyterlab_widgets 3.0.13
langchain 0.2.17
langchain-anthropic 0.1.23
langchain-aws 0.1.18
langchain-cohere 0.2.4
langchain-community 0.2.19
langchain-core 0.2.43
langchain-experimental 0.0.65
langchain-google-genai 1.0.10
langchain-mistralai 0.1.13
langchain-nvidia-ai-endpoints 0.2.2
langchain-ollama 0.1.3
langchain-openai 0.1.25
langchain-text-splitters 0.2.4
langsmith 0.1.146
locket 1.0.0
markdown-it-py 3.0.0
MarkupSafe 3.0.2
marshmallow 3.23.1
matplotlib-inline 0.1.7
mdurl 0.1.2
mistune 3.0.2
msgpack 1.1.0
multidict 6.1.0
multiprocess 0.70.17
mypy-extensions 1.0.0
nbclient 0.10.0
nbconvert 7.16.4
nbformat 5.10.4
nest-asyncio 1.6.0
notebook_shim 0.2.4
numpy 1.26.4
ollama 0.4.1
openai 1.55.1
orjson 3.10.12
overrides 7.7.0
packaging 24.2
pandas 2.2.3
pandocfilters 1.5.1
parameterized 0.9.0
parso 0.8.4
partd 1.4.2
pexpect 4.9.0
pillow 10.4.0
pip 24.2
platformdirs 4.3.6
ply 3.11
prometheus_client 0.21.0
prompt_toolkit 3.0.48
propcache 0.2.0
proto-plus 1.25.0
protobuf 4.25.5
psutil 6.1.0
ptyprocess 0.7.0
pure_eval 0.2.3
pyarrow 18.1.0
pyasn1 0.6.1
pyasn1_modules 0.4.1
pycparser 2.22
pycryptodome 3.21.0
pydantic 2.10.2
pydantic_core 2.27.1
Pygments 2.18.0
pyparsing 3.2.0
pypdf 5.1.0
python-dateutil 2.9.0.post0
python-dotenv 1.0.1
python-json-logger 2.0.7
pytz 2024.2
PyYAML 6.0.2
pyzmq 26.2.0
qianfan 0.4.12.1
referencing 0.35.1
regex 2024.11.6
requests 2.32.3
requests-toolbelt 1.0.0
rfc3339-validator 0.1.4
rfc3986-validator 0.1.1
rich 13.9.4
rpds-py 0.21.0
rsa 4.9
s3transfer 0.10.4
Send2Trash 1.8.3
sentencepiece 0.2.0
setuptools 75.6.0
sgmllib3k 1.0.0
shellingham 1.5.4
six 1.16.0
sniffio 1.3.1
sortedcontainers 2.4.0
soupsieve 2.6
SQLAlchemy 2.0.36
stack-data 0.6.3
tabulate 0.9.0
tblib 3.0.0
tenacity 8.5.0
terminado 0.18.1
tiktoken 0.8.0
tinycss2 1.4.0
together 1.3.5
tokenizers 0.20.4
toolz 1.0.0
tornado 6.4.2
tqdm 4.67.1
traitlets 5.14.3
typer 0.13.1
types-python-dateutil 2.9.0.20241003
types-requests 2.32.0.20241016
typing_extensions 4.12.2
typing-inspect 0.9.0
tzdata 2024.2
uri-template 1.3.0
uritemplate 4.1.1
urllib3 2.2.3
wcwidth 0.2.13
webcolors 24.11.1
webencodings 0.5.1
websocket-client 1.8.0
widgetsnbextension 4.0.13
yarl 1.18.0
zict 3.0.0
zipp 3.21.0