💡 [REQUEST] - Run model v2.6 on Apple Silicon with mps-Pytorch Backend #558
Open
Description
起始日期 | Start Date
No response
实现PR | Implementation PR
No response
相关Issues | Reference Issues
No response
摘要 | Summary
I would like to run the model with Pytorch on my Macbook (in this case M2 Pro) with mps to have greater control about the prompting. Inspiration is the v2.5 model.
基本示例 | Basic Example
Current issue, when trying is:
NotImplementedError: The operator 'aten::isin.Tensor_Tensor_out' is not currently implemented for the MPS device. If you want this op to be added in priority during the prototype phase of this feature, please comment on pytorch/pytorch#77764. As a temporary fix, you can set the environment variable PYTORCH_ENABLE_MPS_FALLBACK=1 to use the CPU as a fallback for this op. WARNING: this will be slower than running natively on MPS.
缺陷 | Drawbacks
No, I can think of
未解决问题 | Unresolved questions
No response