Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Numba #5592

Draft
wants to merge 39 commits into
base: main
Choose a base branch
from
Draft

Add Numba #5592

wants to merge 39 commits into from

Conversation

Rupt
Copy link
Contributor

@Rupt Rupt commented Oct 11, 2023

  • Add initial support for Numba compilation: asm, demangling, execution

Numba wraps Python functions in Dispatcher objects. Each dispatcher contains zero or more compiled argument-type-indexed overloads of its function. We import the user's code as a module, and emit the code from all overloads of all dispatchers that the module publicly exposes.

Name mangling is odd in Numba. It uses a similar syntax to C++, but also encodes non-symbol (r"[^a-z0-9_]") characters as _%02x. This encoding yields valid identifier names, so it is not strictly invertible. Here, I have hard-coded some replacements to decode some common cases.

Example with execution of a generator hehe bad rng

To do

Locally, I have installed a virtual environment with

python3.11 -m venv venv_numba
venv_numba/bin/python -m pip install numba==0.58.0 llvmlite==0.41.0 scipy==1.11.3 numpy==1.25.2
# (I recommend pinning versions because Python packages regularly violate SemVer, and CE wants long-term reproducibility.)

and configure its use with

# compiler-explorer/etc/config/numba.local.properties
compilers=&numba
group.numba.compilers=numba58
compiler.numba58.exe=/home/rupt/venv_numba/bin/python3.11
  • Automatically run python tests?

Locally, I run this test suite with

/home/rupt/venv_numba/bin/python3.11 -m unittest etc/scripts/test_numba_wrapper.py

@github-actions github-actions bot added the ui label Oct 11, 2023
@Rupt Rupt mentioned this pull request Oct 11, 2023
3 tasks
@mattgodbolt
Copy link
Member

I like the idea of this! When we were using numba at work I hacked something similar. But, the main issue is the python installation, which a number of things are blocked on: virtual environments etc are troublesome to create and maintain over multiple OSes, and our installation system is not good with them: we install to a local directory and then relocate atomically to the "install" location at the moment and many python virtual environments don't work with this That's not to say we don't want this at all! I'm excited; but we have some infrastructural things to fix on our side!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants