My apologies for the multiple commits in this pull request. After several attempts, I was unable to figure out how to combine them all into one commit. I'll have to improve my git skills so that I can do this properly in the future.

You don't have to squash your commits here, in fact I believe it is preferred not to. Whoever merges your PR will squash them for you.

However, I believe what you should do if you wanted to squash is:
git rebase -i master
and then pick the first commit and squash all the others.

Thanks @hoodmane, sounds good. Thanks for the git command as well, I wasn't using the rebase command correctly.

Thanks @mgreminger !

I did have to bypass the cmake build due to incompatibilities between a cmake build and the setup.py type build that pywasmcross.py expects.

Following #1027 would you be able to build it directly with cmake under the build/script section of meta.yaml (see examples in that PR)?

@rth, good point, the build/script tag adds powerful capability to the build system. However, it won't work as-is without additional modifications to buildpkg.py. The scripts are only called if library is set to true. buildpkg.py would have to be updated to call the scripts if they're defined for a non-library package. The handling of the .package would need to be updated as well if the scripts are being executed for a non-library packages. It's definitely doable and would make it easier to build python packages that rely on complex external build systems. I'll take a look and see what I can do to make it work for this package, seems like a good test vehicle for this. I'm not 100% sure if it will build with cmake using the emscripten toolchain. It might work, but I've found that cmake and the emscripten toolchain don't always work well together (especially with the version of emscripten that we're using). If it won't build with cmake, we can always revert back to this version of the nlopt package and we'll still have the build/script tag option available for python packages to use.

Sounds great .

we can always revert back to this version of the nlopt package and we'll still have the build/script tag option available for python packages to use

Probably better to do that feature in a separate PR in any case..

@rth, after taking a look at this, It's looking like bypassing cmake for the build step is the best approach for this package. I still use emcmake cmake to configure the package to make sure it is setup correctly for the 32 bit wasm target environment.

I did do a test implementation of script support for non-library packages but as I was trying to get this package to work with cmake, I was splitting build logic between the script section of meta.yaml and setup.py (a custom setup.py cannot be avoided for this package), which would make maintenance of this package more difficult. It makes since to use the script tag for library packages as introduced in #1027 since that handles the entire build. The more I think about it, the current behavior of not using script for python packages makes the most sense. setuptools is a powerful build environment and any custom build logic can be obtained by extending the build_ext class. I think it would make maintenance of Python packages more difficult if we have build logic in both setup.py and meta.yaml for python packages.

In the end, I couldn't get cmake to run the actual build step. When envoking swig, cmake wants to know the Python lib path even though nothing is ever linked against the python libraries. This creates the problem that cmake wants the Python version between pyodide and the host system to match down the micro version level. In the end, setuptools is better equiped to build a package that depends on swig for the python bindings than cmake is.

Thanks for investigating @mgreminger ! Then this approach sound the easiest way indeed. I'm a bit concerned about how to keep that list of sources files/includes in setup.py up to date, but I suppose trying to automate would also add more complexity. Let's merge as is, thanks!

I did do a test implementation of script support for non-library packages

If you still have it, it would be a valuable contribution in general (in a different PR).

Thanks, @rth. Yes, I see your point about the long list of source files. I might be able to get it to work with a simple Path.glob call since I think it's basically including all of the .c and .cc files from the nlopt tarball. I'll investigate and create a pull request if I can simplify the setup.py.

Yes, I'll create a separate PR for the build/script capability for Python packages. It's a pretty small change.

I took a quick look at whether it would be better to try and automatically extract the source file list for the setup.py. There are 16 sources files that are in the nlopt tarball under the ./src directory that don't get compiled into the final library by the default cmake build and there's no clear pattern to exclude them. It doesn't look like there's a better option than just listing them.

Thanks for checking, not too surprising.

Also it looks like there is a pip package, but it's not maintained in the same repo nor by the same maintainers stevengj/nlopt#282 (comment) ? It's a bit confusing. Maybe those maintainers also have a setup.py or would be interested in having this setup contributed there?

Yes, I did start by attempting to use the official pip package setup.py file. The main issue is that they compile using cmake and that doesn't work with pywasmcross.py Some of the options that cmake uses for a gcc build break emscripten builds. Plus pywasmcross.py doesn't work when the build tool changes working directories (the build.log files end up in places where pywasmcross.py isn't looking for them), which cmake does. It became simpler just to rewrite the setup.py file.