For conda I use (and we use in all our production and research system) not Anaconda but miniforge : https://github.com/conda-forge/miniforge

After a few search, it seems it can be related to Bazel version.
What version do you use ?

We're using 5.3.0, though 6.1.0 should work as well

IIUC, the bazel team recommends using https://github.com/bazelbuild/bazelisk which picks the bazel based on what .bazelversion specifies

Succeded with Bazel = 6.3.1, Python 3.10

The problem was due to this combination of versions + the way environment variable are set.

Setting COMPILERS="clang" is different from export COMPILERS="clang" for instance, as it cannot be retrieved with the env dict in the scripts.

Using export make it work.

real	4m31.638s
user	0m0.169s
sys	0m0.174s
+ [[ 0 = 1 ]]
+ echo 'PYDF build / test complete.'
PYDF build / test complete.

Now I have to find where the .whl is and test.

Nice! You can create the wheel with

PY_BINARY=python3.9  # substitute for the python binary you want to build the wheel for
./tools/build_pydf.sh $PY_BINARY

Note that this script packages the compiled binaries from the bazel-bin directories together with the python files and protobuf wrappers and essentially just zip's them into a wheel with setuptools

Thank you!

In fact after double checking, it's not so simple. Maybe it comes from another package, pandas alone is not sufficient to create the issue (installing with conda or pip works when there is only these two packages while I have a bigger config where it fails). I have an environment having the issue but I would like to isolate the minimal combination. Will check it when I will have time and let you know, maybe not soon.

On more question: If I want to use the CLI, how can I compile it and where can I found it ?

On Linux, doing INSTALL_DEPENDENCIES=1 BUILD=1 PACK=1 ./tools/build_binary_release.sh builds the CLI tools and places them (zipped) in dist/cli_linux.zip. Tried on Ubuntu 20.04

On MacOS, installing dependencies is not necessary, so BUILD=1 PACK=1 ./tools/build_binary_release.sh does the trick (tried on Apple M1). The script still assumes bazelisk is available, which you can download with

curl -L -o bazelisk https://github.com/bazelbuild/bazelisk/releases/download/v1.14.0/bazelisk-darwin-arm64
chmod +x bazelisk

You can peek into the tools/build_binary_release.sh script for configuration options, e.g. changing to

  cp -f WORKSPACE_NO_TF WORKSPACE

  BAZEL="./bazelisk"
  FLAGS="--config=linux_cpp17 --config=linux_avx2 --features=-fully_static_link --repo_env=CC=clang-12"

will remove support for Tensorflow data formates but give you faster compilation and probably smaller binaries. You can also change the BAZEL variable to point to your bazel installation.

I'll attach compiled binaries for v.1.8.0 to the github release later today.

I've found the situation producing a segmentation fault.

It comes from pyarrow.

Setup : MacOSX 13 or 14, Mac M hardware

Prerequisite : Install miniforge3

% conda create --name ydfpandasissue
% conda activate ydfpandasissue
% conda install python=3.10
% conda install pandas
% pip install ydf-0.2.0-cp310-cp310-macosx_13_0_arm64.whl

When running this program (ydf_test.py), it works.

import ydf
import pandas as pd
import numpy as np

dataset = {
    "x1": np.array([0, 0, 0, 1, 1, 1]),
    "x2": np.array([1, 1, 0, 0, 1, 1]),
    "y": np.array([0, 0, 0, 0, 1, 1]),
}

model = ydf.CartLearner(label="y", min_examples=1, task=ydf.Task.CLASSIFICATION).train(dataset)
print(model.describe())

Now install pyarrow from conda or pip the result is the same, only the error message is different: it fails

% conda install pyarrow
% python ydf_test.py
zsh: segmentation fault  python ydf_test.py

% conda uninstall pyarrow
% pip install pyarrow
% python ydf_test.py
libc++abi: terminating due to uncaught exception of type std::__1::system_error: mutex lock failed: Invalid argument
zsh: abort      python ydf_test.py

Something strange : when trying from another user account on my mac (Perso vs Pro), it works. I'm investigating.

It works now, after recreating my conda compilation environment instead of uprading the python version of the previous one.

Hi,

yes, we plan to release for MacOS Arm64 very soon (and possibly update the release instructions). AFAICT, once we change the linkopts in yggdrasil_decision_forests/port/python/ydf/cc/BUILD to

pybind_extension(
    name = "ydf",
    srcs = ["ydf.cc"],
    deps = [
        "//ydf/dataset:dataset_cc",
        "//ydf/learner:learner_cc",
        "//ydf/learner:worker_cc",
        "//ydf/metric:metric_cc",
        "//ydf/model:model_cc",
        "//ydf/utils:log_cc",
        "@pybind11_protobuf//pybind11_protobuf:native_proto_caster",
    ],
    linkopts = select({
        "@bazel_tools//src/conditions:darwin": ["-Wl,-exported_symbol,_PyInit_ydf"],
        "//conditions:default": [],
    }),
)

compiling and interplay with pyarrow / tensorflow / tensorflow-decision-forest / protobuf etc. should work.

Update: Support for MacOS has been restored with version 0.7.0

Thanks a lot!

If anyone encounters the same issue as I did, if you try to install it in a conda environment it might not work, as conda by default created an osx-64 env.

To fix it, create the env as follows:

CONDA_SUBDIR=osx-arm64 conda create -n my_env python