I had assumed that if I wanted to have a model predict the values of two or more different labels, that Forests were not the thing to use. In this instance, I meant predicting the value of a single label, but that there are more than two possibilities for the label. So, I mean multi-class here.

I had indeed intended to run multiple classifiers for for the prediction of different labels on the same dataset. For multiple labels, I will experiment with combining them into a single label as the labels will influence each other. Something like 10_222_11, where the three parts are actually different labels. Though I am not sure how well that will work.

Thank you for taking the time to reply. I will look into the possibility of implementing this.

There is multi-class support in YDF/TFDF. Not sure about the Go API support though. I'll check it out later today and see if not, I'll try to add it (is it ok @rstz ?).

About multi-label, sometimes also referred to multi-task (some labels may be classification, others could be regression), in principle it's possible in decision forests, not sure about the status in YDF though. Again the easiest way is to train separate models for each label/task -- so I would recommend just doing this first.

But there are cases where it's beneficial to train together in one tree/forest -- the different tasks help regularize each other. Not sure it is implemented in YDF yet though ? (we discussed this in the past)

A few issues with combining the labels (as you described) that I can see:

• It requires that all examples have every label: for some use cases the examples of each task are different (they have the same features, but the labels are acquired differently), and for each example there is one (or a subset) of labels.
• When reading out, you'll have to separate the probabilities your self: e.g: if the model says you get 50% "10_222_1111" and 50% "10_333_4444", you have to figure out it means "100%: 10; (50%: 222, 50% 333); (50%: 1111; 50% 4444)". Not too hard, but you get the gist.
• It doesn't work with regression. This may not affect you.
• It complicates the split algorithm: let's say you have 2 tasks (labels), and the combined labels of a potential split in the tree are "3_A", "3_D", "3_E" (meaning the first label is always 3, the second is "A", "D" and "E"). Not knowing it, the plain algorithm will find this a bad split, since each label is different. While it should be considered a good split because it clearly classifies the first task -- even if it doesn't help the second. Does it make sense ?

Still, worth a shot trying to combine the labels, if you have enough examples, and not too many labels, it may just work. Let us know how it goes :)

The Go API does not have support for multi-class classification (see here for the relevant part). IIRC the first version only implements engines with a one-dimensional output vector. If you want to take a shot at it @janpfeifer that would be really cool :)

For multi-label problems, YDF offers two modes. Say you have n labels.

• The first mode just trains n independent models - the only possible gain is a speedup during training and inference, but quality will be exactly the same as training the models independently.
• The second mode (added very recently in Version 1.5.0) allows you to specify labels as primary, and others as secondary. The output of the secondary labels is used as input for the primary labels.

Multi-label training was added very recently and is not yet part of the Go API.