what's influence of model ensemble

compare to test-time augmentation? which one will be better?

Well, I see, the model ensembling method is actually more like using a poor model to find missed detections for a good model. In contrast, TTA can also find missed detections by changing the input, while maintaining using the best model.

@Zzh-tju ensembling and TTA are not mutually exclusive. You can TTA a single model, and you can ensemble a group of models with or without TTA:

python detect.py --weights model1.pt model2.pt --augment

@Zzh-tju ensembling runs multiple models, while TTA tests a single model at with different augmentations. Typically I've seen the best result when merging output grids directly, (i.e. ensembling YOLOv5l and YOLOv5x), rather than simply appending boxes from multiple models for NMS to sort out. This is not always possible however, for example Ensembling an EfficientDet model with YOLOv5x, you can not merge grids, you must use NMS or WBF (or Merge NMS) to get a final result.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

How can I ensemble EfficientDet D7 with YOLO V5x?

@Blaze-raf97 with the right amount of coffee anything is possible.

How to solve this problem?
COCO mAP with pycocotools... saving detections_val2017__results.json...
ERROR: pycocotools unable to run: invalid literal for int() with base 10: 'Image_20200930140952222'

@LokedSher pycocotools is only intended for mAP on COCO data using coco.yaml. https://pypi.org/project/pycocotools/

@LokedSher pycocotools is only intended for mAP on COCO data using coco.yaml. https://pypi.org/project/pycocotools/

Thanks for your reply!

@LokedSher I also encountered the same problem as you, but after I read your Q&A, I still don't know how to improve to get the picture given by the author.

I want to ensemble two yolov5x6 models trained on the same data with some variation
I saw the Ensemble() class mentioned in above issues but I was a bit confused on how to implement it in a python script

In other words, how do I exactly use that Ensemble in order to create an ensemble of my 2 models within a python script/function just by passing the models?

@pathikg YOLOv5 ensembling is automatically built into detect.py and val.py, so simply pass two weights:

python detect.py --weights yolov5s.pt yolov5m.pt

@pathikg YOLOv5 ensembling is automatically built into detect.py and val.py, so simply pass two weights:

python detect.py --weights yolov5s.pt yolov5m.pt

Thanks @glenn-jocher for quick reply but I want to do this in a python script
At present, I am loading model from torch hub with my custom weights and then doing the inference on respective images.
At present I've two such models, and I want to make ensemble of the same so is there any way I can do that as well?

I’d follow the code in detect.py and use the Ensemble() module from models/common.py

You mean from models/experimental.py?
cuz there's no Ensemble() module present in common.py :/

yes sorry in experimental.py

@glenn-jocher I really need your help for one of my problems.
I have a dataset trained on cat, dog and pen, and after training, I have got dataset1.pt best file
Now, I trained another model with new data, for example I have taken just images of cow, and got dataset2.pt best file.
Both of the trained model can detect the images separately. But I want to make them one model, so that it will detect all the images (cat, dog, pen and cow) using a single weight file.
Can I do it using ensemble techniques?
will this work?
python detect.py --weights dataset1.pt dataset2.pt --img 640 --source data/images/cat
or how can I do this, is there any way, rather than creating a new dataset will all the images, and trained again. Please reply, thanks

@Zzh-tju ensembling and TTA are not mutually exclusive. You can TTA a single model, and you can ensemble a group of models with or without TTA:

python detect.py --weights model1.pt model2.pt --augment

Hi @glenn-jocher , what would be the best practice for deploying an ensemble model like this? I know we can export the individual models for different deployment frameworks, but how would I export an ensemble?

Hello, I read a previous discussion about ensembling multiple trained networks by simply passing more than 1 weight file during inference.

My question is what technique is being used for fusing the predictions? Is it majority voting on the bounding boxes? Some weighted averaging?

I ended up predicting each image with all of the ensemble members separately, and then combining the bounding box predictions together and doing a second stage NMS to generate a final combined prediction. Seems to work OK. Another possibility is to average the weights of the ensemble members into an averaged model like they do in federated learning, but I have not properly evaluated that method yet.

@michael-mayo that's a good approach! Typically, the ensembling technique involves averaging the model weights and biases instead of the predictions. Here, you are combining the predictions which can be achieved using the NMS algorithm. Keep in mind that it's important to experiment and choose the best method based on the specific problem and the performance of each approach.

I should also add that for each ensemble member I trained using a different global random seed, and a different (5/6) subset of the training data, to improve ensemble diversity.

@michael-mayo That's a great technique to improve ensemble diversity. It can help reducing the chances of overfitting (which can happen if all ensemble members are trained on exactly the same data) and increase the robustness of the final predictions.

Hi @glenn-jocher I have a question: Suppose we train an object detection model separately on two completely separate datasets (datasets A and B) so that the classes are the same in both datasets (for example usask and avarlis in Wheat Head Detection).
After training the same model on 2 datasets, we will have 2 different best.pt files.
Question: How can we combine (or aggregate) these 2 model weights to have a single .pt file including the aggregated model weights?

Hi @mek651 this can be done straightfowardly by loading each model, getting the state_dict for each model (which is a sequence of arrays or tensors I believe), doing a straightforward average of the two state_dicts, then deep copying one the models and assigning the averaged state_dict and saving it. In yolo this is only going to make sense if the classes are exactly the same though. You might get better results by training one larger model on both datasets at the same time though.

Thanks @michael-mayo.
Actually, my goal is to use Federated Learning. the simplest version I am working on is to apply FedAvg model on the output weights (2 or more best.pt). the classes of both datasets are exactly the same.

Do you have any idea about this?

Thanks for the relevant discussion, I am new to YOLOv5 and this platform. I wonder if there is a way to make ensemble training on two different datasets, with one frozen pre-trained model and one from scratch?

Hello,

I did not see the python implementation for detect tasks. Is it same as the predict task?

python detect.py --weights yolov5s.pt yolov5m.pt
If not, how can I run the detect task using python implementation?

Thank you.

Regards,
Bijay

@glenn-jocher I really need your help for one of my problems. I have a dataset trained on cat, dog and pen, and after training, I have got dataset1.pt best file Now, I trained another model with new data, for example I have taken just images of cow, and got dataset2.pt best file. Both of the trained model can detect the images separately. But I want to make them one model, so that it will detect all the images (cat, dog, pen and cow) using a single weight file. Can I do it using ensemble techniques? will this work? python detect.py --weights dataset1.pt dataset2.pt --img 640 --source data/images/cat or how can I do this, is there any way, rather than creating a new dataset will all the images, and trained again. Please reply, thanks

do you solve this？diffrent number of classes will get the error
assert all(model[0].nc == m.nc for m in model), f"Models have different class counts: {[m.nc for m in model]}"
AssertionError: Models have different class counts: [36, 80]

Actually in the ensembling technique I want to use the majority voting one the in this inference or test what type of aggregration is used and how can we use the majority voting python val.py --weights yolov5x.pt yolov5l6.pt --data coco.yaml --img 640 --half

what kind of specific resembling technique is used when we're doing the ensembling with this

python val.py --weights yolov5x.pt yolov5l6.pt --data coco.yaml --img 640 --half