Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Future plan to train the model #2

Open
Conless opened this issue Nov 9, 2024 · 3 comments
Open

Future plan to train the model #2

Conless opened this issue Nov 9, 2024 · 3 comments
Assignees
@VachanVY
Labels
enhancement New feature or request

Comments

@Conless
Copy link

Conless commented Nov 9, 2024

Hi, thanks for the great work! I'm wondering if there are any plans to train the model on some datasets in the near future. Since there are currently no released weights for this model architecture, it would be incredibly helpful if training could be done, even at a smaller scale. This would allow the community to experiment further and potentially build on this architecture.
@VachanVY
Copy link
Owner

VachanVY commented Nov 9, 2024

Hi, yes I plan to train in it on the coco captions dataset, soon! Do you know any story generation datasets that contain both image and text modality (it should not be very large, just like we have MNIST for CV)?

@VachanVY VachanVY self-assigned this Nov 9, 2024
@VachanVY VachanVY added the enhancement New feature or request label Nov 9, 2024
@Conless
Copy link
Author

Conless commented Nov 9, 2024

Hi, yes I plan to train in it on the coco captions dataset, soon! Do you know any story generation datasets that contain both image and text modality (it should not be very large, just like we have MNIST for CV)?

Yes I have found a small one: https://huggingface.co/datasets/sil-ai/bloom-vist

@VachanVY
Copy link
Owner

@Conless, could you prepare a preprocessing script for the dataset (please refer to the README for how the inputs are arranged for the model) and send a PR? If you're okay with it, otherwise I can start working on it in a couple of days as I’m busy. :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@VachanVY VachanVY

Labels
enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@Conless @VachanVY