[datasets] Add MJSynth (Synth90K) #827
Conversation
Codecov Report
@@ Coverage Diff @@
## main #827 +/- ##
==========================================
+ Coverage 94.70% 94.73% +0.02%
==========================================
Files 133 134 +1
Lines 5442 5465 +23
==========================================
+ Hits 5154 5177 +23
Misses 288 288
Flags with carried forward coverage won't be shown. Click here to find out more.
Continue to review full report at Codecov.
|
fg-mindee
added
module: datasets
There was a problem hiding this comment.
Thanks Felix!
General comments on this:
- I think we should discuss integrating new datasets. There are already quite a lot available through docTR, and we should assess whether it's worth adding one first
- about long downloads, that's an important point (major inconvenience): for reviewers, it means it's unlikely that we can download the whole thing to test it :/
- the dataset is synthetic, so we should check whether it's worth integrating considering there is already one that can generate images on-the-fly
- with the number of datasets, we need to find a good way to document / structure datasets in terms of target task: is this an image with OCR label? text recognition ? text detection? etc
felixdittrich92
force-pushed
the
mjsynth
branch
from
7cb7d81 to
37209b0
Compare
felixdittrich92
force-pushed
the
mjsynth
branch
from
37209b0 to
02a26d3
Compare
felixdittrich92
force-pushed
the
mjsynth
branch
from
02a26d3 to
063b9a6
Compare
|
@fg-mindee How i have wrote you the idea behind this is more for research purpose most model benchmarks (recognition) are created on Mjsynth + SynthText merge for train and svt, ic03, ic13, ... for eval .. so if we want to do something like this we need it 😅 And i agree the refactoring into task splits or better to say task specific totally 👍 |
|
Depends on: #891 so for the moment revert to draft 😃 |
|
Just need to resolve conflicts! |
felixdittrich92
force-pushed
the
mjsynth
branch
from
7f4cbce to
4f45239
Compare
|
@charlesmindee ready to merge 👍 |
This PR integrates the ability to load the MJSynth dataset MJSynth
This one is the first pure recognition dataset with ~ 9M english word images
It´s used in many other repositories to train a recognition model (in some cases in addition with crops from SynthText (maybe prepare this later)) before this will be evaluated on IC03, SVT, SVHN, and so on.
I have decided to integrate it not with the download link in fact that the http download takes ~48hrs and the BitTorrent download is a lot faster.
Any feedback is welcome 🤗
sample:
