Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Error when submit result to semantic scene completion benchmark #26

Closed
sj-li opened this issue Dec 19, 2019 · 6 comments
Closed

Error when submit result to semantic scene completion benchmark #26

sj-li opened this issue Dec 19, 2019 · 6 comments

Comments

@sj-li
Copy link

sj-li commented Dec 19, 2019

Hi:

I submit my result to semantic scene completion, but get following erroe:

Traceback (most recent call last):
  File "/worker/worker.py", line 313, in run
    bundles = get_bundle(root_dir, 'run', bundle_url)
  File "/worker/worker.py", line 173, in get_bundle
    metadata[k] = get_bundle(bundle_path, k, v)
  File "/worker/worker.py", line 173, in get_bundle
    metadata[k] = get_bundle(bundle_path, k, v)
  File "/worker/worker.py", line 133, in get_bundle
    z.extractall(bundle_path)
  File "/usr/local/lib/python2.7/zipfile.py", line 1040, in extractall
    self.extract(zipinfo, path, pwd)
  File "/usr/local/lib/python2.7/zipfile.py", line 1028, in extract
    return self._extract_member(member, path, pwd)
  File "/usr/local/lib/python2.7/zipfile.py", line 1084, in _extract_member
    shutil.copyfileobj(source, target)
  File "/usr/local/lib/python2.7/shutil.py", line 66, in copyfileobj
    fdst.write(buf)
IOError: [Errno 28] No space left on device

However, my zip file pass through the test by the script offered by you.
What should I do?
@jbehley
Copy link
Member

jbehley commented Dec 19, 2019

Hi @sj-li,

Thanks for reporting this issue and I looked into this.

Your submission file is unfortunately to large. You have to provide only the predictions for the scans in the testset. (A submission is something around 120-150 MB zipped and 16 GB unzipped.)

Please ensure that the file containes only labels for the files from the testset, i.e., 11/000000.bin, 11/000005.bin, 11/000010.bin, etc.

I will modify the verification script to check that a zip file contains only the necessary files.

jbehley added a commit that referenced this issue Dec 19, 2019
@jbehley
check for unnecessary files (addresses issue #26)
4036c93
@jbehley
Copy link
Member

jbehley commented Dec 19, 2019 •
edited
Loading

Please consult the official zip file: http://www.semantic-kitti.org/assets/data_odometry_voxels.zip

The test set contains every 5th scan. Therefore your submission.zip should also only contain this data.

I updated the validation script to handle these cases.

@jbehley
Copy link
Member

jbehley commented Dec 19, 2019

I deleted your failed submissions on Codalab, since it was an issue from our side. Hope that helps.

@sj-li
Copy link
Author

sj-li commented Dec 19, 2019

Got it! I will try it now.

And I have another question, I see in evaluation code, you use:

    # Map labels "pred_labels" and "gt_labels" from semantic-kitti ID's to [0 : n_classes -1]
    pred = remap_lut[pred]
    target = remap_lut[target]

But my predictions already only contains [0 : n_classes -1], So I need to adjust label by myself?

@jbehley
Copy link
Member

jbehley commented Dec 19, 2019

The submission should contain our original labels (car = 10, etc.). How you map the data for training is an implementation issue that you have to figure out.

We expect to get the original label ids, therefore you have to remap the labels before submission.

@sj-li
Copy link
Author

sj-li commented Dec 19, 2019

Thanks for your help, I submit successful ;)

@sj-li sj-li closed this as completed Dec 19, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@sj-li @jbehley