Finish PR #2432: Imagenet example updates + basic testing #2889

awaelchli · 2020-08-08T16:20:09Z

What does this PR do?

Finishes an old PR #2432 by @ruotianluo

standardize with other examples
a test that runs for 1 epoch making sure the script can run
simplifications

pep8speaks · 2020-08-08T16:20:13Z

Hello @awaelchli! Thanks for updating this PR.

In the file pl_examples/domain_templates/imagenet.py:

Line 209:35: W503 line break before binary operator

Comment last updated at 2020-08-09 02:48:45 UTC

codecov · 2020-08-08T16:35:16Z

Codecov Report

Merging #2889 into master will not change coverage.
The diff coverage is n/a.

@@          Coverage Diff           @@
##           master   #2889   +/-   ##
======================================
  Coverage      90%     90%           
======================================
  Files          79      79           
  Lines        7236    7236           
======================================
  Hits         6530    6530           
  Misses        706     706

Borda · 2020-08-08T17:53:11Z

You can still use this one... =)

…when ddp (#2432) * fix imagenet example: lr_scheduler, loader workers, batch size when ddp * Fix evaluation for imagenet example * add imagenet example test * cleanup * gpu * add imagenet example evluation test * fix test output * test is fixed in master, remove unecessary hack * CHANGE * Apply suggestions from code review Co-authored-by: Jirka <jirka@pytorchlightning.ai> Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>

nateraw

LGTM 🚀 Nice work as always, Adrian.

nateraw · 2020-08-09T05:38:02Z

pl_examples/test_examples.py

+    with mock.patch("argparse._sys.argv", ["any.py"] + cli_args):
+        run_cli()
+
+
 # @pytest.mark.parametrize('cli_args', ['--max_epochs 1 --max_steps 3 --num_nodes 1 --gpus 2'])


Can we remove these comments if we don't need them?

Yeah I was wondering myself. They are outdated and have to be rewritten anyway if we ever get multi-node ddp testing.

nateraw · 2020-08-09T05:38:38Z

pl_examples/domain_templates/imagenet.py

@@ -1,13 +1,27 @@
 """
 This example is largely adapted from https://github.com/pytorch/examples/blob/master/imagenet/main.py
+


This is great info, thanks.

ruotianluo · 2020-09-20T00:59:15Z

It seems the current --evaluate won't load the checkpoint from --resume_from_checkpoint, right?

I saw in the doc:

# (4) test with an explicit model (will use this model and not load a checkpoint)
traiener.test(model)

awaelchli · 2020-09-20T01:22:57Z

oh yes, I think you are right. Would you like to send a PR?

ruotianluo · 2020-09-20T01:47:18Z

I am not sure what is the most elegant solution:
One possibility would just be:

model.load_state_dict(torch.load(args.resume_from_checkpoint, map_location=''cpu)['state_dict'])

But I think it's too ugly.

awaelchli · 2020-09-20T01:48:52Z

we could simply pass the checkpoint path to .test
trainer.test(model, ckpt_path="...") # default is ckpt_path="best", but that's not available
what do you think?

ruotianluo · 2020-09-20T01:50:27Z

https://github.com/PyTorchLightning/pytorch-lightning/blob/8eb77cd06aff18c14242e194f6959752a9b08817/pytorch_lightning/trainer/trainer.py#L538

It won't use the ckpt_path

awaelchli · 2020-09-20T01:52:52Z

ok, correction, then trainer.test(ckpt_path="...") without model? :)

ruotianluo · 2020-09-20T01:54:28Z

Doesn't that assume trainer already has the model? My understanding, if you want to use this, you have to run it after trainer.fit(model) otherwise trainer doesn't know what is the model.

awaelchli · 2020-09-20T01:57:16Z

we could load the state at the beginning, like basically as you proposed, but with all additional params

if args.evaluate:
    model = ImageNetLightningModel.load_from_checkpoint(path, map_location=...)

awaelchli added bug Something isn't working example docs Documentation related labels Aug 8, 2020

awaelchli added this to the 0.9.0 milestone Aug 8, 2020

mergify bot requested a review from a team August 8, 2020 16:20

awaelchli mentioned this pull request Aug 8, 2020

[wip] fix imagenet example: lr_scheduler, loader workers, batch size when ddp #2432

Merged

awaelchli changed the title ~~Finish PR #2432: Imagenet example updates + testing~~ Finish PR #2432: Imagenet example updates + basic testing Aug 8, 2020

awaelchli closed this Aug 8, 2020

Borda reopened this Aug 8, 2020

ruotianluo and others added 18 commits August 8, 2020 20:08

fix imagenet example: lr_scheduler, loader workers, batch size when ddp

5cefb45

Fix evaluation for imagenet example

38abf7b

add imagenet example test

c8c6813

cleanup

39ecb44

gpu

d5cb895

add imagenet example evluation test

851c584

fix test output

9f87199

test is fixed in master, remove unecessary hack

939e8c9

CHANGE

757194e

Apply suggestions from code review

1a85610

image net example

6fc12d4

update imagenet example

3630c08

update example

533d7d7

pep

864edfe

imports

48cf50e

type hint

d0c91ad

docs

934cea2

obsolete arg

ef2839c

awaelchli force-pushed the imagenet_example branch from 74a3182 to ef2839c Compare August 8, 2020 18:10

ruotianluo and others added 5 commits August 8, 2020 20:14

update chlog

c028641

add missing chlog

a49c533

pep

dc51b14

pep

fa3dec1

awaelchli marked this pull request as ready for review August 9, 2020 02:55

nateraw approved these changes Aug 9, 2020

View reviewed changes

ananyahjha93 approved these changes Aug 9, 2020

View reviewed changes

mergify bot requested a review from a team August 9, 2020 07:53

ananyahjha93 added the ready PRs ready to be merged label Aug 9, 2020

williamFalcon merged commit 6ad2718 into master Aug 9, 2020

Borda deleted the imagenet_example branch August 9, 2020 10:31

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Finish PR #2432: Imagenet example updates + basic testing #2889

Finish PR #2432: Imagenet example updates + basic testing #2889

awaelchli commented Aug 8, 2020

pep8speaks commented Aug 8, 2020 •

edited

Loading

codecov bot commented Aug 8, 2020 •

edited

Loading

Borda commented Aug 8, 2020

nateraw left a comment

nateraw Aug 9, 2020

awaelchli Aug 9, 2020

nateraw Aug 9, 2020

ruotianluo commented Sep 20, 2020

awaelchli commented Sep 20, 2020

ruotianluo commented Sep 20, 2020

awaelchli commented Sep 20, 2020

ruotianluo commented Sep 20, 2020

awaelchli commented Sep 20, 2020

ruotianluo commented Sep 20, 2020

awaelchli commented Sep 20, 2020 •

edited

Loading

		@@ -1,13 +1,27 @@
		"""
		This example is largely adapted from https://github.com/pytorch/examples/blob/master/imagenet/main.py

Finish PR #2432: Imagenet example updates + basic testing #2889

Finish PR #2432: Imagenet example updates + basic testing #2889

Conversation

awaelchli commented Aug 8, 2020

What does this PR do?

pep8speaks commented Aug 8, 2020 • edited Loading

Comment last updated at 2020-08-09 02:48:45 UTC

codecov bot commented Aug 8, 2020 • edited Loading

Codecov Report

Borda commented Aug 8, 2020

nateraw left a comment

Choose a reason for hiding this comment

nateraw Aug 9, 2020

Choose a reason for hiding this comment

awaelchli Aug 9, 2020

Choose a reason for hiding this comment

nateraw Aug 9, 2020

Choose a reason for hiding this comment

ruotianluo commented Sep 20, 2020

awaelchli commented Sep 20, 2020

ruotianluo commented Sep 20, 2020

awaelchli commented Sep 20, 2020

ruotianluo commented Sep 20, 2020

awaelchli commented Sep 20, 2020

ruotianluo commented Sep 20, 2020

awaelchli commented Sep 20, 2020 • edited Loading

pep8speaks commented Aug 8, 2020 •

edited

Loading

codecov bot commented Aug 8, 2020 •

edited

Loading

awaelchli commented Sep 20, 2020 •

edited

Loading