Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Bugfix: accumulation and suggestion for learning rate finder #1801

Merged
merged 10 commits into from
May 13, 2020
Merged

Bugfix: accumulation and suggestion for learning rate finder #1801

merged 10 commits into from
May 13, 2020

Conversation

SkafteNicki
Copy link
Member

@SkafteNicki SkafteNicki commented May 12, 2020 •
edited
Loading

Before submitting

  • Was this discussed/approved via a Github issue? (no need for typos and docs improvements)
  • Did you read the contributor guideline, Pull Request section?
  • Did you make sure to update the docs?
  • Did you write any new necessary tests?
  • If you made a notable change (that affects users), did you update the CHANGELOG?

What does this PR do?

Fixes #1767, by implementing a skip_begin parameter (default 10) for the lr_finder.suggestion method

Fixes #1726, by only logging learning rate and loss when optimizer is called (as it should be). This also removes num_accumulation_steps since this is a duplicate of the accumulate_grad_batches trainer arg.

Added tests for both.

PR review

Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.

Did you have fun?

Make sure you had fun coding 🙃

@mergify mergify bot requested a review from a team May 12, 2020 14:45
@Borda Borda added the bug Something isn't working label May 12, 2020
Borda
Borda reviewed May 12, 2020
View reviewed changes
pytorch_lightning/trainer/lr_finder.py Outdated Show resolved Hide resolved
pytorch_lightning/trainer/lr_finder.py Outdated Show resolved Hide resolved
pytorch_lightning/trainer/lr_finder.py Show resolved Hide resolved
pytorch_lightning/trainer/lr_finder.py Outdated Show resolved Hide resolved
@mergify mergify bot requested a review from a team May 12, 2020 21:10
@williamFalcon
Copy link
Contributor

@SkafteNicki want to sneak this last minute into 0.7.6? we're releasing the rc2 in the next hour

@williamFalcon
Copy link
Contributor

need to get these tests to pass

@Borda
Copy link
Member

Borda commented May 13, 2020

need to get these tests to pass

this is fixed In master now, just rebase it...

@Borda Borda force-pushed the bugfix/lr_finder2 branch from af08018 to 02ba1fe Compare May 13, 2020 16:53
Borda
Borda reviewed May 13, 2020
View reviewed changes
pytorch_lightning/trainer/lr_finder.py Outdated Show resolved Hide resolved
pytorch_lightning/trainer/lr_finder.py Outdated Show resolved Hide resolved
@mergify mergify bot requested a review from a team May 13, 2020 17:22
@mergify mergify bot requested a review from a team May 13, 2020 17:25
Borda
Borda approved these changes May 13, 2020
View reviewed changes
pytorch_lightning/trainer/lr_finder.py Outdated Show resolved Hide resolved
pytorch_lightning/trainer/lr_finder.py Outdated Show resolved Hide resolved
pytorch_lightning/trainer/lr_finder.py Outdated Show resolved Hide resolved
pytorch_lightning/trainer/lr_finder.py Outdated Show resolved Hide resolved
@mergify mergify bot requested a review from a team May 13, 2020 17:44
@mergify mergify bot requested a review from a team May 13, 2020 18:16
@williamFalcon williamFalcon merged commit 663b900 into Lightning-AI:master May 13, 2020
@SkafteNicki SkafteNicki deleted the bugfix/lr_finder2 branch June 15, 2020 14:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Borda Borda Borda approved these changes

Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
3 participants
@SkafteNicki @williamFalcon @Borda