Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add explicit train!, unify update!, and auto-translate the two Adams #2082

Merged
merged 18 commits into from
Nov 20, 2022
Merged

Add explicit train!, unify update!, and auto-translate the two Adams #2082

Changes from 1 commit
Commits
Show all changes
18 commits
Select commit Hold shift + click to select a range
c20fb9e
explicit train, take 2
mcabbott Oct 13, 2022
9c22c11
remove train_autodiff macro
mcabbott Oct 16, 2022
fa022b3
make it stricter, to avoid batchmaybe weirdness
mcabbott Oct 16, 2022
4e937df
remove 3-argument train! since this requires impure loss function, an…
mcabbott Oct 16, 2022
36a2bb5
remove issingletontype purity check, too strict
mcabbott Oct 17, 2022
28a7718
tidy up
mcabbott Oct 17, 2022
5d74b04
fix tests
mcabbott Oct 17, 2022
7eaf3ea
use _old_to_new in Optimisers.setup too
mcabbott Nov 10, 2022
f3e1559
oops
mcabbott Nov 10, 2022
63ad543
return nothing
mcabbott Nov 18, 2022
2bd0dad
test NaN + error, tidy up
mcabbott Nov 18, 2022
20326ea
fix test
mcabbott Nov 20, 2022
db2a9b9
remove 2 vs 3 argument comment from docstring
mcabbott Nov 20, 2022
c617807
nice errors for update! with mixed-up input
mcabbott Nov 20, 2022
0389de3
fix doctest by making "using Statistics" explicit
mcabbott Nov 20, 2022
732fa13
also delete Flux.params() from the example completely
mcabbott Nov 20, 2022
d9699c0
fix
mcabbott Nov 20, 2022
db7ad43
fix
mcabbott Nov 20, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
remove 2 vs 3 argument comment from docstring
  • Loading branch information
@mcabbott
mcabbott committed Nov 20, 2022
commit db2a9b90584c8fe5b4b511e2478ea350f4b9b2bf
14 changes: 6 additions & 8 deletions src/train.jl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,8 +16,10 @@ using Zygote: Zygote, Params

This is a version of `Optimisers.setup`, and is the first step before using [`train!`](@ref Flux.train!).
It differs from `Optimisers.setup` in that it:
* has one extra check for mutability
* has one extra check for mutability (since Flux expects to mutate the model in-place,
while Optimisers.jl is designed to return an updated model)
* has methods which accept Flux's old optimisers, and convert them.
(The old `Flux.Optimise.Adam` and new `Optimisers.Adam` are distinct types.)

# Example
```jldoctest
Expand Down Expand Up @@ -80,16 +82,12 @@ It adds only a few featurs to the loop above:

* Show a progress bar using [`@withprogress`](https://github.com/JuliaLogging/ProgressLogging.jl).

Note that the built-in loss functions accept 3 arguments, allowing for instance
`train!(Flux.Losses.mse, model, data, opt)` instead of defining `loss3` as above.

!!! note
This method has significant changes from the one in Flux ≤ 0.13:
* It now takes the `model` itself, not the result of [`Flux.params`](@ref).
(This is to move away from Zygote's "implicit" parameter handling, with `Grads`.)
* Instead of `loss` being a function which typically accepts two arguments
(the input `x` and expected output `y` from each element of `data`)
now it should typically accept three, the first of which is the `model` itself.
* Instead of `loss` being a function which accepts only the data,
now it must also accept the `model` itself, as the first argument.
* `data` must iterate tuples, otherwise you get an error.
(Previously non-tuple types were not splatted into the loss.
Pass in `((d,) for d in data)` to simulate this.)
Expand Down Expand Up @@ -130,4 +128,4 @@ function _rule_to_state(model, rule::Optimisers.AbstractRule)
state
end

end # module
end # module Train