Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

User defined model does not work #597

Closed
rs1909 opened this issue Feb 5, 2019 · 5 comments
Closed

User defined model does not work #597

rs1909 opened this issue Feb 5, 2019 · 5 comments

Comments

@rs1909
Copy link

rs1909 commented Feb 5, 2019

I have defined a model that is a multivariate polynomial. For some reason Flux.jacobian returns zero no matter what. ForwardDiff does not work either.

ForwardDiff gives:
ERROR: LoadError: TypeError: in typeassert, expected Float64, got ForwardDiff.Dual{Nothing,Float64,4}
I believe something is not being tracked, but I have no clue what.

I have posted the issue with a slightly different code here: https://discourse.julialang.org/t/flux-user-defined-layer-problem/20455?u=fastwave

The code is

using Flux
using LinearAlgebra
using DynamicPolynomials
using MultivariatePolynomials

struct PolyModel{S,T}
    mexp::S
    W::T
end

function PolyModel(in::Integer, out::Integer, order::Integer)
    @polyvar x[1:in]
    mx0 = monomials(x, 0:order)
    mx1 = [exponents(mx0[k]) for k=1:length(mx0)]
    return PolyModel(mx1, param(randn(out, length(mx0))))
end

function (a::PolyModel)(z::AbstractArray)
    mon = ones(length(a.mexp))
    for j=1:length(a.mexp)
        for k=1:length(z)
            for p=1:a.mexp[j][k]
                mon[j] *= z[k]
            end
        end
    end
    return a.W*mon
end

Flux.@treelike PolyModel

a = PolyModel(2,2,2)
j1 = Flux.jacobian(a,zeros(2))
j2 = ForwardDiff.jacobian(a,zeros(2))
@KristofferC
Copy link
Contributor

@rs1909
Copy link
Author

rs1909 commented Feb 6, 2019

Thanks for the suggestion. I could make ForwardDiff work. But Flux.jacobian still does not work

The changes are:

...
function (a::PolyModel)(z::AbstractArray{T}) where T
    mon = ones(T, length(a.mexp))
...

and added
Base.convert(::Type{T}, x::Flux.Tracker.TrackedReal{T}) where T<:Real = x.data

The error now becomes:

ERROR: LoadError: MethodError: no method matching back!(::Float64; once=false)
Closest candidates are:
  back!(::Any, ::Any; once) at /Users/rs1909/.julia/packages/Flux/Dkez4/src/tracker/back.jl:63
  back!(::Flux.Tracker.TrackedReal; once) at /Users/rs1909/.julia/packages/Flux/Dkez4/src/tracker/lib/real.jl:14
  back!(::Flux.Tracker.TrackedArray) at /Users/rs1909/.julia/packages/Flux/Dkez4/src/tracker/lib/array.jl:66 got unsupported keyword argument "once"

@MikeInnes
Copy link
Member

The reason that conversion method isn't defined is that it drops gradient information (otherwise we'd just use Float64 everywhere). If you want this to work you'll have to make the ones container Array{Any}.

@dawbarton
Copy link

To construct the polynomials in such a way that Flux can deal with them, try

function PolyModel(in::Integer, out::Integer, order::Integer)
    @polyvar x[1:in]
    mx0 = monomials(x, 0:order)
    mx1 = hcat([exponents(m) for m in mx0]...)
    return PolyModel(mx1, param(randn(out, length(mx0))))
end

function (a::PolyModel)(z)
    zp = z.^a.mexp
    zpprod = zp[1, :] # doing prod(zp, dims=1) would be nicer but doesn't work with Flux
    for i = 2:size(zp, 1)
        zpprod = zpprod .* zp[i, :]
    end
    return a.W*zpprod
end

This works with both Flux and ForwardDiff.

@mcabbott
Copy link
Member

mcabbott commented Feb 6, 2019

Glad to see someone else has a use for prod(A, dims=1), you can try with #524 or equivalently this rough package for now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

6 participants