This is a proposal for adding first class support in Pkg and the code loading system for conditional dependencies.

What is a conditional dependency

Desribing a conditional dependency is easiest with an example. A typical concrete example is for a plotting package to add support for plotting e.g. DataFrames (by adding some method plot(::DataFrame)) but not require a user to install DataFrames to use the plotting package. The plotting package wants to run a bit of extra code (the part that defines the method) when the conditional dependency DataFrames are somehow "available" to the user. The extra code that the package executes when the conditional dependency is active is called "glue code".

Current way of doing conditional dependencies

The way people implement conditional dependencies right now is by using Requires.jl. It works by registering a callback that evaluates some code with the package loading code in Base. The callback gets executed when the conditional dependency is loaded (by e.g. comparing UUID), the code from the callback is evaluated into the module and the functionality for the conditional dependency is provided.

As an example usage:

using Requries
function __init__()
    @require DataFrames="c91e804a-d5a3-530f-b6f0-dfbca275c004" plot(df::DataFrame) = ---
end

What is the problem with Requires.jl

There are a few reasons why the current strategy using Requires.jl to deal with this is unsatisfactory.

1. It doesn't work well with precompilation. The way people tend to use Requires is by include-ing some file when the conditional dependnecy is available. Requires.jl runs inside __init__ which means the code evaluated by the include command does not end up in the precompile file.
2. It is "implicit" in the sense that the conditional dependency is only defined in the Julia code. We typically want to put all dependency information inside the Project file.
3. There is currently no good way to set compat bounds on the conditional dependency.
4. It has performance problems (Improving @require performance JuliaPackaging/Requires.jl#39)
5. Basing the activation criteria of the conditional dependency on it simply being loaded might means that packages loaded from other places than the current project will affect whether the glue code is run or not. It would be better to base it only on the current active project.

The current proposal

How does declare a conditional dependency

One declares a conditional dependency by adding an entry to the Project.toml file as:

[conditional-deps]
DataFrames = "$UUID_DATAFRAMES"

[compat]
DataFrames

An alternative possibility is to just put DataFrames inside [deps] and then have a list of names that are conditional.

[deps]
SomeOtherDep = "..."
DataFrames = "$UUID_DATAFRAMES"

[conditional-deps] = ["DataFrames", ]

Where should the glue code be stored?

Precompilation works on a module granularity so we want a module containing the glue code for each conditional dependency. The gluecode would be stored (based on a documented convention) in a file inside the package, eg src/DataFramesGlue.jl inside Plots where the exact name of the file is yet to be decieded.

An example of a glue file for Plots conditionally depending on DataFrames is:

module DataFramesGlue

using Plots, DataFrames
Plots.plot(df::DataFrame) = ...

end

How is the glue code loaded?

When DataFrames gets loaded, we check all packages that declares a conditional dependency with it. If the version of DataFrames loaded is compatible with the compat entry for a package with DataFrames as a conditional dependency, we load the glue code which will act like a normal package and precompile. We need to teach code loading some stuff about glue packages so it knows how to map the names inside the glue module to the UUIDs in the "main package".

The fact that we are not trying to resolve a set of versions compatible with the conditional dependency avoids cases where we in general need to resolve in arbitrarily many times with potential of cycles.