@perolat, @bartdevylder: any ideas?

Hi @spktrm ,

Thanks for the question.

It should becomes 0 after we update the two regularisation policies. Unless we missed an edge case there should only be continuous interpolations between regularisation networks.

It should go this way:

• alpha goes from 0 to 1 linearly over half of the interval (interpolate from pi_{reg, 0} to pi_{reg, 1}),
• then alpha stays at one for the rest of the interval (here the regularisation policy is pi_{reg, 1}),
• we update the networks (to interpolate between pi_{reg, 1} and pi_{reg, 2})
• start a new interval from alpha=0 (so at the first step of this interval we start with the regularisation policy pi_{reg, 1})

Let me know if you see something that doesn't match this intended behaviour.

Julien

In the current code, on the steps the regularisation nets are updated, alpha = 0. The regularisation nets are updated after alpha is used to compute the parameter updates. As such, the current entropy schedule implementation outputs alpha = 0 when update_target_net is True.

Update: We have identified this as a bug and Eugene Tarassov has a fix. Will be fixed on next sync to github.