Fix limitedParallelism doing dispatches when it has no tasks
#3672
Conversation
To support fairness w.r.t. the tasks that want to run on the wrapped dispatcher outside of its view, `LimitedDispatcher` redispatches the work loop periodically. This can happen even when there are no more tasks in `LimitedDispatcher`'s queue. Typically, this behavior is completely benign: after being redistpatched, the new runner just checks the queue, sees that there is nothing there, and leaves. However, with closable dispatchers, this affects correctness, as this redispatch may happen even after all the dispatched tasks were run and the dispatcher was closed.
dkhalanskyjb
force-pushed
the
fix-limitedDispatcher-extra-dispatches
branch
from
d28ccef to
2ac1644
Compare
|
Testing this is difficult. The problematic behavior only happens when there are no more tasks to run when the fairness dispatch wants to proceed, but before that, the worker had a steady supply of tasks and did not get fired. With a hardcoded |
kotlinx-coroutines-core/common/src/internal/LimitedDispatcher.kt
Outdated
Show resolved
Hide resolved
| handleCoroutineException(EmptyCoroutineContext, e) | ||
| } | ||
| // 16 is our out-of-thin-air constant to emulate fairness. Used in JS dispatchers as well | ||
| if (++fairnessCounter >= 16 && dispatcher.isDispatchNeeded(this)) { |
There was a problem hiding this comment.
Is it correct that this PR addresses the following issue in this snippet:
runexecutes task (:54)- Underlying dispatcher is closed
runfinishes, unluckily enough overflowsfairnessCounter, goes to redispatch and crashes?
There was a problem hiding this comment.
That's one aspect of it, yes. Consider also the case where an external dispatch or dispatchYield happened (for example, from some callback), scheduled a task, waited for a millennia, during which this task and every other one got executed and the dispatcher got closed, and then proceeded to allocate a worker and run it.
Co-authored-by: Vsevolod Tolstopyatov <qwwdfsad@gmail.com>
To support fairness w.r.t. the tasks that want to run on the wrapped dispatcher outside of its view,
LimitedDispatcherredispatches the work loop periodically. This can happen even when there are no more tasks inLimitedDispatcher's queue. Typically, this behavior is completely benign: after being redistpatched, the new runner just checks the queue, sees that there is nothing there, and leaves. However, with closable dispatchers, this affects correctness, as this redispatch may happen even after all the dispatched tasks were run and the dispatcher was closed.