Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dispatchers.IO not available on Kotlin/Native #3205

Closed
charleskorn opened this issue Feb 25, 2022 · 11 comments
Closed

Dispatchers.IO not available on Kotlin/Native #3205

charleskorn opened this issue Feb 25, 2022 · 11 comments

Comments

@charleskorn
Copy link

Dispatchers.IO is only available for Kotlin/JVM according to the docs.

It would be great to have this available on Kotlin/Native, as this would make it easier to share code between JVM and native environments.

(from https://youtrack.jetbrains.com/issue/KT-51449)

@jakobkmar
Copy link

What is the best dispatcher to use for blocking IO operations on Kotlin/Native until this issue is resolved?

@qwwdfsad
Copy link
Contributor

qwwdfsad commented Apr 22, 2022

@jakobkmar newFixedThreadPoolContext with high enough number of threads (50-200) should do the trick

@Thomas-Vos
Copy link
Contributor

Is there a way to create some sort of IO dispatcher with no thread switching from Dispatchers.Default? An IO dispatcher would be very helpful to me.

@saket
Copy link

saket commented Oct 31, 2022

@qwwdfsad wouldn't Dispatchers.Default be a better candidate because it uses dispatch_get_global_queue on native, whose thread count is auto-controlled by the system?

@dkhalanskyjb
Copy link
Collaborator

@saket we don't want Dispatchers.IO to share the behavior of Dispatchers.Default though, as the number of threads dedicated to I/O work (Dispatchers.IO) is typically an order of magnitude higher than the number of threads dedicated to doing work on the CPU (Dispatchers.Default). The reason is, many of those I/O threads most of the time will be doing nothing but waiting for some I/O operation to complete, so having many such threads will not overwhelm the system, whereas having many computation-heavy threads is undesirable: the computer can't physically perform more computations in parallel than it has CPU cores, so having much more threads dedicated to that can lead to a slowdown due to needless context-switching between threads.

@saket
Copy link

saket commented Nov 25, 2022

@dkhalanskyjb is your concern valid for both native and JVM platforms? I was only suggesting using Dispatchers.Default on native where it isn't limited to one thread per core.

@dkhalanskyjb
Copy link
Collaborator

is your concern valid for both native and JVM platforms?

Sure, I don't see any difference in this regard.

@vicpinm
Copy link

vicpinm commented Dec 10, 2022

It's been almost a year since this issue was opened. Is there any update on this?

qwwdfsad added a commit that referenced this issue Jan 4, 2023
* Emulate expect declaration refinement via extension property as the only way to do it in a backwards-compatible manner: in the current model it is impossible to have
  common 'expect' Dispatchers declaration, then refined 'concurrent' Dispatchers declaration with 'expect val IO' and then JVM declaration with JVM-specific members. Current solutions seems to be the less intrusive one
* Elasticity is not supported as K/N Workers API lacks capability to gracefully shutdown workers, meaning that for any unlimited underlying pool, memory consumption is only going to grow in an unbound manner

Fixes #3205
qwwdfsad added a commit that referenced this issue Jan 4, 2023
* Emulate expect declaration refinement via extension property as the only way to do it in a backwards-compatible manner: in the current model it is impossible to have
  common 'expect' Dispatchers declaration, then refined 'concurrent' Dispatchers declaration with 'expect val IO' and then JVM declaration with JVM-specific members. Current solutions seems to be the less intrusive one
* Elasticity is not supported as K/N Workers API lacks capability to gracefully shutdown workers, meaning that for any unlimited underlying pool, memory consumption is only going to grow in an unbound manner

Fixes #3205
@jflavio11
Copy link

Hi Hi 👋 Flavio here

I've created an expect for this

expect class DispatcherProvider {
    ...
    fun io(): CoroutineDispatcher
    ...
}

And the implementation for the iOS module is:

actual class DispatcherProvider {
    actual fun io(): CoroutineDispatcher {
        return newFixedThreadPoolContext(nThreads = 200, name = "IO")
    }
}

However, I'm not pretty sure that the nThreads parameter should be passed as an arbitrary value. The Android IO Dispatcher implementation is based on the number of available processors of the device:

// Dispatcher.kt →  DefaultIoScheduler
systemProp(
            IO_PARALLELISM_PROPERTY_NAME,
            64.coerceAtLeast(AVAILABLE_PROCESSORS)
)

Shouldn't we have something similar for iOS? Limiting the nThreads by a number supported by the device's processor. Or maybe is something that I'm missing?

qwwdfsad added a commit that referenced this issue Feb 15, 2023
* Emulate expect declaration refinement via extension property as the only way to do it in a backwards-compatible manner: in the current model it is impossible to have
  common 'expect' Dispatchers declaration, then refined 'concurrent' Dispatchers declaration with 'expect val IO' and then JVM declaration with JVM-specific members. Current solutions seems to be the less intrusive one
* Elasticity is not supported as K/N Workers API lacks capability to gracefully shutdown workers, meaning that for any unlimited underlying pool, memory consumption is only going to grow in an unbound manner

Fixes #3205
@arkivanov
Copy link

Wondering, is it now available? The doc looks outdated? https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-dispatchers/-i-o.html

@jeffdgr8
Copy link

jeffdgr8 commented Nov 1, 2023

@arkivanov yeah, it's an extension function that shadows the JVM field, documented here: https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-i-o.html

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

10 participants