Description

When allocating a string/seq with newString/newSeq inside of a proc with the {.async.} pragma, the memory is not freed under orc until it reaches 4,550,824KiB (it reaches about 4.4GiB before that point, but then climbs slowly and eventually stops at the number I stated). Under refc, the memory will climb to 400MiB but never go much higher (it may still leak after that point, but very slowly if it does). This is causing loads of memory leaks in all async programs I've used, specifically in HTTP servers where strings are allocated for storing request data.

I have included a reproducer program with a few flags you can use to test different things.

• -d:useAsync - Use async (needed to reproduce the problem)
• -d:useSeq - Use newSeq instead of newString

When you aren't using async, only 100mb is ever allocated at one time, which is the expected behavior. Using a seq instead of a string has no difference in behavior but I wanted to make sure this wasn't an issue specifically with newString.

const useAsync {.booldefine.} = false
const useSeq {.booldefine.} = false

# 100MiB
const dataSize = 1024 * 1024 * 100

when useAsync:
    import std/asyncdispatch

    # Timer is used to make sure sleepAwait gets called, otherwise asyncdispatch will complain about no handlers/timers when calling waitFor
    addTimer(0, false, proc (fd: AsyncFD): bool = false)

    proc doThing() {.async.} =
        let data = when useSeq:
            newSeq[byte](dataSize)
        else:
            newString(dataSize)

        await sleepAsync(1_000)

    proc main() {.async.} =
        while true:
            await doThing()

    waitFor main()
else:
    import std/os

    proc doThing() =
        let data = when useSeq:
            newSeq[byte](dataSize)
        else:
            newString(dataSize)

        sleep(1_000)

    proc main() =
        while true:
            doThing()

    main()

Nim Version

Nim 1.6.12 (latest stable)
Nim 1.9.3 (latest devel, Git commit hash: c71192043b9febbfc058717ad3d4fde3a22c61da)

Nim 1.6.10
Nim 1.6.2
Nim 1.6.0

All tests were run under ArchLinux using Linux 6.2.9-arch1-1 x86_64

Current Output

No output, the code already compiles and runs.
---
The program will allocate an additional 100MiB every second without freeing the last string/seq until it reaches a certain large memory size.

Expected Output

The program will deallocate the string/seq it created from the last call, and the program will never have more than 100MiB allocated at any given time.
---
This is the behavior of the program without using async.

Possible Solution

Make sure memory is freed when the proc is finished

Additional Information

Running this program with -d:useAsync under refc will allocate about 4x the amount of memory that should be allocated, but it will never climb past that. I don't know why this is, but it's also very inconvenient. However, I don't use refc while working with Nim, I exclusively use orc, so this isn't as much of a concern to me.