Summary

Currently, the BCL does not have support for resource pooling of any kind. A Buffer Pool type should be added to allow both the Framework and external engineers to utilize Pooling of primitive and typically-short-lived types.

Rationale and Usage

Current engineers, both internally and externally, are required to create their own Pool system for short-lived arrays or create and destroy arrays on demand, leading to lots of objects to be collected during Generation 0 Garbage Collection and therefore taking a runtime performance hit. Lots of simple pooling scenarios can be solved with a generic Rent-Return contract, allowing for a genetic Pool implementation that will prevent short-term objects or custom Pool implementations for a majority of cases.

Note

The new System.Buffers namespace is not meant to cover all Pooling cases; in some instances, the requirements are so specific to the application being written that it would not be feasible to make a pool for that case in the BCL. This addition is also not meant for object pooling; this addition is specific to buffer pooling of primitive or struct-based types. Object pooling has different requirements that are not necessarily fulfilled by buffer pooling.

Proposed API

namespace System.Buffers
{
    public abstract class ArrayPool<T>
    {
        public static ArrayPool<T> Shared { get; internal set; }

        public static ArrayPool<T> Create(int maxBufferSize = <number>, int numberOfBuffers = <number>);

        public T[] Rent(int size);

        public T[] Enlarge(T[] buffer, int newSize, bool clearBuffer = false);

        public void Return(T[] buffer, bool clearBuffer = false);
    }
}

Class Description

The ArrayPool is a new class that will manage tiered buffers for the caller. The Pool is named ArrayBufferPool due to the specific nature of this pool; the buffers from this pool are expected to be used in managed code and while they can be pinned and passed to Native code, it is not expected that they will be. There are preliminary talks to create a NativeBufferPool to help with the PInvoke scenarios, but that implementation will rely on Span, which is not ready yet. The ManagedBufferPool class will be generic in order to allow callers to Rent arrays of type T, which is expected to be primitive types such as byte or char. The Pool will be lightweight and thread-safe, allowing for fast Rent and Return calls from any thread within the process, along with minimal locking overhead, and 0 heap allocations on most Rent calls (exceptions to this will be called out below in the description of the Rent function).

To allow for resizing and to minimize fragmentation, the Pool will use Bucketing to create different buffer sizes up to the specified maximum. This allows callers to request multiple buffer sizes without needing multiple pools; also, Bucket sizes will be determined ahead of time but will not allocate ahead of time. This trade off means many different Bucket sizes can be specified without putting a strain on memory utilization unless requested.

Usage examples can be seen in the Examples section below.

Public Function Descriptions

Constructor

public ManagedBufferPool(int maxBufferSize = <number>, int numberOfBuffersPerBucket = <number>);

The ManagedBufferPool constructor takes in two arguments: the maximum buffer that is expected to be requested, and the number of buffers per BufferBucket. Both of these arguments are optional and can be used to tweak the Pool to situation-specific circumstances as well as have default values that will be tailored to most situations if they are not specified. The constructor will not allocate the buffers at this time; the pool is Lazy Loaded so that the Bucket sizes will be determined, based on the maximum size, but the memory for each Bucket will not be allocated until requested.

RentBuffer

public T[] RentBuffer(int size, bool clearBuffer = false)

The Rent(..) function is used to request a buffer of a specific size from the Pool. The caller may request that the buffer be cleared before it is Rented, but this defaults to false for performance reasons. The Pool is guaranteed to return a Buffer of at least the specified size; the actual size may be larger to due buffer availability. If the Bucket containing the specified size has not been allocated yet, it will be created at this time; any further Rent calls that hit this Bucket will not allocate any data on the Heap. This function is thread-safe.

EnlargeBuffer

public void EnlargeBuffer(ref T[] buffer, int newSize, bool clearFreeSpace = false);

The EnlargeBuffer(..) function is used to request a larger buffer than the one specified. The new buffer returned will be at least the specified size and will contain all data in the passed in buffer. The buffer musted be passed as a reference since the previous reference will no longer be valid. The caller may also request that any excess space between the end of the previous buffer to the end of the new buffer be cleared; this defaults to false for performance reasons. Like Rent, this call may allocate if the new size hits a Bucket that has not been allocated yet; if the Bucket has been allocated, this call will not allocate any data on the Heap. Only buffers that have been received via calls to RentBuffer should be passed to this function. This function is thread-safe.

ReturnBuffer

public void ReturnBuffer(ref T[] buffer, bool clearBuffer = false);

The ReturnBuffer(..) function is used to give up ownership of a buffer received from calls to RentBuffer. The call takes a reference to a buffer since the reference will no longer be valid after the call returns. The buffer can be cleared by passing true to the optional parameter, but this defaults to false for performance reasons. Only buffers that have been received by calls to RentBuffer should be passed in and a buffer can only be Returned once. This function is thread-safe.

Static Declarations

SharedBufferPool

public static ManagedBufferPool<T> SharedBufferPool {get; }
This static property allows for a Shared pool to be used in cases where multiple components in a system will require access to buffers for the duration of the process lifetime, such as a web server. This instance will be created with the default parameters and is readonly. It follows the same usage and allocation patterns described above, so if the caller does not use it, nothing will be allocated.

Future Entries into System.Buffers

Going forward, we will look into adding more types of resource pooling into the System.Buffers namespace; currently, a NativeBufferPool and an ObjectPool are in the very initial stages.

Examples

Simple Rent and Return

public class Program
{
    public static void Main(string[] args)
    {
        ManagedBufferPool<byte> pool = new ManagedBufferPool<byte>();
        for (int i = 0; i < 50; i++)
        {
            byte[] buffer = pool.RentBuffer(1024);

            // Do something to fill the buffer
            DoSomething(buffer);

            string s = System.Text.Encoding.ASCII.GetString(buffer);
            Console.WriteLine(s);
            pool.ReturnBuffer(ref buffer);
        }
    }
}

Multithreaded Example

public class Program
{
    // We know our packets will be at most 1024 bytes and we need at most 8 of them
    static ManagedBufferPool<byte> _pool = new ManagedBufferPool<byte>(1024, 8);

    public static void Main(string[] args)
    {
        for (int i = 0; i < 8; i++)
        {
            new Thread(new StartStart(DoWork)).Start()
        }

        // Pseudo-code to wait
        WaitForAllThreads();
    }

    private static void DoWork()
    {
        while (true)
        {
            byte[] buffer = _pool.RentBuffer(512);

            // Pseudo-code to get data from the network and do something with it
            GetDataFromNetwork(buffer);
            DoSomethingWithFilledBuffer(buffer);

            _pool.ReturnBuffer(ref buffer);
        }
    }
}

Updates

Update 1

After some thinking and initial feedback, removing the Shared pool due to the possibility of confusing or misuse and the possibility that components will use the Shared pool, growing the Process memory unnecessarily.

Update 2

Updated the API referenced based on the review feedback.