.NET 8.0.10 vs 9.0.0 RC2 GC Server Performance Regression in Sep (CSV Parser) Benchmark (due to DATAS default) #109047
Description
In https://github.com/nietras/Sep (a fast highly optimized CSV parser) I have been comparing performance comparison-bench.ps1 between .NET 8 and .NET 9 RC2 and have observed what appears to be consistent and significant performance regression when using ServerGarbageCollection (true). The benchmark in question is also discussed in https://www.joelverhagen.com/blog/2020/12/fastest-net-csv-parsers
Benchmarks can be run by cloning the Sep repo, checking out branch net9.0 and running the command in the comparison-bench.ps1 perhaps adding --filter *GcServer*Sep* or similar. Details for benchmark, machine are given below via BenchmarkDotNet.
As can be seen this shows regression in a scenario of many medium size object allocations ranging from 500ms/429ms = 1.17x (single thread) to 174ms/102ms = 1.69x (multi-threaded) regression.
I know there have been changes to the GC my question is whether this regression is expected? And just wanted to flag it if it has any interest.
BenchmarkDotNet v0.14.0, Windows 10 (10.0.19044.3086/21H2/November2021Update)
AMD Ryzen 9 5950X, 1 CPU, 32 logical and 16 physical cores
.NET SDK 9.0.100-rc.2.24474.11
[Host] : .NET 8.0.10 (8.0.1024.46610), X64 RyuJIT AVX2
Job-YVJTZC : .NET 8.0.10 (8.0.1024.46610), X64 RyuJIT AVX2
Job-ZDJCYM : .NET 9.0.0 (9.0.24.47305), X64 RyuJIT AVX2
Server=True InvocationCount=Default IterationTime=350ms
MaxIterationCount=15 MinIterationCount=5 WarmupCount=6
Quotes=False Reader=String
| Method | Runtime | Scope | Rows | Mean | Ratio | MB | MB/s | ns/row | Allocated | Alloc Ratio |
|---|---|---|---|---|---|---|---|---|---|---|
| Sep______ | .NET 8.0 | Asset | 50000 | 21.402 ms | 1.00 | 29 | 1363.5 | 428.0 | 14133102 B | 1.00 |
| Sep_MT___ | .NET 8.0 | Asset | 50000 | 5.576 ms | 0.26 | 29 | 5233.7 | 111.5 | 14308501 B | 1.01 |
| Sep______ | .NET 9.0 | Asset | 50000 | 24.444 ms | 1.14 | 29 | 1193.8 | 488.9 | 14133077 B | 1.00 |
| Sep_MT___ | .NET 9.0 | Asset | 50000 | 8.965 ms | 0.42 | 29 | 3255.0 | 179.3 | 14310332 B | 1.01 |
| Sep______ | .NET 8.0 | Asset | 1000000 | 429.654 ms | 1.00 | 583 | 1358.7 | 429.7 | 273063216 B | 1.00 |
| Sep_MT___ | .NET 8.0 | Asset | 1000000 | 102.979 ms | 0.24 | 583 | 5668.9 | 103.0 | 274049328 B | 1.00 |
| Sep______ | .NET 9.0 | Asset | 1000000 | 500.250 ms | 1.16 | 583 | 1167.0 | 500.3 | 273062592 B | 1.00 |
| Sep_MT___ | .NET 9.0 | Asset | 1000000 | 174.802 ms | 0.41 | 583 | 3339.7 | 174.8 | 273973628 B | 1.00 |