Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[memory] Override default block cache config settings #129

Open
patrick-ogrady opened this issue Apr 3, 2023 · 1 comment
Open

[memory] Override default block cache config settings #129

patrick-ogrady opened this issue Apr 3, 2023 · 1 comment
Labels
lifecycle/stale

Comments

@patrick-ogrady
Copy link
Contributor

patrick-ogrady commented Apr 3, 2023 •
edited
Loading

During load testing, I found that avalanchego uses a ton of memory storing parsed blocks (basically 2048 * 2 * <expected block size>):

Showing nodes accounting for 1.94GB, 97.98% of 1.98GB total
Dropped 288 nodes (cum <= 0.01GB)
Showing top 10 nodes out of 65
      flat  flat%   sum%        cum   cum%
    0.95GB 47.83% 47.83%     0.95GB 47.83%  google.golang.org/protobuf/internal/impl.consumeBytesNoZero
    0.82GB 41.52% 89.35%     0.82GB 41.55%  github.com/ava-labs/avalanchego/vms/components/chain.(*State).ParseBlock
    0.16GB  8.07% 97.42%     0.16GB  8.07%  github.com/ava-labs/avalanchego/utils/wrappers.(*Packer).expand (inline)
    0.01GB  0.56% 97.98%     0.01GB  0.56%  github.com/syndtr/goleveldb/leveldb/util.(*BufferPool).Get
         0     0% 97.98%     0.16GB  8.07%  github.com/ava-labs/avalanchego/codec.(*manager).Marshal
         0     0% 97.98%     0.16GB  8.07%  github.com/ava-labs/avalanchego/codec/reflectcodec.(*genericCodec).MarshalInto
         0     0% 97.98%     0.16GB  8.07%  github.com/ava-labs/avalanchego/codec/reflectcodec.(*genericCodec).marshal
         0     0% 97.98%     0.79GB 40.04%  github.com/ava-labs/avalanchego/message.(*inMsgBuilder).Parse
         0     0% 97.98%     0.79GB 40.04%  github.com/ava-labs/avalanchego/message.(*msgBuilder).parseInbound
         0     0% 97.98%     0.79GB 40.04%  github.com/ava-labs/avalanchego/message.(*msgBuilder).unmarshal

This is very likely related to this: https://github.com/ava-labs/avalanchego/blob/7d73b59cb4838d304387ea680b9cc4053b72620c/vms/rpcchainvm/vm_client.go#L65-L70

const (
	decidedCacheSize    = 2048
	missingCacheSize    = 2048
	unverifiedCacheSize = 2048
	bytesToIDCacheSize  = 2048
)
@github-actions
Copy link

This issue has become stale because it has been open 60 days with no activity. Adding the lifecycle/frozen label will exempt this issue from future lifecycle events.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
lifecycle/stale
Projects
Platform Engineering Group
Archived in project
Milestone
No milestone
Development

No branches or pull requests
1 participant
@patrick-ogrady