Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Bug] The blockchain stucks when there are some malicious nodes #10228

Open
fCorleone opened this issue Apr 5, 2022 · 0 comments
Open

[Bug] The blockchain stucks when there are some malicious nodes #10228

fCorleone opened this issue Apr 5, 2022 · 0 comments
Labels
bug Something isn't working

Comments

@fCorleone
Copy link

🐛 Bug

I start a swarm with 10 nodes and I set 3 of the nodes as malicious ones. The malicious node can modify the values of some fields of sending packets to make a chaos in the system. However, I found that all the nodes stop producing new blocks after reaching to the height of 358. And the mint command reports some errors. I think some transactions cannot be processed correctly. I attached the log files too.

To reproduce

  1. Start a new swarm with 10 nodes by using diem-swarm tool.
  2. Start to mint by
cluster-test --mint-file "/home/fuchen/Desktop/SCALE_DIEM/mint.key" --swarm --peers "localhost:44289,localhost:39251,localhost:36947,localhost:38273,localhost:41187,localhost:35033,localhost:36867,localhost:37149,localhost:33861,localhost:37105" --emit-tx --workers-per-ac 1
  1. The bug occurs

Stack trace/error message

2022-04-05T11:24:59.232522Z [tokio-runtime-worker] WARN testsuite/cluster-test/src/tx_emitter.rs:698 [Client { url: "http://localhost:38273/v1", inner: Client { accepts: Accepts, proxies: [Proxy(System({}), None)], referer: true, default_headers: {"accept": "*/*"}, timeout: 10s }, state: StateManager { last_known_state: Mutex { data: Some(State { chain_id: 4, version: 4347, timestamp_usecs: 1649157859030073 }), poisoned: false, .. } }, retry: Retry { max_retries: 20, delay: 500ms } }] Failed to submit request: Error { inner: Inner { kind: Timeout, source: Some(reqwest::Error { kind: Request, url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Domain("localhost")), port: Some(38273), path: "/v1", query: None, fragment: None }, source: TimedOut }), json_rpc_error: None } }
2022-04-05T11:24:59.405464Z [tokio-runtime-worker] WARN testsuite/cluster-test/src/tx_emitter.rs:698 [Client { url: "http://localhost:37105/v1", inner: Client { accepts: Accepts, proxies: [Proxy(System({}), None)], referer: true, default_headers: {"accept": "*/*"}, timeout: 10s }, state: StateManager { last_known_state: Mutex { data: Some(State { chain_id: 4, version: 4347, timestamp_usecs: 1649157859030073 }), poisoned: false, .. } }, retry: Retry { max_retries: 20, delay: 500ms } }] Failed to submit request: Error { inner: Inner { kind: Timeout, source: Some(reqwest::Error { kind: Request, url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Domain("localhost")), port: Some(37105), path: "/v1", query: None, fragment: None }, source: TimedOut }), json_rpc_error: None } }

And after a period, it shows:

2022-04-05T11:27:44.240016Z [tokio-runtime-worker] INFO testsuite/cluster-test/src/tx_emitter.rs:726 [Client { url: "http://localhost:44289/v1", inner: Client { accepts: Accepts, proxies: [Proxy(System({}), None)], referer: true, default_headers: {"accept": "*/*"}, timeout: 10s }, state: StateManager { last_known_state: Mutex { data: Some(State { chain_id: 4, version: 4347, timestamp_usecs: 1649157859030073 }), poisoned: false, .. } }, retry: Retry { max_retries: 20, delay: 500ms } }] Transactions were not committed before expiration: [(02C5EE6BBDD7251D19EE943911184704, 0), (6796EBA02896E4FE4801EFF096E397F1, 0), (8F1F70889CB96E6C4925C3B301046024, 0), (39423FEA1B5F0A43A1BC2E5EF1F75A38, 0), (BC400A390E191174DB7B8E8CF6B4477F, 0), (A64D6206D2DE57BCE17FFA178423BD7C, 0), (05263070B5B3B4F895099F4168BCD595, 0), (2E57B69C5BC8F991E017DA147CAA650D, 0), (8925B6B3D1E4B610355EA28D5281D98F, 0), (C5026F0E19BE7858792223625A345496, 0), (CCFDEFABF423302F9A0033915F034385, 0), (86B661ED2D00DB36CE66A58FFD3057B0, 0), (A6247026D5390833846A0D4C365733C3, 0), (D777B02EF55645CA86EAECA7839A7D39, 0), (49EB8802D9025C9672B8947CCC408521, 0)]
2022-04-05T11:27:44.715434Z [tokio-runtime-worker] WARN testsuite/cluster-test/src/tx_emitter.rs:811 Wait deadline exceeded for account 38F7C8431B6AB4F8EEFA8D2C1ED97544, expected sequence 1, got from server: 0
2022-04-05T11:27:44.715489Z [tokio-runtime-worker] WARN testsuite/cluster-test/src/tx_emitter.rs:811 Wait deadline exceeded for account 8EEA6083920093C891F7FD36D5208251, expected sequence 1, got from server: 0
2022-04-05T11:27:44.715499Z [tokio-runtime-worker] WARN testsuite/cluster-test/src/tx_emitter.rs:811 Wait deadline exceeded for account AD2DCAEA497103AC58C81097687969E9, expected sequence 1, got from server: 0
2022-04-05T11:27:44.715506Z [tokio-runtime-worker] WARN testsuite/cluster-test/src/tx_emitter.rs:811 Wait deadline exceeded for account 897EF36A23B68C3C0F7303FB6BDAAB0D, expected sequence 1, got from server: 0
2022-04-05T11:27:44.715514Z [tokio-runtime-worker] WARN testsuite/cluster-test/src/tx_emitter.rs:811 Wait deadline exceeded for account B662EBF7E65A294649AB5AC0FBDE7D77, expected sequence 1, got from server: 0
2022-04-05T11:27:44.715522Z [tokio-runtime-worker] WARN testsuite/cluster-test/src/tx_emitter.rs:811 Wait deadline exceeded for account 4AC9AEDFE69BEB4A57662A794D2D1EFA, expected sequence 1, got from server: 0
2022-04-05T11:27:44.715530Z [tokio-runtime-worker] WARN testsuite/cluster-test/src/tx_emitter.rs:811 Wait deadline exceeded for account 7AFAB348D11720F5C15E24C534A747E0, expected sequence 1, got from server: 0
2022-04-05T11:27:44.715538Z [tokio-runtime-worker] WARN testsuite/cluster-test/src/tx_emitter.rs:811 Wait deadline exceeded for account C064853F2CB399080D04C4462DD4BF3D, expected sequence 1, got from server: 0
2022-04-05T11:27:44.715546Z [tokio-runtime-worker] WARN testsuite/cluster-test/src/tx_emitter.rs:811 Wait deadline exceeded for account 15C1515E0C6EBECCBA689963DBD34261, expected sequence 1, got from server: 0
2022-04-05T11:27:44.715554Z [tokio-runtime-worker] WARN testsuite/cluster-test/src/tx_emitter.rs:811 Wait deadline exceeded for account C05C1D5068E7D673A4B8AB5D1ABE3F54, expected sequence 1, got from server: 0
2022-04-05T11:27:44.715562Z [tokio-runtime-worker] WARN testsuite/cluster-test/src/tx_emitter.rs:811 Wait deadline exceeded for account 4792927364A42E385AC0E71AFE88959F, expected sequence 1, got from server: 0
2022-04-05T11:27:44.715569Z [tokio-runtime-worker] WARN testsuite/cluster-test/src/tx_emitter.rs:811 Wait deadline exceeded for account 0F3484843DB7455CCEA1354FBF617CD4, expected sequence 1, got from server: 0
2022-04-05T11:27:44.715577Z [tokio-runtime-worker] WARN testsuite/cluster-test/src/tx_emitter.rs:811 Wait deadline exceeded for account 2A34117AA9D1D6203FF14A65E673BD33, expected sequence 1, got from server: 0
2022-04-05T11:27:44.715584Z [tokio-runtime-worker] WARN testsuite/cluster-test/src/tx_emitter.rs:811 Wait deadline exceeded for account A7A4F4AD3E4AF72E7EC84AE5D30B375B, expected sequence 1, got from server: 0
2022-04-05T11:27:44.715595Z [tokio-runtime-worker] WARN testsuite/cluster-test/src/tx_emitter.rs:811 Wait deadline exceeded for account 53D89E01F4AC22AA5BA58583916C0634, expected sequence 1, got from server: 0

Expected Behavior

I think all the transactions should be processed correctly under this situation.

System information

Please complete the following information:

  • Diem: branch testnet
  • Rust version: cargo 1.61.0-nightly
  • Ubuntu 20.04

Additional context

Screenshot from 2022-04-05 19-36-16
Screenshot from 2022-04-05 19-34-21

Log Files

0.zip
1.zip
2.zip
3.zip
4.zip
5.zip
6.zip
7.zip
8.zip
9.zip
@fCorleone fCorleone added the bug Something isn't working label Apr 5, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@fCorleone