Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Network reconstruction #3900

Merged
merged 30 commits into from Jun 26, 2023
Merged

Network reconstruction #3900

merged 30 commits into from Jun 26, 2023

Conversation

jackzhhuang
Copy link
Collaborator

@jackzhhuang jackzhhuang commented May 24, 2023

Pull request type

Please check the type of change your PR introduces:

  • Bugfix
  • Feature
  • Code style update (formatting, renaming)
  • Refactoring (no functional changes, no api changes)
  • Build related changes
  • Documentation content changes
  • Other (please describe):

What is the current behavior?

Issue Number: N/A

What is the new behavior?

  • Put the handshaking logic into the network crate from the network-p2p crate.
  • Put the drive and core crates in the network-rpc crate into the network-p2p crate for reusability.
  • Update the version of libp2p from 0.50 to 0.51.

Other information

@jackzhhuang jackzhhuang linked an issue May 24, 2023 that may be closed by this pull request
@jackzhhuang jackzhhuang requested review from welbon and nkysg May 24, 2023 16:45
@jackzhhuang jackzhhuang changed the title add network reconstruction for handshaking Network reconstruction May 24, 2023

impl BusinessLayerHandle for PeerInfo {
fn handshake(&self, peer_info: &[u8]) -> Result<(), (&'static str, String)> {
let other_chain_info = ChainInfo::decode(peer_info).unwrap();
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这些地方unwrap()失败是不是直接导致节点崩溃了?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这个我后面改一改

@nkysg nkysg requested a review from simonjiao May 25, 2023 00:59
@github-actions
Copy link

Benchmark for 92afc23

Click to view benchmark
Test Base PR %
accumulator_append 615.1±26.53µs 610.9±30.72µs -0.68%
block_apply/block_apply_10 355.4±0.90ms 355.5±0.94ms +0.03%
block_apply/block_apply_1000 36.5±0.03s 36.9±0.22s +1.10%
get_with_proof/db_store 37.0±0.27µs 37.0±0.29µs 0.00%
get_with_proof/mem_store 31.6±0.14µs 31.8±0.14µs +0.63%
put_and_commit/db_store/1 95.9±4.88µs 96.7±4.88µs +0.83%
put_and_commit/db_store/10 864.1±39.18µs 867.2±57.18µs +0.36%
put_and_commit/db_store/100 7.4±0.28ms 7.4±0.27ms 0.00%
put_and_commit/db_store/5 442.0±22.20µs 441.1±22.01µs -0.20%
put_and_commit/db_store/50 3.8±0.16ms 3.9±0.17ms +2.63%
put_and_commit/mem_store/1 62.2±5.67µs 62.6±5.80µs +0.64%
put_and_commit/mem_store/10 586.5±49.39µs 586.3±47.12µs -0.03%
put_and_commit/mem_store/100 5.7±0.84ms 5.8±0.83ms +1.75%
put_and_commit/mem_store/5 293.7±24.63µs 296.3±24.97µs +0.89%
put_and_commit/mem_store/50 2.9±0.19ms 2.9±0.19ms 0.00%
query_block/query_block_in(10)_times(100) 5.0±0.11ms 5.2±0.24ms +4.00%
query_block/query_block_in(10)_times(1000) 50.8±2.13ms 50.6±2.18ms -0.39%
query_block/query_block_in(10)_times(10000) 512.4±17.26ms 511.8±15.72ms -0.12%
query_block/query_block_in(1000)_times(100) 1038.7±11.80µs 1005.8±6.88µs -3.17%
query_block/query_block_in(1000)_times(1000) 10.3±0.10ms 10.1±0.08ms -1.94%
query_block/query_block_in(1000)_times(10000) 103.5±0.74ms 100.1±0.77ms -3.29%
storage_transaction 1141.9±397.72µs 1027.7±416.82µs -10.00%
vm/transaction_execution/1 404.6±0.97ms 408.8±4.99ms +1.04%
vm/transaction_execution/10 123.7±0.84ms 123.8±0.21ms +0.08%
vm/transaction_execution/20 114.2±3.06ms 113.9±0.89ms -0.26%
vm/transaction_execution/5 152.7±1.11ms 153.1±1.32ms +0.26%
vm/transaction_execution/50 128.7±0.33ms 128.7±0.55ms 0.00%

@github-actions
Copy link

Benchmark for b88260e

Click to view benchmark
Test Base PR %
accumulator_append 611.3±26.75µs 612.3±28.36µs +0.16%
block_apply/block_apply_10 356.0±0.54ms 358.5±0.80ms +0.70%
block_apply/block_apply_1000 36.7±0.02s 37.0±0.05s +0.82%
get_with_proof/db_store 36.9±0.45µs 37.3±0.29µs +1.08%
get_with_proof/mem_store 31.9±0.14µs 31.7±0.24µs -0.63%
put_and_commit/db_store/1 97.2±4.79µs 97.8±5.21µs +0.62%
put_and_commit/db_store/10 869.6±39.64µs 867.2±40.25µs -0.28%
put_and_commit/db_store/100 7.4±0.30ms 7.5±0.29ms +1.35%
put_and_commit/db_store/5 450.5±19.99µs 422.6±21.33µs -6.19%
put_and_commit/db_store/50 3.8±0.16ms 3.9±0.16ms +2.63%
put_and_commit/mem_store/1 62.8±5.85µs 62.6±5.70µs -0.32%
put_and_commit/mem_store/10 588.4±49.74µs 587.9±49.67µs -0.08%
put_and_commit/mem_store/100 5.7±0.83ms 5.8±0.83ms +1.75%
put_and_commit/mem_store/5 297.7±26.03µs 296.8±25.66µs -0.30%
put_and_commit/mem_store/50 2.9±0.19ms 2.9±0.19ms 0.00%
query_block/query_block_in(10)_times(100) 5.1±0.15ms 5.2±0.19ms +1.96%
query_block/query_block_in(10)_times(1000) 51.3±1.99ms 50.5±1.11ms -1.56%
query_block/query_block_in(10)_times(10000) 512.9±12.88ms 513.5±13.60ms +0.12%
query_block/query_block_in(1000)_times(100) 1024.5±9.36µs 997.9±10.63µs -2.60%
query_block/query_block_in(1000)_times(1000) 10.3±0.08ms 10.1±0.17ms -1.94%
query_block/query_block_in(1000)_times(10000) 102.2±0.56ms 99.8±0.91ms -2.35%
storage_transaction 998.6±280.70µs 1169.2±428.90µs +17.08%
vm/transaction_execution/1 404.9±0.86ms 406.2±1.31ms +0.32%
vm/transaction_execution/10 124.3±1.57ms 123.9±0.44ms -0.32%
vm/transaction_execution/20 113.6±0.44ms 113.9±0.48ms +0.26%
vm/transaction_execution/5 153.1±1.23ms 152.9±0.24ms -0.13%
vm/transaction_execution/50 129.7±2.65ms 128.9±0.50ms -0.62%

@codecov
Copy link

codecov bot commented May 26, 2023

Codecov Report

Merging #3900 (a587398) into master (62f2b39) will increase coverage by 0.16%.
The diff coverage is 68.87%.

❗ Current head a587398 differs from pull request most recent head d352437. Consider uploading reports for the commit d352437 to get more accurate results

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #3900      +/-   ##
==========================================
+ Coverage   53.73%   53.89%   +0.16%     
==========================================
  Files         618      619       +1     
  Lines       68014    68233     +219     
==========================================
+ Hits        36539    36764     +225     
+ Misses      31475    31469       -6     
Flag Coverage Δ
unittests 53.89% <68.87%> (+0.16%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
cmd/peer-watcher/src/lib.rs 6.25% <0.00%> (ø)
network-p2p/core/src/delegates.rs 100.00% <ø> (ø)
network-p2p/core/src/lib.rs 36.07% <ø> (ø)
network-p2p/core/src/server.rs 93.75% <ø> (ø)
network-p2p/derive/src/helper.rs 76.32% <ø> (ø)
network-p2p/derive/src/lib.rs 100.00% <ø> (ø)
network-p2p/derive/src/options.rs 63.89% <ø> (ø)
network-p2p/derive/src/rpc_trait.rs 66.67% <ø> (ø)
network-p2p/src/lib.rs 100.00% <ø> (ø)
network-p2p/src/protocol/event.rs 16.67% <ø> (ø)
... and 20 more

... and 23 files with indirect coverage changes


Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 62f2b39...d352437. Read the comment docs.

@github-actions
Copy link

Benchmark for 0c8613f

Click to view benchmark
Test Base PR %
accumulator_append 612.2±29.56µs 612.3±30.80µs +0.02%
block_apply/block_apply_10 356.4±0.49ms 360.1±3.84ms +1.04%
block_apply/block_apply_1000 36.9±0.04s 36.8±0.04s -0.27%
get_with_proof/db_store 37.8±0.23µs 37.6±0.25µs -0.53%
get_with_proof/mem_store 32.2±0.32µs 32.1±0.17µs -0.31%
put_and_commit/db_store/1 93.3±3.23µs 92.6±3.30µs -0.75%
put_and_commit/db_store/10 830.7±25.63µs 832.8±25.35µs +0.25%
put_and_commit/db_store/100 7.2±0.21ms 7.2±0.20ms 0.00%
put_and_commit/db_store/5 425.5±13.83µs 407.7±13.36µs -4.18%
put_and_commit/db_store/50 3.7±0.11ms 3.7±0.11ms 0.00%
put_and_commit/mem_store/1 62.8±5.64µs 63.1±5.66µs +0.48%
put_and_commit/mem_store/10 589.0±47.78µs 586.8±46.77µs -0.37%
put_and_commit/mem_store/100 5.8±0.91ms 5.7±0.34ms -1.72%
put_and_commit/mem_store/5 296.9±26.53µs 297.5±24.55µs +0.20%
put_and_commit/mem_store/50 2.9±0.19ms 2.9±0.18ms 0.00%
query_block/query_block_in(10)_times(100) 5.1±0.10ms 5.1±0.16ms 0.00%
query_block/query_block_in(10)_times(1000) 51.1±1.45ms 50.2±2.51ms -1.76%
query_block/query_block_in(10)_times(10000) 506.2±11.06ms 507.0±6.06ms +0.16%
query_block/query_block_in(1000)_times(100) 1007.8±10.87µs 1027.8±9.40µs +1.98%
query_block/query_block_in(1000)_times(1000) 10.1±0.10ms 10.2±0.08ms +0.99%
query_block/query_block_in(1000)_times(10000) 100.6±0.67ms 102.1±1.25ms +1.49%
storage_transaction 920.7±239.00µs 932.7±273.87µs +1.30%
vm/transaction_execution/1 406.6±0.93ms 406.5±1.26ms -0.02%
vm/transaction_execution/10 124.2±0.65ms 123.7±0.52ms -0.40%
vm/transaction_execution/20 113.9±0.61ms 113.5±0.21ms -0.35%
vm/transaction_execution/5 152.9±0.42ms 152.9±0.47ms 0.00%
vm/transaction_execution/50 129.1±1.18ms 128.9±1.13ms -0.15%

self.network_service.update_business_status(
ChainStatus::new(msg.compact_block.header.clone(), msg.block_info.clone())
.encode()
.unwrap(),
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这里ChainStatus::new变量是不是encode一定会成功?是的话这个unwrap()改成expect

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

好的,我改一下

}

impl BusinessLayerHandle for Networkp2pHandle {
fn handshake(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

建议此处只对 status 进行处理,根据其返回状态来处理 notifications_sink和 CustomMessageOutcome,这两个在network这层不关心,放到 network-p2p里面比较合理。

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

已修改,我重新测试一下,跑main和Barnard 1-2天的时间

1, add expect in upadte business status
2, handshake return the result struct hence decouple the relation of network-p2p and network modules
@github-actions
Copy link

Benchmark for bf074b5

Click to view benchmark
Test Base PR %
accumulator_append 616.7±28.68µs 607.8±8.37µs -1.44%
block_apply/block_apply_10 358.0±0.43ms 357.1±0.25ms -0.25%
block_apply/block_apply_1000 36.8±0.03s 36.9±0.27s +0.27%
get_with_proof/db_store 37.1±0.25µs 37.1±0.16µs 0.00%
get_with_proof/mem_store 31.8±0.15µs 31.7±0.14µs -0.31%
put_and_commit/db_store/1 93.0±3.54µs 92.4±3.14µs -0.65%
put_and_commit/db_store/10 824.5±26.80µs 822.3±25.34µs -0.27%
put_and_commit/db_store/100 7.2±0.18ms 7.2±0.20ms 0.00%
put_and_commit/db_store/5 422.1±13.39µs 421.7±14.01µs -0.09%
put_and_commit/db_store/50 3.7±0.11ms 3.8±0.11ms +2.70%
put_and_commit/mem_store/1 62.6±5.65µs 62.5±5.68µs -0.16%
put_and_commit/mem_store/10 583.7±47.03µs 583.6±46.65µs -0.02%
put_and_commit/mem_store/100 5.8±0.83ms 5.7±0.34ms -1.72%
put_and_commit/mem_store/5 296.7±25.68µs 294.4±24.65µs -0.78%
put_and_commit/mem_store/50 2.9±0.18ms 2.9±0.18ms 0.00%
query_block/query_block_in(10)_times(100) 5.1±0.18ms 5.0±0.15ms -1.96%
query_block/query_block_in(10)_times(1000) 49.8±1.81ms 50.7±1.21ms +1.81%
query_block/query_block_in(10)_times(10000) 511.1±15.84ms 507.2±9.06ms -0.76%
query_block/query_block_in(1000)_times(100) 1025.7±9.88µs 1060.6±9.39µs +3.40%
query_block/query_block_in(1000)_times(1000) 10.2±0.10ms 10.6±0.08ms +3.92%
query_block/query_block_in(1000)_times(10000) 103.1±1.14ms 105.3±0.80ms +2.13%
storage_transaction 911.0±223.46µs 1023.5±337.69µs +12.35%
vm/transaction_execution/1 408.6±3.81ms 407.0±1.20ms -0.39%
vm/transaction_execution/10 123.9±0.24ms 123.9±0.47ms 0.00%
vm/transaction_execution/20 113.8±0.19ms 114.3±1.72ms +0.44%
vm/transaction_execution/5 153.8±1.89ms 153.5±1.74ms -0.20%
vm/transaction_execution/50 129.1±0.70ms 129.5±1.81ms +0.31%

@jackzhhuang jackzhhuang linked an issue Jun 22, 2023 that may be closed by this pull request
@jackzhhuang jackzhhuang merged commit 5772b32 into master Jun 26, 2023
3 of 5 checks passed
@jackzhhuang jackzhhuang deleted the network-reconstruction branch June 26, 2023 03:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
3 participants