Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(code): Actor-based node implementation #167

Merged
merged 89 commits into from
Apr 3, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
89 commits
Select commit Hold shift + click to select a range
2ffb3fe
Implement TCP-based broadcast network
romac Feb 6, 2024
b1287cc
Identify which peer sent a message
romac Feb 19, 2024
8d05ecb
Fix warning
romac Feb 19, 2024
abb62cc
Implement `Network` for `broadcast::Handle`
romac Feb 20, 2024
45e2f17
Remove manual impls of common trait and use `derive-where` instead
romac Feb 20, 2024
31c0ca9
First draft of Protobuf definitions for Malachite types
romac Feb 20, 2024
198709d
Cleanup
romac Feb 20, 2024
86d9390
Make network messages agnostic of `Context` by using proto definition…
romac Feb 20, 2024
d6c85ae
Refactor `Protobuf` trait
romac Feb 20, 2024
67babff
Add `Protobuf` impl for `SignedVote`
romac Feb 20, 2024
ac5f202
Cleanup
romac Feb 20, 2024
967587b
Remove Protobuf related code from `malachite-common` and into `malach…
romac Feb 20, 2024
5f06576
Refactor `Protobuf` trait once more
romac Feb 20, 2024
959e8aa
Implement skeleton for basic node
romac Feb 21, 2024
6436ff6
Simpler design for Protobuf encoding
romac Feb 21, 2024
9cacb53
Automatically clean elapsed timeouts
romac Feb 21, 2024
47581ba
Add basic node CLI with support for 3 peers
romac Feb 21, 2024
7c284aa
Allow node to propose a single fixed value
romac Feb 21, 2024
0099fd6
Cleanup
romac Feb 21, 2024
647eb67
Feed back proposal and vote into the driver
romac Feb 21, 2024
a7a6715
Cleanup
romac Feb 21, 2024
c0d3a93
Add a commit timeout
romac Feb 21, 2024
b1ad611
Only process timeouts for the current round
romac Feb 21, 2024
097aa42
Allow consensus to move across multiple heights
romac Feb 21, 2024
859e00c
Verify signature of votes and proposals
romac Feb 21, 2024
2ad08bc
Add `ValueBuilder` trait for building values
romac Feb 21, 2024
277c3d0
Install `protoc` on CI
romac Feb 21, 2024
5418db1
Remove redundant import
romac Feb 21, 2024
4915de4
Add integration for 3 non-faulty nodes
romac Feb 22, 2024
ff4359b
Fix node binary, and print each decisions
romac Feb 22, 2024
b4173d6
Exclude binary and Debug impls from coverage
romac Feb 22, 2024
378c152
Fix coverage
romac Feb 22, 2024
363b6f5
Add height to `ProposerSelector`
romac Feb 22, 2024
70a55cb
Clear prevote/precommit timeout when we get 2/3+ votes for a value
romac Feb 23, 2024
f2617c1
Track current round in span
romac Feb 23, 2024
1ee0e70
Remove `peers.toml` config, generate it on the fly
romac Feb 23, 2024
673796a
Use `turmoil` for writing broadcast test
romac Feb 22, 2024
862d140
Better error handling in broadcast network
romac Feb 26, 2024
d2a1881
Fix integration test and use 10 validators
romac Feb 26, 2024
cb1a5f9
Basic libp2p-based gossip as a standalone crate
romac Feb 26, 2024
37bc4d6
Use mDNS-based peer discovery for now
romac Feb 26, 2024
d12bf3c
Add gossip-based integration test with 10 nodes
romac Feb 26, 2024
59c80ed
Change CLI to use gossip layer
romac Feb 26, 2024
d67d73e
Fix tests on CI
romac Feb 27, 2024
c732936
Add validator address to proposals
romac Feb 27, 2024
d77f9cf
Re-add vote and proposal signature verification
romac Feb 27, 2024
6c47d84
Rotate the proposer each height and round
romac Feb 27, 2024
b442080
Less noisy logging
romac Feb 27, 2024
367f412
Use QUIC instead of TCP+Noise+Yamux
romac Mar 4, 2024
02debd2
Gossip as an actor
romac Mar 1, 2024
f13f2e1
Timers as an actor
romac Mar 1, 2024
42f7e31
Node as an actor
romac Mar 1, 2024
69bafe4
Proposal builder as an actor
romac Mar 1, 2024
ec3fe1d
Move all actors under a common module
romac Mar 1, 2024
97e674f
Add integration test using actors
romac Mar 4, 2024
00d4657
Use 3 correct nodes in integration tests
romac Mar 4, 2024
04fb1fd
Cleanup actors on test exit
romac Mar 4, 2024
f627267
Add a facility for making faulty nodes and + integration test
romac Mar 5, 2024
aad2e4a
Refactor integration test code
romac Mar 5, 2024
8de89d4
Use rand_chacha explicitly
romac Mar 6, 2024
31bb7cd
Increase test timeout
romac Mar 6, 2024
9294c59
Discard all gossip events
romac Mar 6, 2024
28038e8
Add new fault types
romac Mar 8, 2024
3917c68
Fix tests
romac Mar 11, 2024
c8fed3b
Move all actors under crate `malachite-actors`
romac Mar 19, 2024
9c739e1
Disable faulty tests
romac Mar 19, 2024
ba97e94
Cleanup dependencies
romac Mar 19, 2024
1c8d736
Move CLI into its own crate
romac Mar 19, 2024
877fb7f
Use node actor in CLI
romac Mar 19, 2024
4c71de7
Add support for multiple gossip channels
romac Mar 20, 2024
bd458f2
Merge branch 'romac/node' into romac/node-actor
romac Mar 25, 2024
62ad992
When receiving vote or proposal for higher height, move to that height
romac Mar 25, 2024
4b7e0c9
Document forwarding of proposal builder reply
romac Mar 25, 2024
8af6907
Remove outdated, redundant or duplicated code
romac Mar 25, 2024
77e481d
Cleanup
romac Mar 25, 2024
4f38b3c
Rename Node actor to Consensus
romac Mar 25, 2024
31808c7
Graceful shutdown
romac Mar 25, 2024
cd776d5
Cleanup
romac Mar 26, 2024
d667895
Introduce top-level `Node` actor
romac Mar 26, 2024
a4a4fb5
Remove faulty node implementation
romac Mar 26, 2024
3e0be30
Merge branch 'main' into romac/node-actor
romac Mar 26, 2024
63fc3a2
Add basic definitions for config file
romac Mar 26, 2024
c3fc1b6
Use `clap` for the CLI
romac Mar 26, 2024
89e46fb
Setup `tracing-subscriber` and `color-eyre` properly
romac Mar 27, 2024
066cc06
Check vote height.
ancazamfir Mar 30, 2024
e13f68d
Improve debuggability.
ancazamfir Mar 30, 2024
340d6b4
In the driver, discard votes and proposals for heights other than the…
romac Apr 2, 2024
c0e588e
Better logs in gossip
romac Apr 2, 2024
7c8939b
Buffer gossip event received while driver has not started a new heigh…
romac Apr 2, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/codespell/codespell.ini
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
[codespell]
skip = ./code/target
skip = ./code/target,./code/Cargo.lock
ignore-words = .github/codespell/words.txt
6 changes: 3 additions & 3 deletions .github/codespell/words.txt
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
crate
shs
ser
numer
manuel
numer
ser
shs
26 changes: 21 additions & 5 deletions .github/workflows/coverage.yml
Original file line number Diff line number Diff line change
Expand Up @@ -25,10 +25,14 @@ jobs:
steps:
- name: Checkout
uses: actions/checkout@v4
- uses: actions/setup-node@v3
- name: Install Protoc
uses: arduino/setup-protoc@v3
- name: Setup Node
uses: actions/setup-node@v3
with:
node-version: "18"
- run: npm install -g @informalsystems/quint
- name: Install Quint
run: npm install -g @informalsystems/quint
- name: Setup Rust toolchain
uses: actions-rust-lang/setup-rust-toolchain@v1
with:
Expand All @@ -39,7 +43,15 @@ jobs:
- name: Install cargo-llvm-cov
uses: taiki-e/install-action@cargo-llvm-cov
- name: Generate code coverage
run: cargo llvm-cov nextest --workspace --exclude malachite-itf --all-features --ignore-run-fail --lcov --output-path lcov.info
run: |
cargo llvm-cov nextest \
--workspace \
--exclude malachite-itf \
--ignore-filename-regex node/bin \
--all-features \
--ignore-run-fail \
--lcov \
--output-path lcov.info
- name: Generate text report
run: cargo llvm-cov report
- name: Upload coverage to Codecov
Expand All @@ -61,10 +73,14 @@ jobs:
steps:
- name: Checkout
uses: actions/checkout@v4
- uses: actions/setup-node@v3
- name: Install Protoc
uses: arduino/setup-protoc@v3
- name: Setup Node
uses: actions/setup-node@v3
with:
node-version: "18"
- run: npm install -g @informalsystems/quint
- name: Install Quint
run: npm install -g @informalsystems/quint
- name: Setup Rust toolchain
uses: actions-rust-lang/setup-rust-toolchain@v1
with:
Expand Down
8 changes: 6 additions & 2 deletions .github/workflows/mbt.yml
Original file line number Diff line number Diff line change
Expand Up @@ -27,10 +27,14 @@ jobs:
steps:
- name: Checkout
uses: actions/checkout@v4
- uses: actions/setup-node@v3
- name: Install Protoc
uses: arduino/setup-protoc@v3
- name: Setup Node
uses: actions/setup-node@v3
with:
node-version: "18"
- run: npm install -g @informalsystems/quint
- name: Install Quint
run: npm install -g @informalsystems/quint
- name: Setup Rust toolchain
uses: actions-rust-lang/setup-rust-toolchain@v1
- name: Install cargo-nextest
Expand Down
12 changes: 9 additions & 3 deletions .github/workflows/rust.yml
Original file line number Diff line number Diff line change
Expand Up @@ -31,25 +31,31 @@ jobs:
steps:
- name: Checkout
uses: actions/checkout@v4
- uses: actions/setup-node@v3
- name: Install Protoc
uses: arduino/setup-protoc@v3
- name: Setup Node
uses: actions/setup-node@v3
with:
node-version: "18"
- run: npm install -g @informalsystems/quint
- name: Install Quint
run: npm install -g @informalsystems/quint
- name: Setup Rust toolchain
uses: actions-rust-lang/setup-rust-toolchain@v1
- name: Install cargo-nextest
uses: taiki-e/install-action@cargo-nextest
- name: Build code
run: cargo nextest run --workspace --all-features --no-run
- name: Run tests
run: cargo nextest run --workspace --all-features
run: cargo nextest run --workspace --all-features --no-capture

clippy:
name: Clippy
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Install Protoc
uses: arduino/setup-protoc@v3
- name: Setup Rust toolchain
uses: actions-rust-lang/setup-rust-toolchain@v1
with:
Expand Down
72 changes: 57 additions & 15 deletions code/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,15 @@
resolver = "2"

members = [
"actors",
"cli",
"common",
"driver",
"driver",
"gossip",
"itf",
"network",
"node",
"proto",
"round",
"test",
"vote",
Expand All @@ -18,18 +24,54 @@ license = "Apache-2.0"
publish = false
rust-version = "1.77"

[workspace.lints.rust]
# unused_crate_dependencies = "warn"

[workspace.dependencies]
derive-where = "1.2.7"
ed25519-consensus = "2.1.0"
futures = "0.3"
glob = "0.3.0"
itf = "0.2.3"
num-bigint = "0.4.4"
num-traits = "0.2.17"
pretty_assertions = "1.4"
rand = { version = "0.8.5", features = ["std_rng"] }
serde = "1.0"
serde_json = "1.0"
serde_with = "3.4"
sha2 = "0.10.8"
signature = "2.1.0"
malachite-actors = { version = "0.1.0", path = "actors" }
malachite-cli = { version = "0.1.0", path = "cli" }
malachite-common = { version = "0.1.0", path = "common" }
malachite-driver = { version = "0.1.0", path = "driver" }
malachite-gossip = { version = "0.1.0", path = "gossip" }
malachite-network = { version = "0.1.0", path = "network" }
malachite-itf = { version = "0.1.0", path = "itf" }
malachite-node = { version = "0.1.0", path = "node" }
malachite-proto = { version = "0.1.0", path = "proto" }
malachite-round = { version = "0.1.0", path = "round" }
malachite-test = { version = "0.1.0", path = "test" }
malachite-vote = { version = "0.1.0", path = "vote" }

async-trait = "0.1.77"
clap = { version = "4.5.4", features = ["derive"] }
color-eyre = "0.6"
derive-where = "1.2.7"
ed25519-consensus = "2.1.0"
futures = "0.3"
glob = "0.3.0"
humantime-serde = "1.1.1"
itertools = "0.12"
itf = "0.2.3"
libp2p = { version = "0.53.2", features = ["macros", "mdns", "identify", "tokio", "ed25519", "quic", "tls", "gossipsub"] }
libp2p-gossipsub = { version = "0.46.1" }
multiaddr = "0.18.1"
num-bigint = "0.4.4"
num-traits = "0.2.17"
pretty_assertions = "1.4"
prost = "0.12.3"
prost-build = "0.12.3"
prost-types = "0.12.3"
ractor = "0.9.6"
ractor_actors = { version = "0.4.0", default-features= false }
rand = { version = "0.8.5", features = ["std_rng"] }
rand_chacha = "0.3.1"
serde = "1.0"
serde_json = "1.0"
serde_with = "3.7"
sha2 = "0.10.8"
signature = "2.2.0"
thiserror = "1.0"
tokio = "1.36.0"
tokio-stream = "0.1"
toml = "0.8.10"
tracing = "0.1.40"
tracing-subscriber = "0.3.18"
32 changes: 32 additions & 0 deletions code/actors/Cargo.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
[package]
name = "malachite-actors"
version.workspace = true
edition.workspace = true
repository.workspace = true
license.workspace = true
publish.workspace = true

[lints]
workspace = true

[dependencies]
malachite-common.workspace = true
malachite-driver.workspace = true
malachite-gossip.workspace = true
malachite-network.workspace = true
malachite-node.workspace = true
malachite-proto.workspace = true
malachite-test.workspace = true
malachite-vote.workspace = true

async-trait = { workspace = true }
libp2p = { workspace = true }
ractor = { workspace = true, features = ["async-trait"] }
rand = { workspace = true }
tokio = { workspace = true, features = ["full"] }
tracing = { workspace = true }

[dev-dependencies]
malachite-test = { workspace = true }
rand_chacha = { workspace = true }
tracing-subscriber = { workspace = true, features = ["fmt"] }
Loading
Loading