Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reconnect after all nodes were offline #375

Conversation

harunzengin
Copy link
Contributor

@harunzengin harunzengin commented Dec 6, 2024

Fixes the reconnectivity issue after all nodes were down.

Sends a :host_up event whenever a new control connection is established so that Xandra.Cluster.Pool can start a pool and connect to that host.

Also, fixes the issue where the load balancing state diverges from the peers state.

Closes #373.

@harunzengin harunzengin changed the title Fix connectivity after all nodes offline Reconnect after all nodes offline Dec 6, 2024
@harunzengin harunzengin changed the title Reconnect after all nodes offline Reconnect after all nodes were offline Dec 6, 2024
lib/xandra/cluster/control_connection.ex Show resolved Hide resolved
lib/xandra/cluster/pool.ex Outdated Show resolved Hide resolved
@@ -327,8 +317,7 @@ defmodule Xandra.Cluster.Pool do
)
when is_peername(peername) do
# Not connected anymore, but we're not really sure if the whole host is down.
data = put_in(data.peers[peername].status, :up)
data = stop_pool(data, data.peers[peername].host)
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wait, why were doing this and why are we not doing it anymore?

Copy link
Contributor Author

@harunzengin harunzengin Dec 9, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch, we should also maybe_start_pools after stop_pool.

harunzengin and others added 2 commits December 9, 2024 12:22
Co-authored-by: Andrea Leopardi <an.leopardi@gmail.com>
@harunzengin
Copy link
Contributor Author

@whatyouhide This should be ready

Copy link
Owner

@whatyouhide whatyouhide left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Last comment and we're good to go 🎉

lib/xandra/cluster/pool.ex Outdated Show resolved Hide resolved
@harunzengin
Copy link
Contributor Author

@whatyouhide Done

@whatyouhide whatyouhide merged commit 5ef30e9 into whatyouhide:main Dec 11, 2024
5 checks passed
@whatyouhide
Copy link
Owner

Fantastic work @harunzengin 💟

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Xandra can't recover when all nodes are down
2 participants