Rolling restarts

Drain mode, added in 3.2.0, empties a node before you restart it.

/anchor drain lobby-2

1. The node stops receiving new players from the balancer.
2. Players on it get a chat notice and are moved to other lobbies in the same region, a few per second.
3. When the node is empty, or after drain.timeout-seconds, remaining players are moved at once.
4. /anchor status shows the node as drained. Restart it.
5. When it sends heartbeats again it rejoins the balancer on its own.

Scripting it

The Velocity console accepts anchor drain <node> --wait, which returns only when the node is drained. A simple restart loop:

for node in lobby-1 lobby-2 lobby-3; do
  rcon velocity "anchor drain $node --wait"
  ssh "$node" systemctl restart paper
  sleep 30
done

Game servers

Draining a game node only stops new matches from starting there. Running matches finish normally, up to drain.timeout-seconds.