[WIP] RL multinode #152
[WIP] RL multinode #152
Conversation
| @@ -69,6 +69,11 @@ def __init__( | |||
| self.stderr_file: Optional[TextIO] = None | |||
| self.stats = {} | |||
|
|
|||
| # Add node rank awareness | |||
| self.node_rank = int(os.environ.get("RANK", 0)) | |||
| self.port_offset = self.node_rank * 1000 # Ensure different port ranges for each node | |||
There was a problem hiding this comment.
why different port ranges for each node?
There was a problem hiding this comment.
Theoretically we should be able to update the sync port if the one selected by the toolkit environment is being used. But that's not true and I found out that the reason for those clashes is the subprocess subshell instead.
There was a problem hiding this comment.
I'm not sure I understand. These vllms are running on different nodes, aren't they?
| # Check GPU availability | ||
| num_gpus = torch.cuda.device_count() | ||
| print('###############################') |
There was a problem hiding this comment.
please use logger.info, and maybe smth like messages, "I'm rank X, training on Y GPU"
There was a problem hiding this comment.
That's just sanity checking, it will be deleted soon. It was there to check that that number is the same as Num processes, but things are clear now.
| "--deepspeed_multinode_launcher", | ||
| "standard", | ||
| "--same_network", | ||
| ]) |
There was a problem hiding this comment.
Would training multi-node training without accelerate work without DeepSpeed? If not, should add an exception.
| run = None | ||
|
|
||
| def safe_wandb_log(metrics, step): | ||
| if run is not None: |
There was a problem hiding this comment.
Can we just check the rank there and nothing unless it is 0?
No description provided.