Xianyu Zhu

Xianyu Zhu

Posts by Tags

distributed training

Understanding Gradient Compression in Distributed Training

less than 1 minute read

Published:

As deep learning models continue to grow in size, distributed training has become essential for reducing training time. However, the communication overhead between nodes can become a significant bottleneck. This is where gradient compression comes into play.

gradient compression

Understanding Gradient Compression in Distributed Training

less than 1 minute read

Published:

As deep learning models continue to grow in size, distributed training has become essential for reducing training time. However, the communication overhead between nodes can become a significant bottleneck. This is where gradient compression comes into play.

machine learning systems

Understanding Gradient Compression in Distributed Training

less than 1 minute read

Published:

As deep learning models continue to grow in size, distributed training has become essential for reducing training time. However, the communication overhead between nodes can become a significant bottleneck. This is where gradient compression comes into play.