🦌Deep Speed

Overview

DeepSpeed is a library designed for speed and scale for distributed training of large models with billions of parameters.

Example

https://huggingface.co/docs/peft/accelerate/deepspeed-zero3-offload

Reference

https://github.com/microsoft/DeepSpeed