> For the complete documentation index, see [llms.txt](https://aisuko.gitbook.io/wiki/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://aisuko.gitbook.io/wiki/ai-techniques/training/deepspeed.md).

# Deep Speed

## Overview

[DeepSpeed](https://www.deepspeed.ai/) is a library designed for speed and scale for distributed training of large models with billions of parameters.

### Example

<https://huggingface.co/docs/peft/accelerate/deepspeed-zero3-offload>

### Reference

<https://github.com/microsoft/DeepSpeed>