Get in Touch

Course Outline

Introduction

  • Overview of deep learning scaling challenges.
  • Overview of DeepSpeed and its key features.
  • Comparison of DeepSpeed with other distributed deep learning libraries.

Getting Started

  • Setting up the development environment.
  • Installing PyTorch and DeepSpeed.
  • Configuring DeepSpeed for distributed training.

DeepSpeed Optimization Features

  • DeepSpeed training pipeline.
  • ZeRO (Zero Redundancy Optimizer for memory optimization).
  • Activation checkpointing.
  • Gradient checkpointing.
  • Pipeline parallelism.

Scaling Models with DeepSpeed

  • Basic scaling using DeepSpeed.
  • Advanced scaling techniques.
  • Performance considerations and best practices.
  • Debugging and troubleshooting techniques.

Advanced DeepSpeed Topics

  • Advanced optimization techniques.
  • Utilizing DeepSpeed with mixed precision training.
  • Running DeepSpeed on various hardware (e.g., GPUs, TPUs).
  • Managing multiple training nodes with DeepSpeed.

Integrating DeepSpeed with PyTorch

  • Integrating DeepSpeed into PyTorch workflows.
  • Using DeepSpeed with PyTorch Lightning.

Troubleshooting

  • Debugging common DeepSpeed issues.
  • Monitoring and logging.

Summary and Next Steps

  • Recap of key concepts and features.
  • Best practices for deploying DeepSpeed in production.
  • Further resources for learning more about DeepSpeed.

Requirements

  • Intermediate understanding of deep learning principles.
  • Practical experience with PyTorch or similar deep learning frameworks.
  • Familiarity with Python programming.

Target Audience

  • Data scientists.
  • Machine learning engineers.
  • Developers.
 21 Hours

Custom Corporate Training

Training solutions designed exclusively for businesses.

  • Customized Content: We adapt the syllabus and practical exercises to the real goals and needs of your project.
  • Flexible Schedule: Dates and times adapted to your team's agenda.
  • Format: Online (live), In-company (at your offices), or Hybrid.
Investment

Price per private group, online live training, starting from 3900 € + VAT*

Contact us for an exact quote and to hear our latest promotions

Testimonials (3)

Provisional Upcoming Courses (Contact Us For More Information)

Related Categories