Course Outline
Introduction
- Overview of deep learning scaling challenges.
- Overview of DeepSpeed and its key features.
- Comparison of DeepSpeed with other distributed deep learning libraries.
Getting Started
- Setting up the development environment.
- Installing PyTorch and DeepSpeed.
- Configuring DeepSpeed for distributed training.
DeepSpeed Optimization Features
- DeepSpeed training pipeline.
- ZeRO (Zero Redundancy Optimizer for memory optimization).
- Activation checkpointing.
- Gradient checkpointing.
- Pipeline parallelism.
Scaling Models with DeepSpeed
- Basic scaling using DeepSpeed.
- Advanced scaling techniques.
- Performance considerations and best practices.
- Debugging and troubleshooting techniques.
Advanced DeepSpeed Topics
- Advanced optimization techniques.
- Utilizing DeepSpeed with mixed precision training.
- Running DeepSpeed on various hardware (e.g., GPUs, TPUs).
- Managing multiple training nodes with DeepSpeed.
Integrating DeepSpeed with PyTorch
- Integrating DeepSpeed into PyTorch workflows.
- Using DeepSpeed with PyTorch Lightning.
Troubleshooting
- Debugging common DeepSpeed issues.
- Monitoring and logging.
Summary and Next Steps
- Recap of key concepts and features.
- Best practices for deploying DeepSpeed in production.
- Further resources for learning more about DeepSpeed.
Requirements
- Intermediate understanding of deep learning principles.
- Practical experience with PyTorch or similar deep learning frameworks.
- Familiarity with Python programming.
Target Audience
- Data scientists.
- Machine learning engineers.
- Developers.
Custom Corporate Training
Training solutions designed exclusively for businesses.
- Customized Content: We adapt the syllabus and practical exercises to the real goals and needs of your project.
- Flexible Schedule: Dates and times adapted to your team's agenda.
- Format: Online (live), In-company (at your offices), or Hybrid.
Price per private group, online live training, starting from 3900 € + VAT*
Contact us for an exact quote and to hear our latest promotions
Testimonials (3)
I really liked the end where we took the time to play around with CHAT GPT. The room was not set up the best for this- instead of one large table a couple of small ones so we could get into small groups and brainstorm would have helped
Nola - Laramie County Community College
Course - Artificial Intelligence (AI) Overview
Working from first principles in a focused way, and moving to applying case studies within the same day
Maggie Webb - Department of Jobs, Regions, and Precincts
Course - Artificial Neural Networks, Machine Learning, Deep Thinking
It felt like we were going through directly relevant information at a good pace (i.e. no filler material)