Thank you for sending your enquiry! One of our team members will contact you shortly.
Thank you for sending your booking! One of our team members will contact you shortly.
Duration 21 hours
Course Outline
Introduction to Scaling Ollama
- Ollama’s architecture and key scaling considerations.
- Identifying common bottlenecks in multi-user deployments.
- Best practices for ensuring infrastructure readiness.
Resource Allocation and GPU Optimization
- Strategies for efficient CPU/GPU utilization.
- Considerations regarding memory and bandwidth.
- Implementing container-level resource constraints.
Deployment with Containers and Kubernetes
- Containerizing Ollama using Docker.
- Deploying Ollama within Kubernetes clusters.
- Managing load balancing and service discovery.
Autoscaling and Batching
- Designing autoscaling policies specific to Ollama.
- Applying batch inference techniques to optimize throughput.
- Navigating the trade-offs between latency and throughput.
Latency Optimization
- Profiling inference performance metrics.
- Implementing caching strategies and model warm-up procedures.
- Minimizing I/O and communication overhead.
Monitoring and Observability
- Integrating Prometheus for collecting metrics.
- Constructing dashboards with Grafana.
- Establishing alerting and incident response mechanisms for Ollama infrastructure.
Cost Management and Scaling Strategies
- Cost-aware approaches to GPU allocation.
- Evaluating cloud versus on-premises deployment options.
- Strategies for achieving sustainable scaling.
Summary and Next Steps
Requirements
- Practical experience with Linux system administration.
- A solid understanding of containerization and orchestration concepts.
- Familiarity with the deployment of machine learning models.
Audience
- DevOps engineers.
- ML infrastructure teams.
- Site reliability engineers.
Custom Corporate Training
Training solutions designed exclusively for businesses.
- Customized Content: We adapt the syllabus and practical exercises to the real goals and needs of your project.
- Flexible Schedule: Dates and times adapted to your team's agenda.
- Format: Online (live), In-company (at your offices), or Hybrid.
Price per private group, online live training, starting from 3900 € + VAT*
Contact us for an exact quote and to hear our latest promotions
