Skip to content
AWS Machine Learning Blog· Muru Bhaskaran·· 3 h agoAI score34

Using NVIDIA Run:ai GPU Fractioning on Amazon EKS

Using NVIDIA Run:ai GPU Fractioning on Amazon EKS

AI summary

This walkthrough shows how to configure NVIDIA Run:ai GPU Fractioning on Amazon EKS so multiple Pods share one GPU with enforced per-Pod memory limits. On a single g5.xlarge with an A10G, two Pods each get an 8 GiB contract and allocate 6 GiB, while a 9 GiB request is rejected. It uses KAI Scheduler and the GPU Fractioning Operator from the open-sourced NVIDIA Run:ai.

Source:AWS Machine Learning Blog · aws.amazon.com