Scheduling every job on a GPU that can only hold one model

Details werden geladen...

https://dev.to/anyesh/scheduling-every-job-on-a-gpu-that-can-only-hold-one-model-1n

Teilen bei Facebook
Teilen bei Twitter
Teilen bei Pinterest
Per Mail empfehlen

Scheduling every job on a GPU that can only hold one model - DEV Community

I run this homelab on two small boxes. One never turns off and just serves DNS and background services; the other has a GPU with a memory pool that can hold exactly one loaded model at a time, nothing more. This post covers the queue and gateway that constraint forced me to build, and the network shape wrapped around both boxes.

hearth is what I built to schedule every job on my homelab's GPU - DEV Community

https://dev.to/anyesh/hearth-is-what-i-built-to-schedule-every-job-on-my-homelabs-gpu-534c

Login

Scheduling every job on a GPU that can only hold one model - DEV Community

Ähnliche Seiten

hearth is what I built to schedule every job on my homelab's GPU - DEV Community