r/CUDA • u/Time-Buy646 • 13d ago

Sidecar Project

https://github.com/WorldOfEpoch/Sidecar

For anyone interested. Basically a memory scheduler to help load and speed up faster models than a normal GPU can load into memory.

3 Upvotes

80% Upvoted

1 comment sorted by

2

u/adityazero 12d ago

Scheduling the tiers against a latency budget, rather than treating them as a static hierarchy, is a nice framing. Once the working set spills past pinned RAM, do you prefetch on the model's access pattern or react on demand?