GPU

Your local models, Studio images and Studio videos all need the GPU, and one GPU can do one of them at a time. The Engine keeps a queue: every app asks before it uses the GPU, and jobs take turns. Chat goes first, then images, then video. Endpoint models do not use your GPU and never wait in the queue.

Hub > GPU shows the queue. It is for watching only: it has no buttons.

What the screen shows#

How local models take turns#

• Requests for the model that is in memory share the GPU and run together, also from different apps and windows.
• A request for another local model waits until the running answers end. Then the Engine unloads the model, loads the other one and answers.
• Requests are served in the order they arrive. Hangar bots wait behind your other work.
• Endpoint models skip the queue. They answer at the same time as a local model.

Switch grace#

Agents send many short requests in a row. So the loaded model does not lose the GPU between two steps, the Engine waits up to 5 seconds after its last answer for the next request of the same session before it switches. A waiting model is held back at most 20 seconds from when it joined the queue. There is no wait when the other model is already in memory.

What the apps show while they wait#

While a request waits for the GPU or a model loads, the app says why:

Studio and your models#

• When Studio starts an image or video job, the chat model is unloaded from the GPU for a while. The next chat request loads it again, which can make that first answer slower.
• A chat request stops Studio's image engine, which then loads again for the next job.
• Studio's image engine stops by itself after 5 idle minutes and gives the memory back.
• A job that stops reporting progress is removed from the GPU after about a minute, so it cannot block the queue.
• Studio shows its own place in the queue, for example Waiting for the GPU. Hub > Home shows it too, under Right now.

FAQ#