Plan capacity for bursty AI workloads without hiding queue time
An average daily job count can hide the burst that makes a workflow unusable. Measure when work arrives, how much can run safely at once and how long users wait, rather than using the fastest single generation as a capacity plan.
Describe the arrival pattern
Record representative job sizes, submission times and deadlines. Distinguish interactive requests from background batches. A workstation that handles occasional drafts may be unsuitable for many simultaneous users even when each individual graph runs successfully.
Use anonymised job metadata rather than retaining private prompts unnecessarily. Keep failures, cancellations and operator intervention in the record. A queue that appears empty because users abandoned it is not evidence that capacity is sufficient.
Measure safe concurrency
Test one job before adding overlapping work. Record model-loading behaviour, peak resource use and complete-file availability. Then increase overlap deliberately while keeping the task mix visible. Do not assume that multiple GPU processes share model memory efficiently or that available system RAM substitutes for device capacity.
A supported installation is the starting point, not the capacity result. Use the maintained hardware and troubleshooting documentation for compatibility questions, then measure your actual model and optional processing stages under realistic conditions.
Choose an explicit scheduling policy
Decide whether urgent short jobs can precede long batches and how operators can cancel or retry work. Preserve the relationship between a retry and its original job. Avoid unbounded parallel submission that makes failures and waiting times difficult to interpret.
If a remote route handles overflow, record its data boundary, output compatibility and current charges. Use the exact service's pricing documentation rather than a generic cloud estimate. A fallback that produces a different deliverable must be reviewed before it can silently replace the local route.
Report a useful service level
Summarise waiting time, execution time, accepted throughput and failures separately. Include slow and failed cases rather than presenting only an average or the quickest sample. State the tested demand pattern and environment so the result is not read as an unlimited capacity promise.
This revision removes the earlier assumption that commercial news determines local throughput. It provides a capacity-planning method without invented benchmark figures. Reassess the plan when job size, model version or user behaviour changes, and keep a documented stopping rule when the system cannot meet the required service level.