A memory budget for mixed local and hosted AI workflows
A workflow that mixes local generation and hosted calls has several resource budgets, not one VRAM number. Map the stages first so device memory, system RAM, storage and remote-service costs can be measured independently and failures can be assigned to the correct component.
Draw the execution stages
List input preparation, local model loading, generation, remote calls, post-processing and output storage in order. Identify which stages can overlap and which must wait. Mark the boundary where data leaves the workstation and record the endpoint or integration that receives it.
Keep the actual model and software versions with the diagram. A convenient interface may hide several model components behind one button. Its visible simplicity does not establish a small working set or eliminate the need to understand where computation occurs.
Budget separate memory domains
Device memory holds a different set of resources from system RAM. PyTorch's allocated-memory measurement is a process-level tensor metric, not the total memory available to the device. Label each observation and keep the measurement interval and run conditions with the result.
Include model-loading peaks and optional processing stages. A successful sampler does not prove that decoding or enhancement will complete. Avoid counting system RAM as extra physical VRAM, even when offloading makes a previously failing graph complete.
Test overlap deliberately
Run each stage independently before enabling concurrency. Then compare the combined workload under the actual scheduling conditions. If failures appear only during overlap, record that relationship rather than changing several model settings at once.
Consider serialising expensive stages when interactive responsiveness is less important than reliability. Measure the waiting-time tradeoff explicitly. A configuration that completes every job slowly and one that fails intermittently should not be compared using only their fastest successful examples.
Keep commercial and hardware assumptions apart
Use the provider's current pricing documentation for remote charges, with the date and exact model recorded. A change in API billing does not alter the physical memory requirement of an unrelated local model. Keep cost and capacity columns separate in the decision record.
This revised page replaces the earlier headline-driven mixture of commercial speculation and unsupported optimisation claims. It offers a resource-budget method, not a measured performance result. Publish the stage diagram, environment and representative workload whenever a resource recommendation is presented as applicable to a specific deployment.