Instrument an AI workflow's cost before optimising it
Optimisation needs a baseline that includes failures and human correction, not just the cheapest visible request. Instrument the workflow around an accepted output so remote charges, local processing and review effort can be compared without hiding costs in another stage.
Define the event record
Assign a job identifier and record submission, each model or tool stage, validation and final acceptance. Use non-sensitive labels rather than storing full private prompts in routine telemetry. Keep the actual model and endpoint identifiers so usage can be interpreted later.
Separate user waiting time from active compute time. A queued request can feel slow even when processing is quick. Record retries and cancellations as events linked to the original job rather than counting each attempt as a successful deliverable.
Attach the correct units
Consult current pricing documentation for the endpoint's billed units and record the date of the rate used. Do not assume that all tools are charged like text generation or that every returned usage field has the same meaning across providers.
For local stages, record elapsed time and the relevant resource observations. PyTorch memory accounting is a diagnostic metric, not a billing unit or a measure of output quality. Keep resource and financial quantities in separate fields so they cannot be added together accidentally.
Compare interventions fairly
Choose one change, such as reducing repeated preprocessing, separating an optional stage or selecting a different supported route. Run it against the same acceptance set as the baseline. Preserve the old configuration and include cases where the proposed improvement performs worse.
Review whether apparent savings were achieved by lowering output quality, shifting work to a person or increasing failures. If the acceptance rule changes, label the comparison as a different service level rather than a pure efficiency gain.
Keep the evidence usable
Summarise accepted outputs, failed attempts, total observed charges and human intervention separately. Small trial sets should be described as trials, not permanent forecasts. Obtain current rates before a substantial commitment and include operating costs that are not visible in an API response.
This retained article no longer links an advertising headline to invented local performance gains. It provides a measurement design. Repeat the comparison when workload mix, model version or pricing changes, and retain enough configuration detail to explain why the next result differs from the previous one.