This page is a methodology, not a verdict for or against a provider. It separates publisher statements, direct observations and analyst inferences so that each conclusion can be traced back to evidence.
Start with the documents that govern the decision
Hosted service evidence
For OpenAI business or developer services, begin with the current OpenAI Services Agreement, the applicable order form and service-specific terms. The Agreement identifies the provider’s pricing page; use the current OpenAI API pricing entry point and the official model catalogue when recording the available model and rate.
Do not copy a price or a contractual interpretation into a permanent comparison and assume it remains current. Record the page URL, retrieval time, applicable account or order-form context, and a private snapshot or hash permitted by your governance process.
Local workflow evidence
The official ComfyUI workflow documentation defines a workflow as a graph of connected nodes and explains that workflows can be stored as human-readable JSON. That makes the workflow file a useful unit for versioning and reproduction.
Use ComfyUI’s current system requirements to choose an installation path and supported hardware route. Use the official troubleshooting guide to capture environment details, error messages, workflow files and recent changes when a run fails.
Model identity and licence evidence
Do not treat “FLUX.2 Klein” as one interchangeable checkpoint. Black Forest Labs identifies FLUX.2 [klein] Base 4B under the Apache 2.0 licence and FLUX.2 [klein] 9B under its non-commercial licence. Those publisher model cards also state each variant’s intended usage, limitations and hardware guidance.
Record the exact model repository, revision and licence that were evaluated. Never transfer a requirement, result or permission from one variant to another.
Build one comparison packet
Freeze the source record
List every provider page, model card, licence, workflow document and pricing page used in the decision. Store the retrieval time, URL and a content hash or approved snapshot. Mark each statement as publisher fact, observed result or analyst inference.
Define the workload before choosing a route
Describe the input media, intended output, acceptance criteria, data sensitivity, expected usage pattern and operational constraints. Keep the same workload packet for each candidate. If different models or capabilities are compared, disclose that difference instead of presenting the result as a like-for-like model benchmark.
Create the hosted-service ledger
Capture the model identifier, billable units shown by the provider, failed or retried requests, storage or tool charges, rate limits relevant to the workload, data-handling setting, applicable agreement and support route. Calculate cost from the current official pricing source at evaluation time rather than a copied historical rate.
Create the local-inference ledger
Capture the exact checkpoint revision and licence, ComfyUI revision, workflow JSON, node and dependency versions, operating environment, hardware identity, energy-measurement method, administrator time, failed runs and storage requirements. Treat local hardware guidance as a starting condition, not proof of performance.
Run a controlled evaluation
Use the same accepted inputs and output criteria. Preserve raw logs and outputs. Separate setup time from execution time, successful runs from failures, and cold starts from already-loaded sessions. Report observed values only when the evidence bundle contains the method and raw record needed to reproduce them.
Review non-performance constraints
Compare data boundaries, retention controls, applicable usage policies, checkpoint licences, support expectations, operational ownership, portability and incident response. A faster or cheaper run does not settle those questions.
Decision-record template
| Decision area | Hosted evidence | Local evidence |
|---|---|---|
| Terms and licence | Applicable agreement, order form, service terms and policy snapshot | Exact checkpoint model card, licence and dependency licences |
| Cost | Provider usage record reconciled to the current official pricing source | Hardware allocation, energy method, maintenance, storage and operator time |
| Reproducibility | Model identifier, request settings, accepted inputs, logs and outputs | Workflow JSON, checkpoint revision, environment, dependencies, logs and outputs |
| Operations | Limits, support path, outage handling and provider change process | Capacity ownership, patching, monitoring, recovery and security controls |
| Data boundary | Configured handling and retention settings, verified against applicable documents | Storage locations, access controls, telemetry and any external nodes or services |
Publication gate
Publish a conclusion only when the source packet, calculation method, workflow or request configuration, raw observations and reviewer notes are available for audit. If a required item is missing, label the comparison incomplete. Do not replace missing measurements with estimates that read like test results.