Key takeaways
Choose the most cost-effective approach that meets the task’s quality, coverage, timing and risk requirements. Include development, integration, checking, correction, exceptions and maintenance.
Ordinary code suits explicit rules and structured inputs. Task-specific models learn recurring predictions or classifications. Generative models handle varied language and produce new text or other content. A system can combine them.
Low setup effort can make a general-purpose model attractive. Repeated review and correction can reverse that advantage at operating scale.
Compare complete solutions on the same workload. Measure useful completed work, errors and human effort, including cases the system cannot finish.
Introduction
A general-purpose language model can make a new application look inexpensive to build. Describe the task, provide a few examples and it may produce a convincing result. Training a dedicated model or writing detailed rules can look slow by comparison.
But a demonstration leaves several costs unresolved. Someone must supply the right information, connect the output to existing systems, decide whether it is correct and handle the cases that fail. An answer that takes seconds to generate may still require an expert to repeat most of the task before using it. Conversely, maintaining rules for many changing document formats may cost more than checking a model’s output.
The choice is therefore an investment and operating decision. Which approaches can do the required work, and what will it cost to obtain acceptable results over the period the system will be used? This article compares the main options, explains where their costs arise and shows how to make that comparison without assuming that newer, smaller or simpler is necessarily better.
Quality and risk requirements determine which solutions are feasible
Cost-effectiveness requires a defined result. Routing a support request to a queue, forecasting product demand and preparing a sourced technical explanation are different tasks. A method suitable for one may be unable to produce the output another requires.
Specify the information available at the time of the decision, the required output and how that output will be used. Include the expected volume, response time and cases the system must cover. Distinguish errors by their consequences. A routing mistake that someone can correct is different from an incorrect payment or an unsupported statement sent to a customer.
Some requirements are constraints rather than costs to trade away. A solution that cannot meet a privacy restriction, a required error limit or a deadline is infeasible even if its model calls are cheap. For the remaining candidates, compare total costs at the required quality. If one produces a more valuable result, such as covering more cases or supporting a new service, include that additional benefit explicitly.
The current process belongs in the comparison too. It may already use search, templates, rules and specialist judgment. Compare a proposed system with that working process, not with a person doing every step manually from scratch.
Code, task-specific models and generation incur different kinds of work
Ordinary code fits inputs and rules that can be specified directly
Use software directly where the task has explicit inputs, transformations and rules. Reading named fields, calculating a price, checking a permitted value and applying a stated eligibility rule do not inherently require a learned model. Search and database queries may also retrieve the needed information without generating an answer.
Consider a hypothetical order import. An authenticated supplier submits a CSV file with a fixed schema containing product IDs and quantities. The product catalog and quantity limits are available in a database. The required result is a validated order or a list of rejected rows. Parsing, lookups and explicit rules provide a direct implementation. There is no unstructured request to interpret.
That advantage depends on stable assumptions. If suppliers regularly change formats, someone must update and test the importers. If staff resolve many rejected rows manually, that effort belongs in the cost of the code-based solution. Deterministic execution can consistently repeat a mistaken rule. The specification and implementation still need testing.
Task-specific models fit recurring predictions with measurable targets
A task-specific model learns a defined relationship from examples. Options include regression and tree-based models for structured data, and text classifiers or extractors for language. A language model need not generate prose. The BERT paper describes adapting a pretrained encoder for classification and token-level tasks.
Consider a second hypothetical task. Incoming support tickets must be assigned to an established set of queues. Historical tickets have adjudicated queue labels, and genuinely ambiguous requests can go to a triage team. Compare a rules-based router, a trained classifier and a general-purpose language model on the same held-out tickets. The output is a queue assignment, not an open-ended response.
The dedicated model needs representative training examples, label definitions, deployment and monitoring. Its investment may repay at sustained volume if it meets the routing requirements with less recurring processing or review. Changing queues, products or customer language can require new labels and retraining. Stable output categories help, but they do not establish that a dedicated model will be cheaper or more accurate.
Generative AI fits varied interpretation and open-ended outputs
Generative models can produce drafts, summaries, code and proposed structured outputs from varied instructions and context. They can be useful when writing a complete set of rules would be difficult, labeled examples are scarce or the requested output changes between tasks. A pretrained model can offer a practical starting point without a task-specific training project.
Suppose an engineer needs a draft comparison of three technical proposals. The source packet is fixed, each consequential claim must cite a passage, and an engineer will approve the final recommendation. A generative model can organize the comparison and draft the explanation. Search can locate passages, but it does not by itself write the required synthesis. The relevant comparison includes the engineer’s current reading and drafting process, as well as any narrower automation of recurring parts.
The question is how much checking and rewriting remains. Accurate quotations do not establish that the draft interpreted a limitation correctly or considered contrary evidence. Supplying sources and asking for citations helps make review possible; those instructions do not prove the draft is sound. The model’s flexibility is valuable only if the completed result justifies the integration, processing and review work.
Table 1. Typical starting conditions and characteristic costs to investigate. Methods can overlap and be combined; costs are not exclusive and the table is not a performance ranking.
The table highlights typical starting conditions and characteristic costs, rather than exhaustive requirements or exclusive costs. Integration, evaluation, exception handling and maintenance apply across approaches.
Combine methods where their responsibilities differ
These categories can overlap within a model as well as within a system. A generative model can itself be specialized for a task. A support system might use a classifier to route a request, generation to interpret an unusual explanation and ordinary code to calculate a remedy and enforce permissions. A document system might use optical character recognition, an extractor, database reconciliation and a generated review memo.
Each additional component creates integration and maintenance work. One general model can sometimes be easier to operate than several specialized paths. A hybrid is justified by its measured contribution, not by the number of techniques it combines.
Compare setup costs with recurring costs over a realistic operating period
A low-effort prototype and an inexpensive operating process are different achievements. Compare each feasible solution over a common period and expected workload. Separate costs that occur once from those that recur per item or after a change.
Include development and rollout time within that period, the workload each alternative can actually serve once available, and the cost of continuing the current process until the replacement takes over. Earlier availability can add value, but a quick prototype does not establish that integration and rollout will also be quick.
Include the following work:
Build and integrate: clarify requirements, obtain data, label training and evaluation examples, implement the approach and connect existing systems.
Process the workload: run software, models, retrieval and any retries, including infrastructure or provider charges.
Check and resolve: review outputs, correct mistakes, answer clarification requests and finish exceptions or handoffs.
Maintain and reevaluate: update rules, data, models, sources and integrations, then check that the changed system still meets its requirements.
Figure 1. Compare complete costs over the same operating period. Setup, recurring work and maintenance all count; the diagram shows cost categories, not their relative amounts.
Measure staff effort as well as cash spending. Released staff time may have value without reducing payroll. Avoid counting the same review work both in a per-item estimate and again as a separate operating cost. Include the consequences of remaining errors where they can reasonably be estimated, while retaining requirements that cannot responsibly be reduced to a monetary average.
Volume can change the result. A dedicated model with substantial setup work may become worthwhile when its recurring saving per item repays that investment. A short-lived or low-volume task may favor a general-purpose model even at a higher per-item cost. That comparison depends on the expected life of the task, maintenance differences and whether the estimated saving survives actual use.
There is no general cost ordering from code to task-specific models to generation. Rules can be expensive to maintain, a classifier can be expensive to label, and a generator can be expensive to verify. Simplicity matters through those costs and operational requirements, rather than as an independent reason to reject a more capable approach.
Verification effort can change which method is economical
The useful unit of comparison is a method together with the checks and exception handling needed to use its output. Structural checks can reject a missing field or an invalid category. They cannot establish whether a summary omitted the decisive condition or whether a proposed action matches the customer’s request.
Some tasks provide inexpensive independent checks. An order total can be recalculated from trusted prices and quantities. Other tasks require semantic judgment or new evidence. Reviewing a synthesis may mean returning to several sources and assessing whether the explanation represents them fairly. The cost of that review can exceed the cost of producing the draft.
Human review and model-based review both need evaluation. In Correlated Errors in Large Language Models, Kim and colleagues used a judge model’s own multiple-choice answers as reference labels to score other models. Shared wrong answers could then count as correct, inflating scores. Agreement between models is therefore not sufficient evidence that an output is correct. This cited result concerns benchmark judging; it supplies no error rate for the examples here.
A review threshold also changes the operating plan. Sending uncertain classifications to people can reduce errors in automatically accepted work while increasing the manual queue. Test the threshold’s effect on errors and workload rather than treating a confidence score as a guarantee. Guo and colleagues explain the distinction between classification accuracy and probability calibration; neither certifies an individual decision.
A stronger model may be cost-effective if it reduces difficult corrections, produces more checkable outputs or completes more of the required work. A narrower method may win if its output can be checked cheaply and its exceptions remain manageable. Both conclusions require evidence from the intended task.
Test complete alternatives before choosing one
Use representative cases with agreed expected results and consequences for errors. Keep evaluation cases separate from those used to fit models, write prompts or choose review thresholds. Include difficult inputs and unsupported cases, not only the examples that make a demonstration look successful.
Estimate operating error rates, review frequency and costs from cases that reflect the expected input mix. Additional challenge cases can test coverage and failure handling, but a test set enriched with difficult cases does not represent their frequency in routine work. Report those results separately or weight them to the expected mix before using them to estimate operating costs.
Compare feasible alternatives under the same requirements and access to information. Let each use appropriate checks and exception handling, then count the work those arrangements create. Record correct completion, consequential errors, unresolved cases, coverage, latency and human effort. An accurate automated subset with a large unfinished queue can be a poor solution to the complete task.
Start with a comparison proportionate to the decision. Eliminate methods that cannot meet the requirements, test the credible alternatives and investigate the assumptions that could change the result. A small internal drafting aid does not need the same evidence as a system making consequential external changes.
Choose the approach that delivers the required outcome most cost-effectively over its expected use. Record the assumptions behind that choice and when to revisit them, including changed inputs, growing volume, rising review effort or new capabilities.
Further reading
For the broader strategy and implementation choices, see Developing an AI strategy and the capabilities to deliver it. For a fuller treatment of costs, benefits and investment decisions, see The economics of AI-assisted work.