How to calculate the cost of running Agent on a consumer-grade graphics card: electricity bill, depreciation and cost per task

Sat, 11 Jul 2026 10:30:00 +0800

If you want to know whether a consumer-grade graphics card is worth running a local agent, you can’t just look at “how much the graphics card cost”, nor can you just compare the unit price per million tokens of the API. Agents will repeatedly call models, tools, and browsers; long contexts, failed retries, and idle standby will all change the actual cost.

The most practical way is to split the cost into four items: electricity, hardware depreciation, supporting equipment and manual maintenance, and then calculate it according to “each successful task” and “per million output tokens” at the same time.

First determine which costs you want to calculate

caliber	Questions suitable to be answered	Items that are easily missed
incremental cost	I already have a game console. How much does it cost to run Agent in my spare time?	The graphics card is already there, is it included in the depreciation?
full cost	Is it cost-effective to purchase a machine specifically for local agents?	Host, memory, SSD, cooling and depreciation
Cost per task	How much does it cost to automate a document and a code task?	Failure, retry, and manual review
throughput cost	What is the cost per million tokens to build a self-built inference service?	Enter token, KV cache and idle time

If the graphics card was originally used for gaming or other work, it would be more honest to calculate the incremental cost first; only when the entire machine is purchased for Agent, the depreciation of the entire machine is fully included.

Total monthly cost formula

Let’s use monthly caliber first, the data is easiest to obtain:

月度总成本 = 电费 + 硬件月折旧 + 配套服务费 + 人工维护成本

电费 = 实测整机平均功耗(kW) × 实际运行小时数 × 电价(元/kWh)
硬件月折旧 = (购入总价 - 预估残值) ÷ 使用月数
单次成功任务成本 = 月度总成本 ÷ 月成功任务数
每百万输出 token 成本 = 月度总成本 ÷ 月输出 token 数 × 1,000,000

The “average power consumption of the entire machine” should come from a socket power meter or UPS as much as possible, rather than just looking at the graphics card’s nominal TBP. The Agent also runs with CPU, memory, SSD, fans, and monitors; using only GPU power consumption will underestimate the electricity bill. Conversely, periods when the machine is on standby but not processing tasks should not be mixed into pure inference costs and is best recorded separately.

An example of replaceable parameters

Assume that a machine has a measured average power consumption of 280W under the actual Agent workload, runs 6 hours a day, and 30 days a month; the local electricity price is calculated as 0.8 yuan/kWh. The hardware purchased specifically for this purpose is amortized over 7,000 yuan, 36 months, and the residual value is ignored.

1
2
3

电费 = 0.28 × 6 × 30 × 0.8 = 40.32 元/月
硬件折旧 = 7,000 ÷ 36 = 194.44 元/月
月度基础成本 = 234.76 元/月

If 600 accepted tasks are completed this month, the basic cost will be approximately 0.39 yuan/task. The “completion” here cannot only be counted as model return; it should be based on business results such as successful script execution, work order closing, and manual inspection passing.

This is just an example calculation, not a universal price. It only makes sense to replace power consumption, usage time, electricity price, purchase amount and task volume with your own data.

Why is Agent more difficult to estimate than regular chat?

Chat models are often billed by token, and the cost of Agent is also affected by the execution path:

A task may include planning, retrieval, calling tools, reading results, and multiple rounds of corrections.
The longer the context, the more KV cache is occupied; when there is insufficient video memory, it will slow down, unload the CPU/RAM, and even trigger retries.
Browser automation, code testing, and file processing can stretch task times even if the GPU is not constantly fully loaded.
Multi-Agent parallelization will increase throughput, but may also increase memory contention, queuing, and failure rates.
In order to reduce the first token delay, resident model will increase idle power consumption.

Therefore, do not use the tokens/s of a short question and answer to represent the Agent cost. Select at least a set of real tasks, run them continuously for a week, and record input/output tokens, success rate, number of retries, wall clock time, and overall machine power consumption.

Two units that must be seen at the same time

every successful task

This is closest to business decisions. It is suitable for comparing fixed processes such as “local agent automatically processes PR”, “batch data collection”, “customer service draft generation”, etc.

`1`	`每个成功任务成本 = (本周期全部成本) ÷ 成功且验收通过的任务数`

Failed tasks cannot be deleted from the numerator. The power, inference time, and manual troubleshooting they consume are the true operating costs of local systems.

per million tokens

This is suitable for comparing on-premises inference services and cloud APIs, but distinguishing between inputs and outputs. The input of many agents includes long tool logs and historical context, and the input tokens are much more than the output tokens; if divided by the output tokens only, the number will be high. A more reliable approach is to record input, output and total tokens at the same time, and maintain the same statistical caliber.

How to collect your own data

There is no need to build a complex monitoring system from the beginning. First create a table and record the following fields for each run:

Field	Recording method
start and end time	Agent log or task queue
Whole machine power consumption	Smart outlet, power meter or UPS
input/output token	Inference service logs or client statistics
Success, failure, retry	Task status and error log
Manual review time	Work order, PR or random inspection record
Model and quantified version	Avoid mixing different models in the same set of data

Start with 20 to 50 representative tasks as a baseline. Then, tasks with short contexts, long contexts, many tool calls, and many retries are tested separately to see the impact of memory and model selection on costs. Video memory planning can be judged in conjunction with Local Model Quantification and Video Memory Selection.

How to calculate depreciation so as not to mislead yourself

There is no one answer to depreciation, but the assumptions must be written down first.

Existing graphics card: You can only calculate the electricity bill and additional maintenance as incremental costs; you can also set an opportunity cost for the graphics card as a more conservative full cost.
New equipment purchased: Graphics card, host, memory, SSD, and radiator should be included together; don’t just spread the price of the graphics card.
Lifespan: 24, 36 or 48 months will work, the key is comparing on-premises vs. cloud using the same assumptions.
Residual value: When the equipment is expected to be resold, the estimated residual value can be deducted from the purchase price; if it is uncertain, set it to 0, and the conclusion is more conservative.

NAS or small hosts usually consume less power, but memory, cooling, and expansion capabilities also limit models and concurrency. If you consider this route, you can first read Performance Judgment of NAS Deployment Ollama.

When comparing to cloud APIs, don’t just compare unit price

The cloud side can also be converted into cost per task:

`1`	`云端每任务成本 = 模型 API 费用 + 工具/API 费用 + 必要的存储与网络费用 + 人工复核成本`

Local is usually better suited for these situations:

There is a stable and predictable workload every day, and machine utilization is high.
Data cannot or is not expected to be sent to external services.
Already have a working GPU, the incremental cost is mainly electricity.
Self-maintainable models, drivers, service and crash recovery are acceptable.

The cloud is often better suited for these situations:

The load is sporadic and the machine will be idle most of the time.
Tasks require large models, very long contexts, or peak concurrency that far exceed local video memory.
The team is unwilling to take on the driver, model upgrades, monitoring and troubleshooting.
The task results are of high value, and the strongest model or multi-modal capabilities are required first.

Local deployment is not just a choice between two options. Stable, privacy-sensitive steps are placed locally; complex, low-frequency steps or steps that require stronger models are called to the cloud, which is often easier to control the total cost.

A practical decision line

Record the local monthly fixed cost as F, the variable power and maintenance costs per task as v, and the cloud per-task cost as c. When c > v, the approximate amount of tasks required to achieve local breakeven is:

`1`	`每月盈亏平衡任务数 = F ÷ (c - v)`

If c is less than or close to v, local will not be significantly cheaper due to scale; privacy, latency, controllability, and offline capabilities should be considered more at this time. This formula is only a financial filter and still requires failure rates and manual review time.

Common misunderstandings

Only look at the power consumption of the graphics card: The power, standby and cooling of the entire machine also need to be paid.
Think of nominal tokens/s as throughput: Real agents will be slowed down by context, tools, network, and test steps.
Ignore failed tasks: The process that is retried most frequently is often a cost black hole.
Only model charges are calculated: Browser, search, vector library, storage and human review charges may also apply.
Mixing data from different models: The quantization version, context length, and concurrency are different, and the results cannot be compared horizontally.

Summarize

The cost of running Agent on a consumer-grade graphics card should not just be answered by “a few kilowatt-hours of electricity per hour.” First use the measured power consumption of the entire machine to calculate the electricity cost, then add the hardware cost according to your own depreciation assumptions, and finally use the number of successful tasks and the amount of tokens to amortize it. After continuously recording a period of real workload, you can clearly judge whether to continue running locally, expand the video memory, or hand over some steps to the cloud API.

Cost Calculation on KnightLi Blog