LFM2.5-24B-A2B: Fast On-Device AI Agents
Explore how LFM2.5-24B-A2B brings tool-using AI agents to local devices, what its benchmarks show, and how to test it in your own stack.
LFM2.5-24B-A2B: What Fast Local AI Agents Could Change
Cloud AI is convenient - until latency, privacy, availability, or per-request cost becomes the thing that stops your project. That's why more teams are paying serious attention to on-device AI agents. The real question isn't just whether a model can answer a prompt locally. It's whether it can reason through a task, call tools reliably, and do it with a speed and memory footprint that actually match the hardware you have.
Liquid AI's LFM2.5-24B-A2B is built around that problem. It's a 24-billion-parameter mixture-of-experts model that activates about 2 billion parameters per token, with an emphasis on reasoning, tool use, and agent workflows. Liquid AI reports strong results on several agent benchmarks and fast decoding on selected Apple, AMD, and NVIDIA hardware. Those results are worth digging into - but they don't automatically make it the right choice for every app.
This article breaks down what the architecture means in practice, where the benchmark results look strongest, how to run the model, and what to verify before putting a local agent in front of users or business data.
What LFM2.5-24B-A2B is built to do
LFM2.5-24B-A2B is a sparse mixture-of-experts (MoE) model. Its name points to two practical properties: it has 24 billion parameters in total, but only around 2 billion are active for each token. That distinction matters. Total parameters affect storage and memory needs. Active parameters are more directly tied to the computation for each token. With MoE, you can get a larger pool of learned capacity without running everything for every step.
But "2B active" doesn't mean the model behaves like a dense 2-billion-parameter model on your device. The expert weights still have to be available to the inference system, and the real memory use depends on precision, quantization, runtime, and context length. Treat "2B active" as an efficiency clue - not as a guarantee that the whole model will fit on every device that can run a smaller LLM.
A reasoning-first model, not just a chat model
Liquid AI says the model is trained to reason before producing its final answer. The training setup is aimed at multi-step work: choosing a tool, building arguments for a call, interpreting the result, and continuing until it can answer. That lines up better with agent behavior than setups optimized only for short, back-and-forth conversation.
It also keeps the prior LFM2-24B-A2B architecture: 40 layers, 64 experts per MoE block with top-four routing, plus a hybrid design using gated short convolutions and grouped-query attention. The context window grows from 32,000 to 131,000 tokens. The tokenizer expands from 65,000 to 128,000 tokens, with the stated goal of improving tokenization for non-Latin languages.
A longer context can help when an agent needs to work through long documents, codebases, or extended multi-step histories. It also has costs. More context can increase memory use and slow down prompt processing, especially if your system keeps feeding the full conversation back into every request. Summarization or retrieval often beats "keep everything forever" in practice.
Training designed around tools and agent trajectories
The training recipe matters because agent quality isn't just a function of parameter count. Liquid AI reports scaling pre-training from about 17 trillion to 33 trillion tokens, then mid-training on about 2 trillion tokens at 32K context and 300 billion tokens at 131K context. The mid-training mixture focuses on reasoning, mathematics, code, instruction following, multilingual text, and long-context data. Tool calls and multi-turn trajectories show up throughout.
After that, it goes through multiple post-training stages: supervised fine-tuning, teacher specialization, non-agentic and agentic multi-domain on-policy distillation, and agentic reinforcement learning. The last stage uses agent harnesses including OpenClaw, Hermes Agent, Pi, and OpenCode.
The takeaway is pretty direct: the model is shaped for tool-using workflows, not only retrofitted to them afterward. That helps, but you still need application controls. Your agent still needs clear tool schemas, permissions, timeouts, and error handling - and a reliable way to stop when it's finished.
What the benchmark results actually tell you
Liquid AI evaluates LFM2.5-24B-A2B against Qwen3.6-35B-A3B, Gemma-4-26B-A4B-it, and Nemotron-3.5-Lightning across 16 benchmarks spanning tool use, agent workflows, knowledge, instruction following, mathematics, and coding. The results use vLLM and each benchmark's recommended generation settings. That context is useful, but these are vendor-reported numbers. Your prompts, tool implementations, serving stack, and hardware can absolutely change what you see.
The results also aren't a simple "best model" ranking. LFM2.5 leads or matches on several tool-use and instruction-following tests, while larger models hold advantages on some knowledge and coding evaluations.
| Benchmark | LFM2.5-24B-A2B | Qwen3.6-35B-A3B | Gemma-4-26B-A4B-it | Nemotron-3.5-Lightning |
|---|---|---|---|---|
| AutomationBench | 27.69 | 8.72 | 5.46 | 2.61 |
| BFCLv4 | 65.01 | 65.00 | 57.09 | 53.78 |
| ToolSandbox | 80.71 | 75.02 | 72.09 | 75.13 |
| LiveClaw | 48.09 | 38.19 | 41.25 | 37.72 |
| PinchBench | 83.18 | 89.93 | 77.12 | 84.20 |
| WebArena Verified | 26.75 | 37.62 | 25.62 | 28.69 |
| IFEval | 96.75 | 93.06 | 92.00 | 94.19 |
| SWE-Bench Verified (OpenCode) | 51.00 | 66.20 | 25.80 | 53.40 |
Where the model looks strongest
If you're building an agent, the most relevant signal is the cluster of tool-use and instruction-following results. According to the release, LFM2.5 leads or matches the listed competitors on AutomationBench, BFCLv4, ToolSandbox, LiveClaw, and τ³-Bench Banking. On AutomationBench, it scores more than three times the next-highest result in this comparison. It also leads three instruction-following benchmarks reported in the source: IFEval, IFBench, and Multi-IF.
Those strengths can show up in customer support workflows, local desktop assistants, internal operations tools, and any app where the model needs to follow constrained instructions while interacting with APIs. But benchmarks don't automatically predict what happens with your exact CRM action, your payment flows, or your recovery behavior when an API response is malformed. Those are app-level tests, not model-level trivia.
Where you should be cautious
LFM2.5 doesn't lead every category. Qwen3.6 scores higher on GPQA Diamond, WebArena Verified, SWE-Bench Verified, and SciCode in the provided comparison. Gemma-4 leads on HLE. Results on AA-Omniscience also vary a lot across models.
For knowledge-heavy work, Liquid AI's recommendation makes sense: pair LFM2.5 with retrieval or web search rather than assuming the model already contains everything your product needs. A model can help decide what to retrieve, but a search index or another trusted source still has to supply current or domain-specific facts.
For coding agents, you should treat benchmark differences as a reason to test on your own repository tasks - rather than assuming any single score will translate into delivery quality.
Before picking a model, build an evaluation set from real tasks. Include successful cases, ambiguous requests, tool failures, permission boundaries, and scenarios where the correct move is to ask the user instead of pushing ahead. Measure task completion and harmful or unintended actions, not only response quality.
Speed, privacy, and deployment trade-offs
Liquid AI reports 184 output tokens per second on an Apple M5 Max and 114 tokens per second on a Ryzen AI Max+ 395, describing this as up to 1.7 times faster than the next-best compared model while using less memory. On a single NVIDIA B300 GPU, it reports more than 21,000 output tokens per second at high concurrency.
Those numbers come from different setups. The edge figures are more relevant for local interactive use, while the B300 result looks like a server throughput scenario. Also: high-concurrency throughput isn't the same thing as "how fast one user's response will feel." Performance will change with quantization, prompt length, context size, batching, and the inference backend. If speed matters, benchmark your expected workload on the hardware you plan to deploy.
For runtimes, Liquid AI says it has day-one support across llama.cpp, MLX, vLLM, SGLang, and ONNX. That reduces integration friction, but different runtimes still have different performance and compatibility characteristics. Choose based on your target hardware and operational needs, then verify tool calling, streaming behavior, long-context performance, and memory use on your exact setup.
Local execution is a privacy control, not a privacy guarantee
Running a model locally can keep prompts and responses on a device or inside your organization's network. That's useful for internal documents, regulated data, or work that shouldn't rely on a third-party model endpoint. But "on-device" doesn't automatically mean private or compliant.
Your app may still send data to external tools, log prompts, sync conversations, or expose outputs through an agent harness. Look at the whole data path: model server, client, telemetry, tool APIs, backups, and how permissions work for users. If you're under requirements like GDPR, you'll need to assess actual processing and retention - not treat local inference as a blanket compliance shortcut.
Local operation also shifts the economics. You might reduce recurring API charges and avoid network round trips, but you'll pay upfront for hardware and the ongoing cost of maintenance. A local setup that's slower, harder to update, or weaker for a key task can end up costing more overall than a managed service. Compare total cost per completed task, including infrastructure and engineering time, not just token pricing.
Fit the model to a specific job
LFM2.5-24B-A2B is most convincing when the workload benefits from tool use, strict instruction adherence, and local execution. Examples include an offline-capable assistant that searches local files, an internal agent that calls approved business APIs, or a workstation coding assistant working against private repositories.
It's less clearly the right pick when the task depends on broad factual recall, top-tier coding performance across the board, or easy access to current web information without a retrieval layer. In those cases, retrieval, a web tool, a specialist model, or a hosted model may fit better. In practice, hybrid systems tend to be more reliable than forcing one model to do everything.
How to try the model in a real agent stack
Start with the model card and the source release instead of assuming a benchmark table matches your setup. The LFM2.5-24B-A2B model page on Hugging Face includes model files and implementation details. The release article covers Liquid AI's overview of training, evaluation, and deployment claims.
Load it with Transformers
The source instructions call for a recent Transformers install compatible with version 5.0.0 or later:
pip install -U transformersA basic Python example using the chat template and streamed generation looks like this:
from transformers import AutoModelForCausalLM, AutoTokenizer, TextStreamer
model_id = "LiquidAI/LFM2.5-24B-A2B"
model = AutoModelForCausalLM.from_pretrained(
model_id,
device_map="auto",
dtype="bfloat16",
# Uncomment on a compatible GPU:
# attn_implementation="flash_attention_2",
)
tokenizer = AutoTokenizer.from_pretrained(model_id)
streamer = TextStreamer(
tokenizer,
skip_prompt=True,
skip_special_tokens=True,
)
prompt = "What is C. elegans?"
input_ids = tokenizer.apply_chat_template(
[{"role": "user", "content": prompt}],
add_generation_prompt=True,
return_tensors="pt",
tokenize=True,
)["input_ids"].to(model.device)
model.generate(
input_ids,
do_sample=True,
temperature=0.1,
top_k=50,
repetition_penalty=1.05,
max_new_tokens=512,
streamer=streamer,
)
This is a starting point, not a production configuration. Confirm the selected dtype is supported on your hardware, measure memory use, and set generation limits appropriate for your application. If you're comparing models, keep prompts and generation settings consistent.
Connect an agent harness through an OpenAI-compatible endpoint
A common setup is to serve the model locally behind an OpenAI-compatible API, then point your agent harness at that endpoint. Liquid AI provides an example for Hermes Agent:
hermes config set model.provider custom
hermes config set model.base_url http://localhost:8080/v1
hermes config set model.default LFM2.5-24B-A2B
hermes config set model.context_length 131072
hermes config set model.api_mode chat_completions
hermes config set agent.tool_use_enforcement true
The port depends on the serving backend. The release lists 8080 for llama.cpp and MLX, 8000 for vLLM, 30000 for SGLang, and 1234 for LM Studio. Check the backend's current configuration before you copy a port into your app.
You can adapt the same overall pattern to other harnesses, but model identifiers, tool-call formats, and context settings need to match what the server actually implements.
For a first test, keep the agent's tool set small. Give it one read-only tool and one narrowly scoped action. Then verify that it chooses the right tool, passes valid arguments, handles errors, and stops instead of repeating calls. Only add write permissions once the read-only flow behaves predictably. It's not as thrilling as going broad on day one, but it's much easier to debug - and safer to operate.
Conclusion
LFM2.5-24B-A2B is a focused release: it targets a practical gap for local agents that can reason and use tools without shipping every request to a remote model service. The reported performance on agentic tasks, along with support across multiple inference runtimes, makes it a reasonable candidate for local assistants and private workflows.
Still, strong benchmark scores don't replace workload testing. It doesn't lead every knowledge or coding benchmark, and local deployment adds its own work: hardware planning, integration, and governance. Start with one narrow agent task, compare it against a baseline using real examples, and measure reliability, latency, memory, and total operating cost.
Explore the model on Hugging Face, then test it with a constrained tool set on the hardware you plan to use. That will tell you more than any headline token-per-second number ever will.