New Research Proposes Using Small Models for Function Calling and Inference Optimization

New Research Proposes Using Small Models for Function Calling and Inference Optimization

Recent research published on arXiv suggests that smaller language models may be better suited for specific tasks within larger AI systems than previously understood.

According to a paper titled “Small Reasoning Models are Instruction Followers in Function Calling” (arXiv:2608.22472), researchers have introduced a framework called Instruction-Followed Function Calling (IFFC) that “decouples function-calling logic from the primary LLM and delegates it to a dedicated smaller model.” The research demonstrates that large language models “achieve superior accuracy in function calling in instruction-following contexts (i.e., standard user-assistant interactions) rather than a tool calling context.” The method “consistently outperforms both native function calling (NFC) and prompt-based function calling (PFC) baselines, with particularly strong gains on reasoning-oriented LLMs,” according to the paper.

In related work, the CacheSpec framework (arXiv:2607.20507), accepted by EMNLP 2026, explores using small models for “lightweight and structured operations such as variable extraction, program binding, and generation acceleration.” According to the research, experiments showed latency speedups of “up to about 3.1× latency speedup” and throughput improvements of “about 2.8× over PoT-style methods.”

The CacheSpec paper notes that “the sweet spot for small models in large-model inference systems lies not in solving complex tasks independently, but in performing lightweight, structured, and verifiable auxiliary operations.”