Paper Detail

Heartbeat Thinking: Speculative Reasoning and Tool Execution While Listening in Realtime Voice Agents

Zhenyu Zhang, Kewei Fu, Yongqi Tong, Chenzhi Zheng, Wei Wu, Jiang-Ming Yang, Jianshe Li

huggingface Score 17.5

Published 2026-10-19 · First seen 2026-10-23

General AI

Abstract

Tool-using voice agents often leave users waiting after speech ends, even when actionable information arrives earlier in the request. Speculative approaches begin work during speech, but chunk-bounded or interrupt-on-update generation can truncate a tool call before its arguments are complete. This couples computation time to speech updates, limiting useful work completed in advance. We introduce Heartbeat Thinking, which handles updated input through new Thinker invocations while allowing ongoing invocations and dispatched tools to finish. Periodic invocations read the latest partial transcript and a shared record of completed reasoning and tool results. These results help resolve missing arguments and initiate dependent calls, allowing preparation to progress across speech updates. Across SpeechFCEval, FDBench v3, and AudioCRAG, Heartbeat improves aggregate speculative tool coverage or retrieval reuse over a SHANKS-style windowed baseline, with benchmark-dependent effects on final task quality. On FDBench v3, it increases tool coverage by 16 percentage points and reduces mean final-response latency by 36.4\%. The results show that heartbeat thinking paradigm preserving ongoing computation during speech can turn listening time into reusable preparation for faster and more capable voice-agent responses.

Workflow Status

Review status
pending
Role
unreviewed
Read priority
now
Vote
Not set.
Saved
no
Collections
Not filed yet.
Next action
Not filled yet.

Reading Brief

No structured notes yet. Add `summary_sections`, `why_relevant`, `claim_impact`, or `next_action` in `papers.jsonl` to enrich this view.

Why It Surfaced

No ranking explanation is available yet.

Tags

No tags.

BibTeX

@misc{zhang2026heartbeat,
  title = {Heartbeat Thinking: Speculative Reasoning and Tool Execution While Listening in Realtime Voice Agents},
  author = {Zhenyu Zhang and Kewei Fu and Yongqi Tong and Chenzhi Zheng and Wei Wu and Jiang-Ming Yang and Jianshe Li},
  year = {2026},
  abstract = {Tool-using voice agents often leave users waiting after speech ends, even when actionable information arrives earlier in the request. Speculative approaches begin work during speech, but chunk-bounded or interrupt-on-update generation can truncate a tool call before its arguments are complete. This couples computation time to speech updates, limiting useful work completed in advance. We introduce Heartbeat Thinking, which handles updated input through new Thinker invocations while allowing ongoi},
  url = {https://huggingface.co/papers/2610.26233},
  keywords = {speculative generation, chunk-bounded generation, interrupt-on-update generation, Thinker invocations, partial transcript, shared reasoning record, dependent tool calls, SpeechFCEval, FDBench v3, AudioCRAG, SHANKS-style windowed baseline, tool coverage, final-response latency, huggingface daily},
  eprint = {2610.26233},
  archiveprefix = {arXiv},
}

Metadata

{}