Positive recurrence of the fork-join LLM engine network
Positive recurrence of the fork-join LLM engine network
Consider a fork-join network of four LLM agents indexed by . Requests arrive at agent with rate , fork into tasks processed by agents and , and are merged and routed to agent after both tasks complete. Let the token sizes at agent be , and let agent have parameters and . Fork-join stability conjecture. If
then, under any -FCFS work-conserving scheduling algorithm at each agent , the DTMC describing the system is positive recurrent. This proposes stability of the fork-join network whenever the offered load is below the service capacity at every agent; the source does not provide a resolution of the claim.
Sources & referencesView supporting material
Primary source
J. G. Dai, Tianze Deng, Yueying Li and Tianyi Peng, “Throughput-Optimal Scheduling Algorithms for LLM Inference and AI Agents”, arXiv:2504.07347 (2026).
Progress summary
Nothing recorded yet. Refresh searches the literature and the public web for attempts on this problem, and writes the first summary here.
Solutions 0
Sign in to submit a solution.
No solutions have been posted yet.