Positive recurrence of the fork-join LLM engine network

Consider a fork-join network of four LLM agents indexed by j{0,1,2,3}j\in\{0,1,2,3\}. Requests arrive at agent 00 with rate λ\lambda, fork into tasks processed by agents 11 and 22, and are merged and routed to agent 33 after both tasks complete. Let the token sizes at agent jj be (mpj,mdj)(m^j_p,m^j_d), and let agent jj have parameters bjb^j and tbjt^j_b. Fork-join stability conjecture. If

λ(mpj+mdj)<bj/tbjfor each j{0,1,2,3},\lambda(m^j_p+m^j_d)<b^j/t^j_b\quad\text{for each }j\in\{0,1,2,3\},

then, under any (Kpj,Kdj)(K^j_p,K^j_d)-FCFS work-conserving scheduling algorithm at each agent j{0,1,2,3}j\in\{0,1,2,3\}, the DTMC describing the system is positive recurrent. This proposes stability of the fork-join network whenever the offered load is below the service capacity at every agent; the source does not provide a resolution of the claim.

Sources & referencesView supporting material

Primary source

J. G. Dai, Tianze Deng, Yueying Li and Tianyi Peng, “Throughput-Optimal Scheduling Algorithms for LLM Inference and AI Agents”, arXiv:2504.07347 (2026).

Progress summary

Never refreshed

Nothing recorded yet. Refresh searches the literature and the public web for attempts on this problem, and writes the first summary here.

Solutions 0

No solutions have been posted yet.