22 problems

Quadratic value-function conjecture. We conjecture that $V(t,x)$ is quadratic in $x$:

Markovian value-function conjecture. In view of the Markov structure, a solution $(V_t)_{t\geqslant 0}$ to this recursion must have the form

Let $[n]=\{1,\ldots,n\}$, let $\mathfrak{S}_n$ be the set of permutations of $[n]$, and let $\mathfrak{C}_n$ be the set of $n$-cycles in $\mathfrak{S}_n$. A join/union expression i…

The superhedging sets are subsets of $\mathbb{L}^2$ spaces indexed by time, and their approximate versions satisfy a set-valued Bellman's principle relating approximate superhedgin…

Near-optimality conjecture. The obtained results are close to the theoretical optimum under the assumed simplifications.

Extension conjecture. The main results obtained for MDPs should extend to generalized ADPs with state-dependent discounting under suitable stability and irreducibility assumptions.

The p-harmonic tug-of-war value-function conjecture. $p$-harmonic functions are related to the value functions of tug-of-war with noise and to solutions $u_{\varepsilon}$ of the dy…

Let $P_0,P_1,P_2,P_{11},P_{12},P_{21},P_{22}$ be the transition matrices defining the active pieces of $\hat{L}_{\pi}^{\dagger,0}$, let $x_0,x_1,x_2,x_{11},x_{12},x_{21},x_{22}$ be…

Let $\hat{U}^{\dagger,0}$ be the modified operator for the unknown stochastic shortest path problem, and let $\mathrm{CP}_{\mathrm{primalUnknown}}^{\dagger,\ell_1}$ be its associat…

Let $\hat{U}^{\dagger,0}:\mathbb{R}^N\to\mathbb{R}^N$ be the operator defined for the unknown stochastic shortest path problem, and let…

A reductive process has transition dynamics that can be represented by an upper-triangular structure, while an RMC is a recursive Markov chain. Weak reductivity conjecture. Allowin…

Let $T$ be the set of time periods, let $X$ be the set of feasible states, and let $x$ be a state strictly inside $X$. In affine value function approximation, the gradient of the a…

The non-linear stochastic dual dynamic programming algorithm can exhibit decreasing profit-generation performance over time under certain demand factors and slot capacities. Conjec…

Consider the update rule … where $Gamma_t=gamma_t\operatorname{diag}\{p(1),p(2),\ldots,p(n)\}$ and each $p(i)$ is the probability of selecting state $i$, with every state having no…

Consider sets $Xt subseteqmathbb{R}^{ntimes T}$ for $0leq tleq T$, a control set $Usubseteqmathbb{R}^m$, and cost functions … Let $\mathcal{F}$ denote the set of pairs $(f,x_0)$ su…

Iterative-refinement convergence conjecture. The observed convergence of the upper bound and of the cumulative moving average is due to the fact that Algorithm GBDP refines the val…

Multi-product pricing conjecture. The more general model where a purchase consumes units of multiple products (that is, components) can be solved using this framework and can gener…

Let a two-player zero-sum stochastic game have values that converge as the discount factor tends to $0$, with convergence uniform in the state space. For an initial state $k$, cons…

Risk-aware dynamic programming estimates risk at each iteration by solving an exact saddle-point optimization problem, while RaQL uses stochastic approximation with a stochastic ap…

Consider the linear double integrator problem $\ddot x=a$, and let the AMVI scheme denote the algorithm discussed for solving the associated dynamic programming equation. AMVI unde…

Piecewise computability conjecture. The dynamic programming algorithm for finding the minimizer of the ${\bar R}_C$-risk, together with exponential smoothing, could be used to find…

Conjectured optimal-policy form. The optimal policy of Player 1 is a stochastic matrix satisfying