Based on a graph analysis and transformation of MDPs, we address these problems. First we introduce an interval iteration algorithm , for which the stopping criterion is straightforward. Then we exhibit its convergence rate. Finally we significantly improve the upper bound on the number of ...