yes... they are using what is known as the "power method" of iterating to where the steady state vector is. There's a ton of literature out there particularly on finite state markov chains, which discusses the nuances behind this. Iterative methods, particularly for ...
This novel technique does not rely on the numerical solution of the steady-state Kolmogorov equations of the Continuous Time Markov Chain as it is common for this kind of systems but it considers the system in its Markov Decision Process setting. This technique, known as value extrapolation, ...