Is DP applicable to all environments?
Value iteration: In the value iteration method, we compute the optimal value function by taking the maximum over the Q function (Q values) iteratively:
Where . After finding the optimal value function, we extract the optimal policy from it.
Policy iteration: In the policy iteration method, we compute the optimal value function using the policy iteratively:
We will start off with the random policy and compute the value function. Once we have found the optimal value function, then the policy that is used to create the optimal value function will be the optimal policy.
If you look at the preceding two equations, in order to find the optimal policy, we compute the value function and Q function. But to compute the value and the Q function, we need to know the transition probability of the environment, and...