Description
Problem 1.
Please watch the “Lecture 14 – Rational Decision (part 2): Discounting section” for the description
of the problem.
Question: For which 𝛾 are left and right equally good when in state b? Please explain it
mathematically.
Problem 2.
Consider a Markov chain for X specified by the following transition diagram. Please express all
final answers as simplified fractions.
2.1. Given that X0 = 1, find P (X1) and P (X2).
P(X1 = 1) = _____________
P(X1 = 2) = _____________
P(X2 = 1) = _____________
P(X2 = 2) = _____________
2.2. Find P (X∞), the stationary distribution of our Markov Chain.
P (X∞=1) = ____________
P (X∞=2) = ____________
