5 Each level of a fantasy computer game is set in a single location, Alphaworld, Betaworld, Chiworld or Deltaworld. After completing a level, a player goes on to the next level, which could be set in the same location as the previous level, or in a different location. In the first version of the game, the initial and transition probabilities are as follows.
Level 1 is set in Alphaworld or Betaworld, with probabilities 0.6, 0.4 respectively.
After a level set in Alphaworld, the next level will be set in Betaworld, Chiworld or Deltaworld, with probabilities \(0.7,0.1,0.2\) respectively.
After a level set in Betaworld, the next level will be set in Alphaworld, Betaworld or Deltaworld, with probabilities \(0.1,0.8,0.1\) respectively.
After a level set in Chiworld, the next level will also be set in Chiworld.
After a level set in Deltaworld, the next level will be set in Alphaworld, Betaworld or Chiworld, with probabilities \(0.3,0.6,0.1\) respectively. The situation is modelled as a Markov chain with four states.

1. Write down the transition matrix.
2. Find the probabilities that level 14 is set in each location.
3. Find the probability that level 15 is set in the same location as level 14 .
4. Find the level at which the probability of being set in Chiworld first exceeds 0.5.
5. Following a level set in Betaworld, find the expected number of further levels which will be set in Betaworld before changing to a different location. In the second version of the game, the initial probabilities and the transition probabilities after Alphaworld, Betaworld and Deltaworld are all the same as in the first version; but after a level set in Chiworld, the next level will be set in Chiworld or Deltaworld, with probabilities \(0.9,0.1\) respectively.
6. By considering powers of the new transition matrix, or otherwise, find the equilibrium probabilities for the four locations. In the third version of the game, the initial probabilities and the transition probabilities after Alphaworld, Betaworld and Deltaworld are again all the same as in the first version; but the transition probabilities after Chiworld have changed again. The equilibrium probabilities for Alphaworld, Betaworld, Chiworld and Deltaworld are now 0.11, 0.75, 0.04, 0.1 respectively.
7. Find the new transition probabilities after a level set in Chiworld. OCR is committed to seeking permission to reproduce all third-party content that it uses in its assessment materials. OCR has attempted to identify and contact all copyright holders whose work is used in this paper. To avoid the issue of disclosure of answer-related information to candidates, all copyright acknowledgements are reproduced in the OCR Copyright Acknowledgements Booklet. This is produced for each series of examinations, is given to all schools that receive assessment material and is freely available to download from our public website (\href{http://www.ocr.org.uk}{www.ocr.org.uk}) after the live examination series.
   If OCR has unwittingly failed to correctly acknowledge or clear any third-party content in this assessment material, OCR will be happy to correct its mistake at the earliest possible opportunity. For queries or further information please contact the Copyright Team, First Floor, 9 Hills Road, Cambridge CB2 1PB.
   OCR is part of the Cambridge Assessment Group; Cambridge Assessment is the brand name of University of Cambridge Local Examinations Syndicate (UCLES), which is itself a department of the University of Cambridge.