DTMC Applicatios Radomized Routig TELE4642: Week12
Radom Walk: Probabilistic Routig Radom eighbor selectio e.g. i ad-hoc/sesor etwork due to: Scalability: o routig table (e.g. geographical routig) Resiliece: ode failures, multiple paths Simplicity: easy algorithm Ca be modeled as radom walk i graph Formulate as a discrete-time Markov chai E.g.: 1 is source, 6 is destiatio 1 5 2 3 4 6 Network Performace 12-2
Metrics of iterest Ca we solve the Markov chai? Not irreducible, so o L Metrics of iterest: Where is the packet i k hops? Iterative: p [ k + 1] = p[ k] P How may hops are eeded for a 90% chace of the packet reachig the destiatio? Will the packet always get to the destiatio? What is the average umber of hops eeded? What is the impact of chagig trasitio probabilities? How would you compute the above aalytically? Network Performace 12-3
Exercise 1: Gambler s Rui You go ito a casio with $2 i your pocket, ad play i a slotmachie. Each play costs $1. I each play, if you wi (which happes with probability p), you get back double your moey (i.e. $2), whereas if you lose (with probability = 1-p), you lose your dollar. You stop playig ad leave the momet you either reach $4, or whe you have lost all your moey, whichever happes first. Draw the Discrete Time Markov chai (DTMC) that models your performace at the casio. Does this Markov chai have a steady-state solutio? Justify clearly. Compute the probability (i terms of p ad ) that you lose all your moey. Hit: Let u i deote the chaces of losig all your moey startig at state i. Compute the probability of losig all your moey whe p = = ½. Explai your aswer ituitively. Network Performace 12-4
Exercise 2 1 2 3 4 5 Cosider a small orgaizatio that sells a software product, ad has a corporate itraet cosistig of just five web-pages liked as below: Page 1 is the compay s mai page. It has liks to the software dowload page (page 2) ad the customer support page (page 3). Page 2 provides the software product for dowload. It liks back to the mai page (page 1). Page 3 is the customer support page, ad liks to the software dowload page (page 2), istallatio support page (pages 4) ad tech support page (page 5). Page 4 provides istallatio support for the software. It liks back to the mai page (page 1). Page 5 provides techical support for the software (bugs, patches, etc), ad has a lik back to the software dowload page (page 2). A search-egie uses the discrete-time Markov chai based PageRak algorithm for rakig these five web-pages, with the followig simplifyig assumptios: There are o liks to ay web-pages exteral to the orgaizatio. Each lik o a web-page is eually likely to be clicked by the user. A user avigatig the corporate web-site is assumed to click liks ad ot explicitly type the URL (i.e., the parameter α of the PageRak algorithm is set to 1; euivaletly, the user ever restarts the walk o the web-graph). Searchig for a word or phrase oly returs the top two web-pages that match. Network Performace 12-5
Exercise 2 (cotd.) Questios: Write the trasitio probability matrix P for the Markov chai correspodig to the above web-graph. Argue why the Markov chai for the above web-graph is irreducible ad aperiodic (ad coseuetly has a steady-state statioary solutio). What are two ways by which you ca compute the state probabilities for the above web-graph? Which method would you prefer to employ (usig pe ad paper, ot a computer), ad why? Now compute the statioary probabilities for each of the five states. A user searches for a keyword that is preset i all five web-pages. Which are the two top-raked web-pages that are retured by the search egie? Network Performace 12-6
Exercise 2 (cotd.) Now suppose you are the director of the customer support divisio, ad are free to modify web-pages 3, 4, ad 5 i ay way you wat, but you caot modify web-pages 1 ad 2 (which are maaged by aother group). Further, you are give the costraits that: (a) you ca add additioal liks i pages 3, 4, ad 5 (icludig liks from a page back to itself), but (b) you caot remove ay existig liks from these pages. Qualitatively (i.e. usig words, ot umbers) argue what liks you might wat to add to improve the rak of page 3 (the mai customer support page). Draw the modified web-graph that represets your solutio. Write the trasitio probability matrix P for the modified web-graph. For your modified web-graph from the previous part, compute the statioary state probabilities for all five web-pages. A user ow searches for a keyword that is preset i all five web-pages. Which are the two top-raked web-pages that are retured by the search egie? Network Performace 12-7
Exercise 3 4 1 2 3 Cosider a etwork of N = 4 idetical hosts itercoected i a rig, as show i the figure below. Oe day, a hacker ifects host 1 with a etwork virus that spreads i the etwork i the directio idicated by the arrows i the figure The virus spreads as follows: every morig, betwee 2am ad 6am, the virus o each ifected host has a ifectio probability p I = 0.2 of spreadig to the host adjacet i the clock-wise directio, provided the latter host is ot already ifected. Whe the system admiistrator comes i to work at 9am, he is able to idetify the hosts that are ifected, ad attempts to disifect (i.e., remove the virus from) hosts i the order i which they were ifected. Further, he is able to disifect at most oe host per day, with probability p D = 0.5, before he leaves at 5pm. Sice the virus always spreads i the clockwise directio ad the system admiistrator disifects them i the order of ifectio, ote that at ay poit of time, the ifected hosts are cotiguous o the rig. Describe the state of the system at midight each day, ad draw the Markov chai that models the spread of the virus. Is this Markov chai irreducible, i.e., is it possible to reach ay state from ay other state i a fiite umber of steps? Justify or show a couter-example. From the previous part, would you say that the Markov chai has a statioary solutio for the state probability vector? Network Performace 12-8
Exercise 3 (cotd.) Now suppose the hacker moitors the etwork betwee 2am ad 6am each day, ad if he fids that o host is ifected (i.e. the virus is eradicated from the etwork), reifects host 1 with probability p R = 0.2. Draw the Markov chai for this revised system. Is this Markov chai irreducible ad aperiodic? Justify. O a radom day what is the probability that the etwork is virus-free? Network Performace 12-9