Queuing theory

How about revisiting queuing theory for a while? You must be thinking - queuing theory?learnt in undergrad and never wanted to go back there.


But, some of the insights from queuing theory make far more sense when you have to manage resource effectively. What is a good way to manage resources, say a web servers? Run one web server at 100% utilization or  run at around  80% but add additional sever as required. For example, run 2 servers at 100% or run 3 at say 70%?


No right or wrong answer here? Both work equally well because 2 servers working 100% is almost same as 3 servers working 70%? Numerically yes but the difference is on system performance. Only time 2 servers working at 100% is better than 3 servers working at 70% is when the jobs arrive exactly at the same rate and every job takes exactly the same time. Test it on a simple scenario - consider a bank where every task takes 5 mins and customers arrive at the rate of one customer per 5 mins. So, the teller runs at 100% utilization. He finishes the task of one customer and the next one is just-in-time. Let's say the teller fumbles one task and it takes 1 minute longer. So, the customer now has to wait for 1 min and so is the next customer. If the rate of customer increases then wait gets worse. System gradually degrades. Only if the rate of job decreases and system gets to catch up. Moreover, no where there is so much certainty about how long does the job take and how is the rate. We need to approximate both and determine how many full time resource we need and then use variability to add some slack such as instead of 2 full time resource, using 3 resources at 70%.


The the impact of wait as seen in the above example becomes worse if there are down stream activities. More the number of steps in the process, worse it gets and the whole system becomes less effective.


This is why no resources such as web servers, db servers are run no more than 80% average utilization.


If we see the sense in this then why do we many times insist that people should be 100% busy at work. Your developers can not be and should not be 100% busy. Managers trying to achieve that are simply not reasonable. If you require 2 full time resources, plan to have 3 with 70% utilization. If the work load varies, you have a good mechanism to prevent negative impact from queuing. Let your people have some down time which they can use to do whatever. That downtime helps them put more hours when required.


Effect queuing theory is nicely described in the book 'Lean development'.


Lean Software Development: An Agile Toolkit for Software Development Managers (Paperback)
by Mary Poppendieck, Tom Poppendieck

Cheers!

Powered by Qumana