Stoplight Estimates

Traditional projects thrive on estimates and humans are terrible at estimating. No really, we’re bad at this. Bad enough that college papers have been written correlating our failures with evolution. Bad enough that we’ve developed several competing terms to describe different perspectives on this same phenomena. Hofstader’s Law, Parkinson’s Law, and the Planning Fallacy all deal with the idea that you basically have no idea how long it will take you to finish what you’re doing. This is somewhat better than people who generally have no idea what they’re doing. At least your only problem is with time.

Poor estimations are more costly at the beginning of a project. This happens because a task done early is usually done early because other tasks depend on it. If the earlier task is misestimated and therefore delayed, this delay will be transferred to every other task that follows. For organizations doing operations, it’s easy to visualize how a machine going down early along the assembly line can impact the rest of the assembly line.

To compensate for poor estimates we’ve come up with some clever techniques. Jeff Sutherland talks about playing poker and Delphi estimating in his book on Scrum. Those techniques are great at helping human beings overcome errors in project estimations, but they can also cost quite a bit of time. Since time costs money, there are some scenarios where the estimation process isn’t worth the amount of effort to create the estimation.

Accurate estimates can significantly increase output, but making them timely can be a real challenge. Organizations conducting projects can put the estimations in the planning phase of their project. Organizations with ongoing operations need to incorporate the estimation process into their daily routine. Many organizations rely on historical data to do this, but doing this based upon historical data without checking its accuracy is a sure scenario leading to implementing Parkinson’s Law.

Every once in a while you meet someone who really gets it. You know the type. They don’t just understand their job, they really get how their job plays into the larger scheme of things. The United States Military Entrance Processing Command (USMEPCOM) has the responsibility to screen each applicant for military service. It’s a large job scaled across 65 different Military Entrance Processing Stations throughout the country. The MEPS have a job that focuses on meeting ongoing operational requirements, while its commanded by individuals primarily trained in more project oriented settings.

Recently I had the opportunity to visit the MEPS in Salt Lake City. There I discovered how their local management team has come up with an elegant estimation system to improve the efficiency of the entire MEPS. Applying their estimation system only cost 90 seconds in the first phase of processing, but gave the managers control over the processing flow for the entire organization.

The MEPS has the responsibility to review the applicant’s aptitude, physical health, and legal background. A typical applicant will arrive at the MEPS on an afternoon and take (or verify) their Armed Services Vocational Aptitude Battery (ASVAB) test. Afterwards, they’ll spend the night in a local hotel to arrive early the next morning at the MEPS for a medical evaluation, job selection, and background check. The medical evaluation is first, and as the first step of the day, it controls the processing flow for the rest of the MEPS.

The first step in a series controls the flow for the rest of the organization because it is a bottleneck for processing. If every applicant were a raw material and was processed the same exact way then the amount of processing time would be fixed for each iteration.

People aren’t raw materials. They’re our most valuable asset. A delay in medical can impact all the other sections. Because applicants and their medical history are different the amount of processing time varies between applications. Sometimes it varies significantly. Knowing how long to anticipate for each applicant can make a significant difference in controlling the flow at the MEPS.

The staff in Salt Lake generated an effective estimation system. At first, I mistook their ingenious method for a prioritization system. Discovering the beauty of its simple elegance has gotten me so excited that I’ve wanted to pen this post!

Most MEPS’s medical section will review the records of scheduled applicants a day or so before their arrival. In Salt Lake the team conducting the review and the team doing the medical processing was large enough that not everyone would remember each applicant just from a review the day before. Instead, after their records were reviewed someone would subscribe a green, yellow, or red dot to the applicant’s name tag that they would wear for the day. Green tagged applicants had records that were thinner than red tagged applicants. First thing in the morning the green tagged applicants would be front-loaded in the line to get some applicants on to the next phase of processing and reduce the down time in succeeding sections. People waiting to do work is a waste of those people’s efforts. This system increased efficiency within the MEPS, not just within a particular section.

Using this tagging system the medical section could control the pace of processing for the rest of the day. Once a small queue had built up of green tagged applicants a few of the yellow and red tagged applicants integrated into the system would be ready for further processing.

If this tag system were a priority system it would not increase efficiency even though the number of applicants processed through the medical section for the first hour would increase. Local efficiencies do not translate to organizational efficiencies. Imagine red tagged applicants being seen last by the medical staff, taking longer, and getting to the other sections later. If the processing queue at the other sections was exhausted while red tagged applicants were still in medical there would be no efficiency gained for the organization. This is why it’s not a priority system, it’s an estimation system that’s used to control flow for the entire organization.

People are bad at estimating, but we’ve compensated for our biology by creating techniques that allow us to overcome our bias. In order to be adopted, these estimation techniques must be efficient and simple to use. Leveraging the three color system the employees see every day on their way to work marries audience expectations with intended purpose.

An estimation technique is nothing more than a method for communicating and using the stop-light estimation system at the SLC MEPS is an efficient system for controlling processing flow for the next generation of military applicants. A good system can’t kelp you know what you’re doing, but it can help you get a better idea of how long it will take. Take that biology!

Management Reserves & Estimated Monetary Value

In project management, a Management Reserve is “an amount of the project budget withheld for management control purposes. These are budgets reserved for unforeseen work that is within the scope of the project. The management reserve is not included in the performance measurement baseline” (PMBOK).

There are three different types of management reserves identified in the PMBOK. These are listed as a Management Reserve, Contingency Reserve, and Activity Contingency Reserve.

Management reserve: Controlled by the management, but not necessarily the PM. This is an overall management reserve that could be spent on any project and has to be approved by the managers above the PM.

Contingency Reserve: a fund controlled by the PM and part of the project budget for any particular contingency within the scope of the project.

Activity Contingency Reserve: a fund specifically earmarked for a high-risk activity that could require more resources to complete. This fund is controlled by the PM and a part of the budget.

Earned Value Management is the process of removing the ambiguity between actual cost and planned cost within a project. EV is calculated as % complete X project budget. When EVM is applied to a project it gives the managers a clear line from which to measure project progress against other known calculations relative to the project’s time line and overall expenses.

Expected Monetary Value (EMV) is another tool in calculating risk costs. EMV is calculated by multiplying the value of each possible outcome by its probability of occurrence. The results of an EMV analysis can be used to determine the size of the reserves for the project. The decision tree listed as Figure 11-6 in the PMBOK shows how EMV can also be used to understand the monetary value of certain outcomes. When charted this decision tree matrix can show managers the probable consequences of their decisions to the project in relation to its cost and profitability to the organization.

Exaggerated Estimates

In this post I’d like to discuss how estimating is one of the riskiest aspects of any project.  The best book for understanding this isn’t on the reading list for my classes at UMUC, but that’s probably because it’s too engaging to be used as a text book.  

The resource in question is Goldratt’s novel on the Critical Chain.  In the book his characters are faced with a situation where each individual adds their own safety to their portion of the project.  Added up this cumulated safety takes up more time than the project itself does.  What his novel does better than anything else I’ve written is to explain how to work with actual human beings to remove as much safety from each individual line of effort and aggregate a safety for the project as a whole at the end.

Estimating is an extremely dangerous technique for managers.  Organizations with a low tolerance for failure will get unrealistically long estimates from their team members.  Why?  Because adding in the safety to the individual’s line of effort is key to maintaining their employment status.  The opposite extreme is also detrimental.  At this point some pragmatist would argue that picking the point in the middle as the right course of action, and they’d be wrong as well.  

Either extremes or the middle are significantly dangerous ways to run an organization when it comes to gathering estimates.  The right answer is to select a zone near one of these three points that depends on organizational culture, project complexity, and individual capabilities of the team members.  It’s important to pick a zone to operate within on this spectrum because as the project evolves its perception by team members will change and motivational tools (such as the value of estimates) will need to be responsive to these changes.

In conclusion estimates are dangerous to a project because people will want to give themselves as much cushion as possible.  To avoid this one needs to adopt a zone of influence where there are consequences for estimates with too much safety or unrealistic deadlines.  Senior leadership needs to adopt communications lines in a way to be able to understand the framework of the estimates below them.  An arbitrary application of an unrealistic timeline can destroy a great deal of good faith in an existing system and cause an adverse reaction for future projects.  Again, the best book to explain this isn’t on my college reading list.  It’s Goldratt’s novel on the Critical Chain.

Politicizing the Estimate–Someone Might Need To Look Good

When creating a project’s estimated cost it is often acceptable to pad or low ball the estimate depending upon the audience and intent.  For Scotty in Star Trek it was multiplying his actual estimates by four.  But Scotty’s audience was a fictional captain, and a loyal fanbase.  

Quite a few of the projects I’ve worked on involve logistical transportation of equipment by rail and by air.  In the case of rail movement the wooden blocks used to secure the rail cargo can sometimes be defective.  Padding the estimated number of blocks by 15% is an approved practice that has helped make me successful as some of the blocks will naturally be defective or damaged as part of transport.

For air movement each type of aircraft has certain restrictions on load capacity and placement of the load.  Accuracy is paramount and nothing but the actual weights and balances for the equipment is authorized.  For this type of estimate the loadmaster is the intended audience.  Since he is responsible for the safety of the cargo accurate information is valued above all else.

Low-balling a project’s estimate can also be a valuable tool.  In this scenario the company may be better off choosing one project over another.  Low-balling the estimate to entice the organization to choose the project may be acceptable, but hopefully the margin of error isn’t large enough to call into question the integrity of the project manager.  

A high estimate that results in a lot of cost savings at the end could serve to enhance the reputation of the project manager within the organization.  While the cost saving version may be easier to distinguish and measure, an accurate projection met on budget and on schedule is a highly commendable feat.  According to the average salary for an IT project manager in the U.S. is nearly $85,000.  The website also lists bonuses and other benefits ranging between approximately $15,000 and $20,000.  The potential to take home between 17%-23% of one’s income based upon performance against this measuring stick is certainly an influence to be conscious of the product as a useful tool.

Call me old fashioned by I’d prefer to be known for the accuracy of my estimates.  It could be my father’s voice in my head reminding me that honesty is the best policy or just the fact that honesty has worked for me this far.  I’ll still apply an industry approved margin of error, but I’ll be transparent about it in my report.