Thursday, May 28, 2015

Challenges in Productivity Measurement

The classical definition of productivity is Productivity = Output/ Input where output can be number of parts, work items, documents, lines of code etc... and input is typically the effort in terms of number of hours, person days, resources etc... So unit for productivity will be something like work items per hour or line of code per day.

Now going by the definition, it sounds deceptively simple to measure productivity. All you need is a unit for the output and the input! But, productivity measurement is unfortunately very tricky business.

The main challenges that you should look out for are:

Clear Definitions

Work unit definitions form the crux of your measurement system. Every work item needs to have a clearly defined scope and size. Ideally, size is essential before you can begin with any kind of measurement. In reality, you will come across several challenges in sizing:
  • Sizing measures are either not available across projects or are so varied that it is difficult to process the data
  • Project members are not trained on a sizing technique and might not be feasible to train them in one shot to start sizing
  • If the resources are trained on sizing, there is also a chance that their understanding varies and hence when you dig deeper, the basis for their sizing turns out to be different
Now, some might argue that these challenges can be overcome or should not exist if the sizing program is implemented correctly but in a large organization, these challenges are often present at a ground level.
One way to get a quick start on the measurement is to at least have a very clear definition of your work item and preferably some complexity buckets to help you estimate better

Data Sanity (Measurement System Analysis)

In Six Sigma, before we begin analysing the data, we first validate the data sanity or the measurement system analysis. We check for any unknown variations, consistency in data collection method, whether people are able to identify the categorization of data correctly etc...

This exercise is extremely important when establishing productivity baselines as well. Productivity data will be used to measure a team's effectiveness and any measurement error in it gives an erroneous picture of the team. Also the basis of improvement actions will be wrong and will lead to wasted efforts.

Measurement Unit

The measurement unit can be any unit which will give you an appropriate representation of the work done and it's complexity. For example, if we were to measure a Developer's productivity, the number of function points per day or per month would be appropriate. Some people even measure the lines of code per day.

The measurement needs to accurately reflect the quantity and the quality of work delivered. So if lines of code is a measure, it will definitely tell you the quantity but LOC cannot help you with the quality. I can write one function really well and finish it in 40 lines and a badly written code can go up to 100 lines. So it is necessary to ensure that the measure reflects quantity and quality/ complexity.

Another ambigous measure is "number of tickets per day" which is typically measured for the service industry. Just the number will not be sufficient so we will also need to know complexity like "simple", "medium", "complex", "highly complex" etc... Each of these categories should also have a very clear definition. For example, a simple ticket will not include modification of any code or will not exceed 5 hours, etc...

The definition, data sanity and measurement unit are the three main challenge areas that need to be addressed before beginning any productivity measurement program.