When an organization executes a certain number of projects over a period of time, it needs to calculate certain summaries in order to evaluate the performance of the company.
Some of the metrics that must be calculated are the variation in net effort or the variation in total effort with respect to planned effort. During the planning phase of the project, a project manager estimates the effort required to complete a task. A task in a software engineering company can be an analysis task or a programming task. So during the planning phase of the project, the planner indicates that a specific scheduling task would take a certain number of hours to complete.
When the project is executed, the actual effort (say) is measured and recorded against the planned activity, there may be a variance or a difference between the two values. The same goes for the project schedule. On a related note, the project schedule should be derived from effort and not independently from effort by using separate models for effort and schedule, as schedule is statistically correlated with effort.
When planning the schedule and then measuring the actual, there may be a difference or a variance. During some review period, an organization publishes organizational baselines with summary information for effort, schedule variance, and also for the number of defects occurred, the productivity index, etc.
Care must be taken to calculate (say) the net variance of effort; for example, you do not have to add all the variances in the project to obtain the cumulative variance. To explain why this should not be done, it is possible that many of these projects were running concurrently and many of them may have a common cause of variation, for example if there was a server crash on a particular date, time Downtime can affect many projects evenly and can extend the time it takes to complete a task. Adding all of these variations without doing a causal analysis will lead to reporting a higher number. What can be done is to mathematically divide the effort / schedule variance among all the projects that are affected by it.
An analysis of variance should also be performed and false positives or false negatives checked by hypothesis testing. Stratified sampling should also be used to analyze the net variance. For example, if a group of projects with lower developer skill is dominating the measurements, corresponding scale factors should be applied to each measurement obtained from individual projects so that one sample group alone does not dominate the others.
In summary, the variance obtained after comparing the real in the project with the plan should be subject to standard ANOVA tests. Also, the true value of the variance should be filtered for repeated measurements of the same deviation caused by multiple projects.