On time on target
A creative approach to process management could cure many IT project delivery problems. E&T explains.
Despite how much governments around the world invest in IT to improve effectiveness and efficiency, public confidence in their ability to deliver large (£100m upwards) and complex IT projects remains poor.
Public sector's recent project history is a litany of budget-busting. For example, UK Government's Department for Work and Pensions Central Payment System's initial budget was £90m, with an estimated final cost of £153m when it was cancelled. The criminal justice system had an initial budget of £234m and estimated final costs of £500m when full roll-out was cancelled.
The Child Support Agency system had an initial budget of £427m, and final cost of £526m. And the NATS Swanwick system had a £337m budget with a near £150m overrun.
For IT projects, an industry study by the Standish Group found that average cost overrun was 43 per cent. It also found that 71 per cent of projects were over budget, over time, and under scope (CHAOS Report, 10th Edition, January 2004).
Figures and rigours
The accuracy of estimates and hence the level of potential overrun depends on the stage at which the project is at when budgets are defined.
Often-used error figures for final cost estimates done at the following stages are: at the project inception, around 50 per cent; at the post requirements definition phase, around 30 per cent; at the post design phase, around 15 per cent.
One solution is to use off-the-shelf products, where, for example, £50m will buy an integrated CRM or ERP solution for a telecommunications operator.
However, the applications for government still require bespoke development as off-the-shelf products do not exist for most of the core requirements.
The UK has, consequently, a government with multi-year, bespoke software development programmes with budgets of £100m to near £1,000m. The length of time before benefits are reaped is longer than the sub-one year expectation in the commercial world.
In the early phases of a project, the uncertainty comes from either unfixed functionality or an uncertain solution.
Provided the user is not demanding continual ongoing change, defining and fixing functionality is not an issue.
In the mainframe era we could write a million lines of code for a commercial application - all in Cobol - and the key challenge is to produce the code as error-free as possible in budget and time.
The last 20 years has brought considerable architectural complexity, from client/server solutions to Service Oriented Architectures (SOAs). Reviews of many projects over the last 15 years indicates that implementation teams leave parts of the solution unconfirmed deep into the programme.
The uncertainty in the development phases of projects requires teams to create solutions. Creativity during the development of the solution is the underlying driver of estimate uncertainty; remove the creativity, remove the uncertainty, and be left with accept--able (estimating) variances.
A team using object-orientated design techniques developed a solution whereby the object model covered a wall when printed out. Yet the object request broker could not handle the number of objects, so the system had to be reverse engineered into five sub-systems.
On a different project the system ran so slow it was unusable. A programmer had creatively used a language feature that enabled instructions to be created as strings, and then executed. After much effort, it was found that calls were being dynamically generated that were 'joins' across databases; huge volumes of data were being moved to execute the function.
Perhaps we should reapply lessons learnt in the past. It is not just about better cost control and project management, but trying to understand the fundamentals of 'why' - and then managing out the 'why'.
Back in 1968, in response to overrunning projects, a Government Steering Group reported on Development Cost Estimating. The central concept was the identification of technical uncertainty and exploration of these 'grey areas' with experimental work to find the solutions - not an approach frequently seen in IT.
Creativity is required to develop solutions to these complex requirements, but in the right place and at the right time. It is important to differentiate between creativity and skill. The development phases of a large project can be packed with skill.
However, the creativity required to generate a solution belongs within the architecture team, just as it would in an architect-led building development.
The teams that erect the building have the skills; but would you really want the bricklayer to create new cement mixtures?
New technologies and methodologies introduce the potential for creativity into the development of IT solutions.
The development team must already have the necessary skills for using the new techniques before taking on a major development. You do not want them learning on your project.
Compared to other engineering professions, IT is young. We were accustomed to accepting creativity in development; indeed we still accept, without challenge, such methods.
However, large parts of a development are straightforward, and, with the right tools and skills, can be estimated and developed within budget.
Thought equals time
The track record of the system integrator business has many such examples: well-defined specifications, well-developed architecture implemented using established techniques.
An oil-blending plant management and control system for Shell comes to mind. Shell created a shortlist of three system integrators, and paid for each vendor to provide an experienced person to a Shell-managed functional specification team.
At the end of three months the team was disbanded, and the completed function specification issued to each vendor. IT services company Sema Group won the bid, and 14 months later, on time, on budget and with full functionality, the system went in, and Sema was awarded a discretional bonus by Shell.
All the architectural and system design was carried out in a layered approach with the coding teams writing against one page 'mini specs' and ladder diagrams.
Creativity in development causes overrun because it requires thinking time - and you cannot estimate thinking time. Therefore, a project littered with thinking time has matching uncertainty in the estimates for these activities.
Typically used in larger complex projects is the Waterfall technique. This demands confident estimates early in the lifecycle of the project. As we have identified, the estimation accuracy and confidence drops dramatically in the elements that contain creativity.
We, therefore, require ways of managing out creativity in the development stages of a project.
In complex projects, creativity will be required in some parts of the development phase unless we plan otherwise. The aim is to use appropriate tools and techniques on those uncertain elements earlier - i.e., move the uncertain elements forward so that solutions can be found and estimates generated with confidence before the main development phase.
The explicit steps to take towards a creativity-free main development can be clearly delineated. The first is to review the development elements and identify those that use creative processes.
These can be declared as 'grey areas' that require clarification as soon as possible in the project plan - i.e., we manage the development by 'clarity'.
For example, a common error is to neglect system design until the first build fails to perform fast enough and then to look for creative solutions to save the day; the death knell of many a runaway project.
System Design teams are common in the defence world, but are rarely a clearly-defined function in the IT project arena.
Once the grey areas have been identified, the project plan must then include the resolution of them with appropriate techniques as early as possible.
These techniques include: extreme programming; rapid application development; and prototyping (with potential throwaway).
The ideas behind the extreme rapid application development techniques, such as 'Scrum', are specifically designed to attack and solve complexity and demand that a slice of functionality is delivered in the first 'sprint', thus eliminating grey areas early in the project. Finding problems late in a project multiplies the cost of fixing them.
Using such techniques would disenfranchise many stakeholders in government - and require a significant change of thinking.
The fundamental clash of philosophy can be understood by imagining a bid defence meeting where a vendor begins his presentation with "we are going to deliver 80 per cent of your business benefit with 20 per cent of the functionality specified in the RFP", and "we will sort out which 20 per cent of the functionality to provide on a progressive basis once we have the contract".
The approach would fail to meet the basic criteria for bid success in government procurement.
However, a close fit to the 80/20 rule emerged during a review of an e-forms solution that had automated 38 business processes and showed that one process, Payment Request, accounted for 45 per cent of all usage. The top ten most-used processes accounted for 90 per cent of all usage.
The issue of creativity, as opposed to skill, must be tackled head-on to drive confidence in the estimating of large government projects - and to kill the overrun syndrome.