Discovering requirements: from wish list to want list
IT projects falter when managers do not properly reconcile contending requirements and expectations. Discovering requirements is a discipline that will help resolve potential pitfalls at the outset, explains E&T.
It was said in the 20th century that a lot of things had been invented in the past hundred years, and most of them plugged into a wall. Even more things have been invented in the past few decades, and many of them plug into the wall only very occasionally - but almost all of them contain software.
Enterprise users expect to be able to consult secure databases from anywhere in the world, while they are on the move, even on holiday. Many people expect to be able to telephone, email, text, navigate, and listen to their home radio station wherever they are. It really is an information revolution; but the fact that software forms a large and growing percentage of every enabling product is often ignored.
Projects are becoming riskier, too. Few projects have an empty 'green field' to work in. Usually, they are tightly constrained by the need for compatibility with the legacy systems that they are replacing. Even apparently stand-alone embedded software products such as MP3 players and GPS receivers are constrained by an increasing range of interfaces, such as USB connectors, Bluetooth, and Wi-Fi, plus the MP3 and GPS standards themselves.
Despite all these constraints, projects have to succeed, and do so quickly. Roll-out problems are apt to cause extra expense and embarrassment; a sure-fire way of getting roll-out/integration problems is to go ahead and develop without an adequate grasp of the real requirements.
As requirements become more complex, conflicts grow more likely: for instance, mobility and security often pull in opposing directions. Trade-offs between such conflicting factors often govern system design.
The systematic discipline of Discovering Requirements is increasingly recognised as a key element in project management. The IT function, familiar with specifying products and services, can help to guide the rest of the organisation towards good requirements discovery practice. Here are some of the basic elements.
A curious software development dogma asserts that requirements must be complete before design begins, just as design must be complete before testing begins. In fact, there is little basis for these beliefs: useful testing can begin almost from day one, with requirements validated using paper prototypes. The trade-off approach to requirements discovery similarly turns dogma on its head: you do not know what your requirements really are until you have matched what people want against what you can possibly provide.
The trade-off cycle begins with people, the stakeholders in your project. They each state their goals for specific functions; for operability; for performance; for safety, security, reliability; for compliance with standards; for low-cost, low-weight (if it is a portable device), low power consumption, and so on.
The designers can then get to work, identifying candidate solutions - software architectures; specific algorithms; alternative platforms; off-the-shelf components. Very likely, more than one approach is imaginable. If so, the approaches will differ in how well they meet the different goals. For example, a handheld device can do most of its processing locally, or access a remote service. If you go local, you have limited processing and battery power, but results can be delivered quickly and reliably - until the battery is exhausted - with no communications overhead. If you go remote, you can have all the server horsepower you want, but you may have to wait for results to be delivered over a slow network: and if the connection fails, you get no service at all.
These considerations are important trade-offs. The right choice depends strongly on the context. If the purpose is safety-related, for instance, reliability is extremely important. On the other hand, if a service is for entertainment, then performance may outweigh reliability.
Once a project has identified its stakeholder goals and candidate solutions, trade-offs must be evaluated. Mathematically, goals like performance, operability, reliability and so on represent different dimensions. Operations per second to mean time between failures, for instance, cannot be added: they are independent measures of quality. Where there are just two such factors to consider, each candidate solution can be plotted on an X-Y chart: perhaps the ideal solution lies at the top right of the chart, with say both high reliability and high performance.
But what if there are 10 or more dimensions? The multi-dimensional space is hard to visualise. There are many possible approaches. Some of these have become popular in certain industries.
A simple first step is just to make a table or matrix of your candidate solutions against a project's goals, used as evaluation criteria. Each cell contains a score (maybe a number, maybe just Yes or No, maybe a value between 333 and 777) showing how well a given solution meets a given goal.
If scoring is tricky, you can ask each of several independent experts to evaluate each solution in their area - your security specialist evaluates each solution's security, and so on. If you are worried about subjectivity, you can average scores provided by several security specialists, but obviously this increases your costs.
Now you have a more-or-less objective, more-or-less detailed evaluation of all the options on all the goals. If you are lucky, an obvious winner will emerge - if Option A scores 333 on nearly everything, while no other option scores more than 3, then no clever analysis is needed; but usually, you need to do more to reveal the patterns.
A simple trick is to plot a histogram for each candidate solution against the goals. You get a set of little graphs; if you display these in a column, one above the other so the histogram bars for factor 1 are on the left, and so on, you may be able to pick out the winner visually. If that's not enough, you need a means of reducing the number of dimensions to a manageably small number (no more than two or three) so you can identify the winner easily (see 'Criteria Weighting Justification') .
Analytic Hierarchy Process
One approach that is certainly more defensible than naïve weighting is the analytic hierarchy process (AHP), or Sophisticated Weighting. This essentially asks the user to compare each factor (criterion) with each other factor, pairwise, indicating which is more important. AHP then calculates weights, reducing the risk of subjective bias. The scores are combined into a single league-table style result.
An advantageous quality of AHP is that it can detect inconsistencies: if you say safety is much more important than performance, and performance is more important than cost, but cost is as important as safety, then AHP points out the problem, and can measure the size of the disparity; it remains a weighting approach - apples are still being added to oranges, however subtly and indirectly.
Principal Component Analysis
A completely different approach is PCA (principal component analysis), or human decision-making. This works by finding the line (an eigenvector) in the multi-dimensional space that explains as much of the difference between the candidate solutions as possible. This line is a new dimension, the first component of a flattened-down space.
The next component is the line that explains as much as possible of the rest of the difference between the candidates: it is by definition at right angles to the first component. If PCA goes well, the first two or three components explain perhaps 75 per cent of the total variance.
You can then safely choose between candidate solutions on the basis of these two or three readily-graphed new dimensions. PCA does not work every time: there may not be much difference between the candidates. But when PCA works, it gives a clear picture of the differences. Reassuringly, the components are mathematically guaranteed to spread out the candidates as widely as possible. It then remains a human responsibility to interpret the data.
Goal into requirements
Trade-offs both identify the best solution approach, and determine which project goals are feasible requirements given the chosen solution approach. Some goals may be weakened or sacrificed entirely. Trading-off is a crucial step in requirements discovery. It does not depend on exhaustive documentation 'up front' and in advance of design; rather, it consists of a dialogue between stakeholders' goals and possible designs, ending with a well-considered choice of the best approach.
Effective IT leaders invariably already put into practice many aspects of requirements discovery and trade-offs. Relating problems to solutions is a vital element of software and systems engineering. Trade-offs are here to stay.
Ian Alexander is co-author with Ljerka Beus-Dukic of the book 'Discovering Requirements, How to Specify Products and Services' (Wiley, 2009).
Criteria weighting justification
Many projects try adding up the scores on the different criteria. For example, suppose your project awards scores for each candidate solution from 1 (worst) to 7 (best) as shown above.
That gives Option A a good-looking average of 5.0, despite a worst-possible showing on Safety. Clearly this approach is unsatisfactory, so projects try to remedy the situation by assigning weights to each goal. Suppose Safety is the most important goal, so it gets a weight of 0.7; all the others get 0.1. That pushes the score for Option A down to a less impressive 2.6 (out of 7). But if a score of 1 on Safety means 777('Very Unsatisfactory'), then it is not right to average this with other scores, weighted or not. You could try to patch up the approach with the rule "reject any option that scores 1 on Safety", but it is hard to avoid the conclusion that weighting is unjustifiable.
Naïve weighting is open to abuses, including the temptation to adjust the weights until the 'right' answer is obtained. This distorts the process. In the worst case, it is equivalent to ignoring the stakeholders' goals. The success of the project then depends not on the supposedly mathematical trade-off, but on whether peoples' gut instincts are correct. The likely outcome is a product that fails to perform as the stakeholders wanted, and may be entirely unfit for purpose. Trade-offs matter, and must be done properly.
|To start a discussion topic about this article, please log in or register.|
"Is augmented reality the next big thing or a marketing gimmick? Is it fundamental to the future or a fashion faux pas?"
- Sellafield MOX Plant Lessons Learned [10:02 pm 21/05/13]
- Philips 8051 assembler [09:53 pm 21/05/13]
- Isolation for repair of transformer feeder [08:46 pm 21/05/13]
- The Energy Bill and What is an Investment Contract [07:30 pm 21/05/13]
- "Contracts for Difference" in the Explanatory Notes to the Energy Bill [06:58 pm 21/05/13]
Tune into our latest podcast