Institute of Internet Quality

Applying ISO9000 to Software Development

The ISO9000 series of standards were originally developed for manufacturing, and specifically for manufacturing in the aerospace industry. However, they have been modified (the third major release is due in the next year or two) to make them more applicable to other enterprises, including software development.

That's not to say that they are ideal for application to software development

because of their history they're not truly ideal for any application outside aerospace manufacturing.
But the ISO9000 standards do contain a lot of very useful practices and disciplines, which can be applied to any organisation.

I’ll briefly explain the way in which the ISO9000 standards work in practice, and then give a background on how they relate to other software quality initiatives. Then I’d like to talk about the implementation of ISO9000 in a software development environment. By the way, during this discussion I’ll use the term ‘ISO9000’ to refer to both the ISO9001 and ISO9002 standards.

The usual basis for the use of ISO9000 standards is a concept called ‘external audit’, in which an external organisation comes into your premises (at your invitation) and checks whether you are complying with the requirements of the standard. If you are, they will issue a certification which you can then display and use as part of your marketing.

Some purchasers (particularly government purchasers) also make certification a prerequisite, or at least an advantage, in responding to tenders. In this respect, ISO9000 is similar to other initiatives such as CMM and MRP2 - all of these are initiatives which rely on 'third-party certification'.

The history of this form of certification goes back to the days of major government purchasers sending their own auditors into supplier organisations. In this way, not only could the purchaser ensure compliance, but they could also vary the level of compliance required depending on how critical the particular goods or services were to their own enterprise. The problem, though, was that if a supplier dealt with more than one purchaser, they would have to submit to an external audit from each purchaser. This was costly for both the supplier (who had to field multiple audits from multiple purchasers) and for the purchaser (who had to audit each of their suppliers).

So a system of 'third-party auditors' were set up - in most countries, based around the national standards bodies themselves, plus other large consulting companies - who could provide a single audit which would satisfy (in theory at least) all of the purchasers from a certified supplier. There are a number of problems with this, though, and I'll relate them from an Australian perspective. I have to add that I do not have direct experience of the Singapore certification bodies or quality management consultants, and my remarks do not apply to them.

One problem that is faced in Australia is the way in which third-party audit bodies are funded. The more clients they get, the more money they get. There is a perception that in order to win business, audit organisations (and there are more than a dozen of them now in Australia) might be less rigorous than purchasers themselves in applying audits.

I`n my opinion, this has caused a major dilution of the value of certification (and of the perceived value of ISO9000 itself). This is particularly true of the auditing of software development organisations, where there is a serious lack of skills and supervision due to the specialised nature of the industry.

Another problem is the practice of allowing 'onion-skin' certifications, where in a large organisation, the department that deals with government can be certified, and meet the demands of its customer that way, while the majority of the goods and services that are supplied are actually from the 'inside' of the onion - the uncertified bulk of the supplier organisation. A similar problem applies to importer organisations.

Microsoft is not certified to ISO9000, and according to Bill Gates, never will be.

The volunteer labour that develops Linux is unlikely to get certification either. So if a government purchaser wants to buy from a certified supplier, but wants to buy a PC operating system, they have little choice but to settle for a certified supplier of goods that were developed by an uncertified source.

But an even larger problem is the fact that the manufacturing industry in Australia was the first to adopt ISO9000 - it fits this industry best, of course, because of the background of the standard itself. This means that most of the consultants, and most of the quality managers, who were available in Australia a few years ago had experience in manufacturing, and in the forms of control suitable to manufacturing organisations.

A typical life history was for a manufacturing organisation to decide to get ISO9000 certification - they might employ an outside consultant to help them. That consultant’s first task would be the identification of an individual within the manufacturing organisation to become the quality manager for that organisation. The company achieves certification, and the brand-new quality manager finds that their skills are in demand elsewhere - a larger manufacturing organisation wants certification as well, and will pay handsomely for an experienced quality manager. The quality manager changes jobs, a year or so goes by and they decide that they can make more money as a quality consultant, so off they go and rent a serviced office.

So now you have a population of ex-quality managers working as consultants.Their experience has been as quality manager of two manufacturing organisations. Their view of how to implement a quality management system has three sources:

ISO9000 itself (which is a little like a holy text - subject to much interpretation)
the quality management consultant that put in the system at the first manufacturing organisation
the cultures of the first and second manufacturing organisations

Even if the first-generation quality management consultant had a wide experience in multiple industries, the second-generation consultant has a narrow experience in one industry. If you employ one of these second-generation consultants to implement a quality management system in a software company, you get a manufacturing quality management system implemented in a software company. I have seen this myself many many times.

All of these problems, plus a few more, have contributed to the fall from grace of ISO9000 as a certification standard for software in Australia. Unfortunately, newer initiatives such as SPICE are also in danger of being ‘tarred with the same brush’. Having said that, there is still, I believe, an enormous amount of value in ISO9000. Despite its manufacturing background, and despite the problems with certification.

My organisation, for example, which is a consultancy, and about as far removed from aircraft manufacture as you could get, maintains its ISO9001 certification despite the fact that purchasers never ask for it. Why? I hope to explain why during the rest of this talk.

ISO9000 covers a number of organisational issues, but all are related back to the concept of ‘quality’ - a slippery concept at the best of times, but defined in ISO9000 circles as ‘meeting customer requirements’.

If it were just a question of agreeing to meet customer requirements, then ISO9000 would be very simple.

But in order to meet customer demands, you must record and agree them with the customer.

One of the most important requirements of ISO9000 surrounds the area of ‘contract review’.

Now in an aerospace context, customer requirements come to the organisation in the form of a contract, or more usually a tender which results in a contract.

The specifications for the work are set out in detail in the contract, and so a close review of these specifications can reveal whether:

the specifications are complete an unambiguous
it is possible for the supplier to meet the specifications

Unfortunately, when you move outside manufacturing, you have to be a little imaginative in implementing this requirement.

Contract review occurs at the point at which the customer and the supplier agree what is to be supplied.

Where does contract review occur, say, in an organisation that accepts equipment for repair?

One of our clients had this situation: they repair cameras.

A customer will bring the camera in and hand it over a counter.

At that point, it’s not clear whether the camera can be repaired or not - it may be beyond repair.

Applying contract review in this situation means looking beyond the wording of the standard, and using first principles.

The answer in this instance was to have a clearly-worded receipt for the customer, that said that repair might not be possible, and that the cost of repair would be agreed with them by phone before the repair was carried out.Applying contract review in a software context needs just as much imagination.

A typical manufacturing-based quality manager will point to the tender for a software development project, and say that a review of the tender document constitutes contract review.

Most third-party audit companies would agree with that.

But software professionals know that the most difficult part of a development project is requirements elicitation.

In a spiral development the requirements are being developed right to the end.

In this context ‘meeting customer requirements’ does not end with the tender document.

Having decided on the requirements, ISO9000 expects you to begin design.

There has been a vigorous debate in the software standards community about whether software development should be regarded as design, or whether - since software development is all a software development company does - it should be considered as manufacture.

In my opinion, this debate was caused simply by the miss-match between the standard and the industry.

If we were building aeroplanes instead of software, there would be no debate.

In order to do manufacture (or design) repeatably, you have to have a set of procedures and standards.

Now let me first describe what a bad set of policies and procedures looks like, and see if anyone finds them familiar.

When the quality manager, or external consultant, is given their task, it was stated like this: “Get XYZ organisation certified to ISO9000 standards”. It wasn’t: “Improve the quality of XYZ goods and services by getting XYZ certified to ISO9000 standards.” So the whole focus of the activity was on the standard itself.

Now the standard is split into 20 elements, so in drawing up a set of policies, the manager makes up a ‘quality manual’ which has one company policy covering each of the elements of the standard.

So far, so good.

Now armed with this, the quality manager decides to write a set of high-level procedures under each of these policies.

This makes a nice neat picture, with the standard at the top, and the policies underneath, and then the procedures:

Now in a manufacturing company, many people don’t work in offices.

They work on the factory floor.

When they come to use a particular lathe, they need instructions on how to turn it on, how to maintain it, and so on. And when they are applying a particular process (which may be unique to a certain customer) they need instructions on how to do that.

So the concept of ‘work instructions’ is born.

These fit right under the procedures, and extend the pyramid again:

This neat picture may work perfectly well in a manufacturing environment.

It should - it was designed for a manufacturing environment..

But there’s nothing in the standard that says that you have to do it this way..

All the standard requires is that you have a set of instructions about how to do each critical task in the organisation, and that they’re available and up to date. Now I’ve seen software development, and also major financial services organisations, saddled with a pyramidal set of documents. .

The tradition in banks and insurance companies is to have ‘policies and procedures’..

But on implementing ISO9000, a typical manufacturing-based quality management consultant will introduce the full three-level pyramid..

Why? Because that’s all they know..

There’s nothing in ISO9000 that says you have to have this structure..

And what you find in practice is that it’s so unworkable that most organisations develop of their own accord a separate set of procedures like this:.

Because the ‘quality’ procedures are designed around the standard, no-one except the quality manager and the external auditor can actually find anything in them.

So in order to actually get the work done, another set is required.

A much better model for a services or software organisation looks like this:

Here the procedures actually map to the processes that take place within the organisation.

They are on-line, and available through an intranet.

Because all workers have a PC on their desk, access is not an issue.

And all the quality manager and external auditor need is an index to the 20 elements of the standard.

In fact, there’s nothing in the standard that says that you have to have procedures in writing.

I’m looking forward to the day when written procedures are replaced by videos, or perhaps audio tapes.

This will be particularly important for workers who have face to face customer contact, such as banking and insurance front desk staff, because they need to actually learn the procedures rather than continually look them up.Is there value in the rigorous approach to documentation and document control called for by the ISO9000 standard?

Yes there is.

What’s missing is an implementation free from the manufacturing history of both the standard and its implementors and auditors.

I’d like to look at one more ISO9000 element in the time I have left, and that’s training.

ISO9000 calls for a regular review of the training needs of staff, and the training requirements of each job.

In a manufacturing context, this is straightforward - if someone wants to drive a forklift, they have to have a license.

But in the software industry things are not so straightforward - you can’t send staff on training courses every time they need to use a new version of a development environment, or a new Visual Basic widget.

In general, things move too fast in this industry - by the time you’ve organised the training, the version after next will be out.

Now the traditional manufacturing-based approach to this problem is to ensure that during each employee’s annual review, the question of training is raised. The manager says something like: “Can you think of any training courses you’d like to go on” and the employee says “no”, or “well I’d like to train in XYZ, but I don’t have time.” Formal training in development tools is just not a common feature of the software industry, because of the speed at which things move.

Again, an application of first principles helps.

What the standard is trying to say is that you need to make sure that people are properly trained in order to do their jobs. In a help desk organisation I saw an implementation of this which complied with the spirit and the letter of the standard, and actually delivered better quality to the customer.

The help desk supervisor had a spreadsheet, which showed each staff member, and the number of hours they had spent playing with each version of each application. Help desk staff are allocated a particular version to play with until they feel that they understand it well enough to answer most basic questions.

When they feel that they are ready, the supervisor enters the fact into the spreadsheet, which is used to allocate staff to particular incoming call queues.

Of course, even if you have good procedures for the management of training (and, by the way, this particular discipline forms a major part of the new term for all of this: knowledge management), they will still need to be improved.

A number of times I’ve heard people say: "we’ll get our procedures right first, then we’ll implement them", or "of course, once the procedures are in place we won’t want to change them", or even worse: "we don’t want to put our procedures online until we have finished them". Any system (and this is particularly true of computer systems) is a work in progress for ever. I don’t know of any software application that doesn’t have bugs. Even if it didn’t have bugs it could still be improved. You won’t find a single software engineer that will claim that a system they know of is absolutely, finally complete and will never be changed.

So why do the same people think that you can get procedures right first time? Or that you can’t put them into practice and then improve them afterwards?

One of the worst things you can do in the implementation of a quality system is to make it hard to change. In chemistry, there’s a concept called ‘activation energy’. This is the amount of energy it takes to get your initial set of chemicals moving fast enough so that they can react to form the products that you want. The higher the activation energy, the fewer molecules will make it over the hump, and the slower your reaction will be. In the same way, the more work you make someone do to implement a change in a procedure, the fewer changes you will get happening.

It’s very simple. Let’s say that in order to change a procedure you have to fill in a form (on paper) and then submit that form to the quality manager, who then allocates a number and puts it to a management committee, who then approves the change (if you are lucky), and sends it back to the quality manager for implementation, who then sends it to the document control officer to be typed into the document. And let’s say that some developer sees a small problem in the document - an extra column would be handy on a form, for example. Do you think that the developer is going to go to the trouble of filling out the change request form just to add another column? Not unless they have nothing better to do.

So the number of improvements to your documents are going to depend on your change control system. That means that over time, your change control system is going to be a direct predictor of the quality and usefulness of your quality system. It works like this:

the more elaborate your change control, the worse your quality

Of course, people don’t like producing low-quality products. The developer who is stymied in trying to add another column to a form is likely to just draw one in. Or leave the information off the form. Over time, people in your organisation get the impression that the quality procedures bear less and less relation to reality. They do not represent "the way we do things around here". They get ignored. They become shelfware.

Is there a solution? Well, in our organisation and a number of our clients, we have simply allowed the document owner to go online and edit their documents any time they like, without reference to any other person. No permissions, no forms to fill in, no delays. If they see something that needs changed, they change it.

Our auditors didn’t have a problem with this, because it complies with both the letter and the spirit of ISO9000:

"documents ... shall be ... approved for adequacy by authorized personnel. ... Changes to documents ... shall be reviewed and approved by the same functions ... that performed .. the original review ..."

And the spirit of ISO9000? The whole document control structure of ISO9000 is intended simply to ensure that the processes within an organisation are controlled. There’s nothing sacred about the procedures themselves - their only purpose is to control the processes. If you can assure an auditor that that’s what’s happening, you have no problem.

Our normal practice when implementing ISO9000 is to implement documents that are incomplete deliberately so that the new document owner feels obliged to repair them as soon as possible, and thereby takes ownership for the document’s content as soon as possible. After all, we know that every single document in every single quality system ever implemented is either incorrect, or could be improved, just as every single piece of software ever written has bugs, or could be improved.

By removing all possible barriers to change in a quality system, you allow the system to move and grow, and keep up with the environment it lives in, and to keep up with the new ideas generated by the people who work with it.

Internal audits are another area where an application of first principles helps. The word ‘audit’ has unhelpful implications in many cultures - particularly when it’s a ‘tax audit’, or a ‘performance audit’. An ‘audit’ by the ‘quality manager’ is very seldom seen as a reason for rejoicing. At the very best, it’s viewed as a waste of time. Going back to ISO9000 first principles, what is the purpose of an internal audit?

There are two:

ind out whether the written procedure matches reality ("to verify whether quality activities ... comply with planned arrangements"), and
ct as a driver for improvement, by alerting management of any problems

There is no reason why this needs to be a PR disaster for the quality system. Instead of ‘internal quality audit’, we prefer to use the term ‘internal process review’, which puts the stress on reviewing the process, rather than the person. Instead of the quality manager, there’s no reason why managers within the organisation can’t carry out peer reviews of other managers’ system.

There are a number of advantages to this:

ets more people involved in the running of the quality system
ives managers an insight into the processes in other areas
managers who do the auditing have direct experience of what it’s like to be audited
person doing the auditing has experience of what it’s like to be a manager

The standard doesn’t mind you doing audits in this way. It specifies that the audit should be carried out by "personnel independent of those having direct responsibility for the activity".

The internal audit training needs to be suitable, too. There’s no point in taking managers out of their area for a day, cramming them with ISO9000 concepts and sending them back into the field to do audits. After all, an audit is a comparison between the procedure and reality: you don’t need to be an ISO9000 expert for that. The quality manager can run continuous desk audits to ensure that the requirements of ISO9000 are actually being met, and leave it to the internal auditors simply to find out whether what’s in the procedure is what’s happening.

So the internal auditor training needs to be in:

operation of the internal audit procedures and processes themselves
niques for collecting direct evidence (that the procedures are being followed)
listening techniques (for eliciting suggested changes)
non-threatening behaviour (to allay the fears of those being audited)

If you implement audits this way, you end up with a system for strengthening your quality system, rather than just frightening people into using it.

If any of the things that I’ve suggested here today seem outlandish, they’re not. We’ve implemented all of them, both in our organisation and at numerous client sites, over a period of years.

Something else, too. All of the things that I’ve talked about are worth implementing whether you’re interested in ISO9000 implementation or not. These are some of the key ideas that lie behind ISO9000, and that make it worthwhile

So to summarise, ISO9000 comes from a manufacturing background, and so do most of the people who implement it. Applying it to services or software organisations in these circumstances is difficult, and often seen (rightly) as an unnecessary burden that does nothing to deliver quality.

But by looking back to the original document itself, and asking the question: “What is it actually trying to achieve?”, you can use ISO9000 to improve both organisational efficiency and effectiveness, and so to deliver better quality solutions to your customers

Source: http://www.hci.com.au/hcisite2/articles/Singapore.htm