The recent launch of website Wheedle offers a useful case study of the importance of acceptance testing in IT development contracts.
Wheedle – a potential competitor to online auction site TradeMe – was launched on a Monday in early October, only to be taken down by Tuesday lunchtime amid reports that users could reset other users' auction reserve prices and questions around the security of users' passwords.
The developers and owners of Wheedle would no doubt have been confident that the platform worked as it should have prior to go-live, but it seems that whatever testing was carried out did not identify some of the holes and weaknesses that an eager public (including no doubt some less than benevolent users looking for flaws) were able to pick at within a matter of hours of launch.
While most software or platform launches may not be as high profile or public-facing as Wheedle, the same lesson applies to any IT development project – before flicking the switch, the customer should have had the chance to make sure everything works as it should.
This involves more than just kicking the tyres. Robust IT development contracts should set out detailed acceptance testing processes and criteria. Ideally, these will require the parties to work together to identify the important things that need to be satisfied during the testing phase and allow the customer ample time to carry out the testing without the pressures of a short deadline. Testing in an offline test environment will help ensure that all the features and functionality are present, and allow a safe and controlled environment for penetration testing to make sure that users are only seeing the content they're meant to and that security measures are up to scratch.
If appropriate, a soft launch with a closed user group can help to replicate real-world scenarios and enlist the help of users to identify and report issues for resolution, before the product or site opens to the public.
If the product or site does not pass acceptance testing, then the contract should require the supplier to remedy any shortcomings within a defined period of time. The parties may also look to agree additional incentives in the form of liquidated damages, fee rebates or service credits if the remedial work causes project delays; though at the end of the day, the customer may prefer for it to be done right, rather than done quickly.
However, as Wheedle may have discovered, even if a platform has passed robust acceptance testing with flying colours, there may still be undiscovered issues which do not surface until after go live. In those circumstances, customers should consider what remedies the contract offers if defects are discovered further down the line (such as the ability to hold back a portion of fees, obligations on the supplier to remedy or a right to demand service credits or bring warranty claims).