|
Testing constitutes no less than 40% of any y2k project. Some estimates place it as high as 75%.
Few organizations have developed a testing plan.
The conclusion is obvious: y2k cannot be fixed.
No one wants to say this. It's politically incorrect. That's because it's true.
This is from CIO MAGAZINE (March 1), the magaxzine for Chief Information Officers -- the people who head up corporate computer systems.
* * * * * * * *
Fixing your year 2000 problems before the New Year's Eve deadline isn't enough--you have to test your applications and make sure they'll really work. . .
If You Test, You're in the Minority
For most companies, unfortunately, testing is at best a rusty skill set. The Software Engineering Institute at Carnegie Mellon University in Pittsburgh estimates that 67 percent of organizations have no formal Y2K testing program in place. Even the praiseworthy 33 percent are finding Y2K testing to be knottier and more problematic than anything they've done before. "Dates are just so all-encompassing that when you start messing with them, every aspect of the system-and all the interrelationships between systems-needs to be tested," says Albert Kern, an assistant vice president and the Y2K project leader at Boston's Commercial Union Insurance.
To add to the burden of making sure all systems are functional, there's the necessity of documenting every last step of the process. "You can't just run these tests," says Greg Pope, president of Azor Inc., a test-tool supplier in Palo Alto, Calif. "You've got to have evidence that you've run the tests and gotten the correct results because if you face litigation [later], you need to be able to prove that you did the testing and were in good shape."
With Y2K projects, testing dovetails with documentation. The two are longtime bugaboos of the IT industry. "What's the first thing that you skimp on when you're running up against a tight time frame and a tight budget?" asks Brian Keane, co-president of systems integrator Keane Inc. in Boston. "Testing, with documentation a close second."
Yet testing is expected to represent half a Y2K project's effort and expense, according to the Gartner Group Inc. Some project leaders, like Nabisco CIO Joseph Farrelly, see the test phase growing to 60 or even 75 percent of the total work. He points out that testing involves third-party software, embedded systems, PCs, custom-written software and, of course, mainframe operating systems and hardware. Even brand-new packages that purport to be Y2K-ready need to be poked and prodded for faults.
All that can be dubbed internal testing. Its various stages include unit testing (looking at individual program components), system testing (ensuring that the units work together) and integration testing (verifying that all the systems interact correctly). An entirely different facet is external testing-making sure the organization's remediated systems will work with those of its business partners, regulators and customers. . . .
Start Early and Test in Tandem
Testing can't be the last task on a Y2K project time line. "It starts in the beginning, not somewhere in the middle or at the end," says Les Fondy, senior manager for IS process solutions at DHL, the Redwood Shores, Calif.-based U.S. division of DHL Worldwide, a $4 billion air-freight company.
"It takes a long time to set up the hardware and software environment for testing," adds Joseph Allegra, president of Princeton Softech, a software development company in Princeton, N.J., that makes a product that "ages" data for Y2K testing purposes. "It also takes a long time to develop your testing plan."
Testing should begin during the initial assessment phase, when systems are evaluated for their ability to function in a post-2000 world. Systems that are dubbed compliant need to be thoroughly put through their paces. "We've never found a system that was 100 percent Y2K compliant," says William Ulrich, president of Tactical Strategy Group Inc. in Soquel, Calif. . . .
Once conversion begins, other testing tasks can proceed in parallel: the design of test cases, the development of an appropriate Y2K testing environment and the assembly of a testing team. And as soon as code is converted, it should be run through the testing process. "Early tests help validate the remediation process you're using," says DuWayne Peterson, president of IT consultancy DuWayne Peterson Associates in Pasadena, Calif., and the former CIO at Merrill Lynch & Co. Inc. "It may be that there's a flaw, and you'd like to know that sooner rather than later." . . .
Unfortunately, there's no consensus on who should actually perform the testing. . . .
Regardless of who does the testing, everyone agrees that one part of a test team is indispensable: the application experts. "The application experts need to be the intellectual leaders in testing," says Phil Carrai, president and COO of McCabe & Associates, a Columbia, Md., company that makes assessment, visualization and testing software for the year 2000. "You need their expertise to develop test cases." . . .
The experts' services will be in high demand as the test phase gears up, since there's a finite (and, by the way, insufficient) number of people who thoroughly understand a given application. Chase Manhattan, for one, has run into bottlenecks as a result. "This isn't something you can throw money at," says Robbins. "These are people who have to know the business and know what the results should be." . . .
How Do You Know When You're Done?
The IS executives we've cited all aim to have testing completed by late 1998 or early 1999. How will they be sure they're done? "The answer is, when you have enough documentation to tell your board, your audit committee and your trading partners that you have tested to the best of your abilities," says McCabe & Associates' Carrai. Azor's Pope advises that that milestone is rarely reached after just one or two test cycles. "People think they're just going to run one test and it'll work," he says. "They don't realize that fixes always cause new problems." . . .
Whether testing wedges its way into every organization's culture or not, there's no denying that it has achieved something of a celebrity status as a result of Y2K. The stakes are high, and testing is crucial to a successful conversion effort. And some of the biggest testing hurdles--for most companies--still remain to be discovered. "We need to get a dialogue going around testing," says Judith List at Morristown, N.J.-based Bellcore, which consults with telephone companies all over the world. "We're only beginning to scratch the surface to see what some of these challenges are going to be."
|