Reply #74: Spec test is just the beginning not the end..

[ddeclue]

No you DON'T hand the source code to the actual people performing the spec test but people in the same testing company also ought to be looking at it for logic errors, etc. which often dont' show even when testing the boundary conditions.

You really have to do BOTH. Test to the spec is just a START. Code analysis is part two. Beta testing is part three.

In my world of high speed data-acquisition and test at least, testing is very much timing dependent and defining testing boundaries is often almost impossible unless timing issues are carefully thought through and this requires analyzing the code design for multithreading and timing dependent issues.

I ALREADY said you can't test all the combinations because of the factorial nature of the problem but even testing boundary conditions is often just a spot check. Things can also happen in the middle of the envelope as well as on its edges because we make incorrect assumptions about where the edges actually are in the absence of the source.

As for code quality, I have worked on military projects in the recent past and found amazingly poor, non-object oriented code with UI's intermixed with code, etc. so I don't really expect to find nice clean interfaces in other people's code that I can just test around the edges.