May 30, 2008
-
testing
I guarantee you this post will make absolutely no sense to anyone but me. But what else is new?
The test is inadequate and irrational because it perpetuates a preference for data structures that are impoverished. The classical test orders entities hierarchically and usually linearly and through that linear ordering allows individuals to make comparisons and establish preference orderings. A linear data structure only allows for one comparator and most testing linear data structures usually use either trivial comparators, that is those that create equi-distant partitions between metric points, or vague undefined and often inexplicable comparators. Ie a tester might say entity A scores higher than entity B and not be able to really explain what exactly makes entity A’s score higher. Worse, on occasion tests are so poorly defined that the entity relationships result in breaking rules of transitivity, associativity, or commutativity. ie you might have a person institute a test that results in entity A being superior to entity B and entity B being superior to entity C but then entity C being superior to entity A!
There is nothing fundamentally wrong with that of course, it’s just that such a design often invalidates the use for which the test ideology was investigated in the first place, namely to help the test implementer to make *decisions* or create favorable outcomes. The problem is, decision making tries to be too concrete. It’s an over simplification of the phenomenon at hand. Rather, we need to model the data with more complex data structures that package entity characteristics together in independent functional units that interrelate through well defined messages and interfaces. These advances data structures can then be compared holistically and valued for their intrinsic qualities without the need to establish absolute orderings between then. Indeed any absolute ordering would become virtually meaningless because the entities themselves would have so much depth to them that stating the ordering would just be in all ways less useful toward whatever end you are trying to achieve than digging into the interface of the entities you are trying to examine individually.
Ordering is possible with complex data structures if you really desire it, though it rarely has utility. However the orders you get can be managed along multiple comparator vectors all at once creating a web of relationship between variable characteristics within the entities being compared. A richer picture arises of the nature of the relationship at hand enabling users to develop a deeper understanding of all related entities.
So why then do we insist on privileging these simplistic data structures so much? Every institution struggles to find the best “tests” to pass judgment on the entities in their purvue and individuals devise tests to establish preferences in their daily living. It’s a means of oversimplifying reality to justify predetermined judgments often based on irrational constraints.
But biggest problem tests have is not their simplicity or our over-obsessive devotion to them. It’s that tests ahve a problem of introducing unpredictable uncertainties into systems. Since tests are not independent, detached, observations they do not keep the system pristine. Rather they are integrated within the system, the entities “take” the tests either knowingly or unknowingly but in any case their behavioral trends may shift as a consquence of the “test” being introduced into the system. They might act differently than they would have had their been no test in the first place! And we all know that by adding stress to any system we can inadvertently change the destiny of the system occupants. And usually, almost always, that shift is for the worse (though sometimes you do get lucky).
So overall…
Just don’t test people.It’s stupid.
