When all the coding for an app is done the last things I do before testing are to tidy up the artwork and write the manual. Writing the manual actually forms the basis for a lot of the testing as walking through all of the app, taking screen shots and explaining the sequence of work throws up quite a few inconsistencies and bits of poor labelling which need to be fixed. This happens as I try to pre-empt the user asking why or how something is done at any particular juncture.
Despite this, later testing occasionally turns up what should be fairly obvious faults that familiarity has left me blind to. (These are precisely the things that always surface as soon as you show the work to someone else.) You can only get so far guessing where the problems will lie for others.
I remember reading some 20 years ago about a “cardboard assistant” which you would sit in a chair next to you and explain a problem you were trying to solve. The process of collecting your thoughts and presenting them in a coherent manner would often provoke a spontaneous insight into the solution, or so it was claimed. This is very similar to Eliza, an early natural language processing program which either ridiculously parodied or acutely mimicked a visit to a psychiatrist depending on your point of view. Eliza gives an impression of sentience by using trigger words in your responses to ask related but seemingly insightful questions, leading you ever on. Both of these tools, if they are to have any chance of success, require you to have correctly framed the correct problem. (See earlier post on Russell Ackoff to see that we are not intuitively very good at this).
This issue is described by Edmund Crispin in his detective story The Glimpses of the Moon as “The Chesterton Effect”. In this story an unidentified man is found dead, minus an arm, in a tent during a village fete. As the tent also contains what its elderly spinster owners imagine to be a very valuable painting, the comings and goings of people are completely observed. The only person to leave who could possibly have been carrying the arm is the Vicar who had his cricket bag with him. It is quickly established that he was not carrying the arm. The greater part of the book is spent, diversions aside, in establishing how the man’s arm was removed from the tent. The solution when found, is described as only being possible as the problem as set represents the Chesterton Effect, viz:-
1. The wrong question has been asked, and
2. The answer to the right question is a paradox.
In Crispin’s book, instead of asking how the arm was removed, the question should have been why? The paradoxical answer is “because it wasn’t there”. Upon establishing that the victim was a recent amputee, his identity and relationship to a suspect is established and the case is quickly solved.
First then, I need to find a better way to establish some correct questions to be asking when testing. Wisdom in this sort of area I often find in the work of Stafford Beer and his “Viable Systems Model” so I’ll have to have a dig through some boxes. Then I’ll give up and buy another copy of “Platform for Change” (I must already have two or three). Then I’ll find one of the others propping up a dodgy table leg or similar. Ho-Hum.