Software testing lessons taken from music

By Jennette Mullaney



You may not think so, but software testing and music have similarities. Michael Bolton and Nick Wolf demonstrated this at CAST 2008.



The Association for Software Testing's annual conference, CAST, had a unique theme this year -- Beyond the Boundaries: Interdisciplinary Approaches to Software Testing. Speakers and presenters really took this theme and ran with it. The result was a refreshing and enlightening experience. "Delightful" isn't an adjective normally associated with testing conferences, but it applies to CAST 2008.

Attendees at Michael Bolton's and Nick Wolf's (filling in for an ill Jonathan Kohl) session, "Testing and Music: Parallels in Practice, Skills and Learning" were treated to a wonderful musical performance in addition to learning valuable lessons about testing. Bolton opened the session with a question: Are there things about music that we can use to help us understand testing?

Bolton wanted to get away from using engineering as the sole metaphor for testing. Through his experiences with traditional Irish music, the testing guru discovered many similarities among the styles, composition, performance, and culture of music and those of testing.

Software testing, like music, has varying styles and schools of thought -- context-driven, factory, agile, quality, and test-driven development. Bolton made a point of emphasizing a positive, rather than a religious, approach to these schools.



"That's not my style, but what can I learn from it?" he asked. "If you can identify with somebody else's modes of thought, it might save you from saying 'that's crap' or 'that's irrational.' "

There are parallels in composition and performance as well. "Interpretation, observation, and evaluation of product and of testing itself" applies to music as it does to testing.

In terms of traditional Irish music, a style that Bolton has embraced for several years, he's noticed even more similarities.

"In Irish music there are a lot of geeks, and they're startlingly introverted for performers," he said. When Irish musicians play in front of an audience, it's not a performance as much as collaboration, according to Bolton. "They are participatory but not done for the audience," he said. "The audience in the pub is not an audience to us, and we are rarely conscious of them."

Structures and patterns in music also find their counterparts in software testing. The chromatic and rhythmic patterns of music, and the ways in which musicians interpret theme, correlate to patterns in testing. Testers and musicians use "various heuristics to figure out what happened," said Wolf. "There is a vocabulary, a way of talking," he continued. Much of this is "nonverbal, done via a pile of heuristics -- many we don't understand cognitively."



Bolton and Wolf played a traditional Irish song for the audience and then played a MIDI version of the same song. The MIDI may have been technically perfect, but no one clapped for it. The live performance, however, was greeted with hearty applause. This demonstration, they explained, was meant to show the startling difference between an automated process and a human process.

"In terms of testing...emotions are a strong driver of detecting what we value in a piece of software," said Bolton. "Something that's not functionally correct can still have value for people."

Mistakes have their place in testing as in music.

"Variation is something that is interesting to people, it's memorable," said Bolton. "Part of the testing dynamic is that different people bring a sense of discovery to what's going on. Far too often...certain people think about testing something that must be repeatable. In music, unless it's coming out of a machine, exact repetition isn't an option."