Documents may also be tested with a focus group -- a small group of people who represent the target audience. "It's critical to test a document with the people it's written for," Gerry stresses. "It's no good asking university students or professional copywriters if they understand it, because they're not typical of the kind of people who will be reading it."
The client specifies the audience for a particular document. For example, brochures and forms issued by the Department of Social Security are used by people who get allowances, pensions or benefits. Bank documents are read by anyone with a bank account or credit card, while information brochures about new government policies may be relevant to the entire adult population.
Quadrant Research Services recruits focus group participants in three ways. It has networks of community, scout and other types of clubs in Sydney and Melbourne. The club nominates members who meet particular criteria and who want to be involved. As focus group members are usually paid a fee of $25-30, this helps the club to raise funds, and allows the market researchers to get feedback from people they wouldn't otherwise reach.
"If we need a very specific type of person, we virtually find them by random phone calls," explains Gerry. "We ring up people over the weekend and run a screening questionnaire past them. If they fit the bill we use them. We also recruit focus group members through our fortnightly omnibus surveys which contain questions from different clients. At the end of the survey, we ask people if they'd like to be involved in a focus group. If they say yes, we keep them on file. So if we need a redhaired 40 year old man, we can find him."
Gerry says that focus groups often produce more meaningful and fuller feedback than one-to-one interviews, due to the stimulus of, and interplay between, participants. "When you draw up the questions for an interview, you virtually have to predict the pitfalls of the document," he explains. "But when someone in a focus group says, 'I don't understand this,' you may not have anticipated that misunderstanding."
"We like people in focus groups to respond to the document, rather than just read it. When they read it they often say, 'It's fine -- I understand all that.' But when you put a scenario to them -- if you were going to apply for a pension tomorrow what steps would you take? -- you find out whether they really understand it."
Another benefit of focus group testing is that it pinpoints any problems, particularly with forms. "When many people fill in a form incorrectly, you know it's badly designed, but very often it's not obvious why," points out Gerry. "In a focus group, you can find out why they answered that way."
"The problem with many documents is the starting point," he sums up. "Government documents often start with legislation. Private sector documents setting out terms and conditions sometimes begin as a contract. You need to put yourself in the client's shoes and start from there. The main changes we recommend are cutting out jargon, acronyms and legalese."
Gerry reports that, as more organisations understand the importance of plain English, the quality of his clients' draft copy has vastly improved. Today, design tends to be the stumbling block. "Now that desktop publishing is so much more sophisticated, the drafts we get are almost finished quality. But many of them are too fancy."
He summarises the main design problems as:
- a cluttered design which makes the reading order unclear
- mixed typefaces which make the document look messy and confusing
- a lack of headings or boxed summaries, so that the reader needs to scan the entire document to find the information they need
- inappropriate cartoons which appear to make fun of a serious subject.
When should organisations market test their documents? "A new document providing completely new information does need to be tested," stresses Gerry. "Most documents are revamps or updates. In that case, if the organisation is already using plain English, they wouldn't need to test every one. But I do recommend an audit process. If an organisation puts out 30 or 40 brochures a year, they should test three or four at random, just to make sure they're on track."
People are often surprised to find how few people are needed for focus group testing. This is where recruiting the right people is vital. "You don't need hundreds of people to find the one exception," points out Gerry. "You need to use only 20 or 30 people representative of your target group. If one person misunderstands your document you've learnt something. After all, you need only a spoonful of wine to know whether the bottle is okay." Three or four focus group discussions involving 20 or 30 people cost around $8,000 -- a minor part of the budget for documents developed for a mass audience.
Gerry says audience testing is like an insurance policy. "It prevents waste. If your document is misleading or misunderstood -- or not read because it's too daunting -- the whole thing is wasted. If you've invested a lot of time and money in a document, why not take out an insurance policy for a few thousand dollars and make sure it works."
5 February 2004