Devil's Advocate: The perils and pitfalls of 360° appraisal

'To the man with a hammer, every problem's a nail'. In the field of Human Resource Development it seems that at present, 360° appraisal is that hammer. A useful tool certainly, but only if applied appropriately and skillfully. Articles abound about the merits of 360°appraisal; in this article I will play Devil's Advocate and outline some of its perils and pitfalls.

Do you know what you are measuring?
Underpinning most 360° appraisals is a competency model. Those companies that create their own 360° instrument at least know that it reflects the skills and behaviours that are most important to them. But many off-the-shelf products are bought without due regard to content. Ask yourself:

  • What is this tool measuring?
  • Are these the competencies that are most critical to our organisation?
  • What research and validation has been done? /in which organisations? /how long ago?
  • Does the questionnaire let you rate the importance of each competency to your own organisation?

    What sort of feedback practices do you want to encourage?
    The term 'multi-rater instrument' is now often used in preference to 360° appraisal. It is more accurate in that 360° feedback simply indicates who provides the feedback, rather than the means of collecting it. It is quite possible to gather feedback from multiple sources without the benefit of a multiple-choice questionnaire. It may well be preferable to do so. The use of questionnaires can set the expectation that feedback is given anonymously, in writing and at infrequent intervals. It may be the only option that is workable in some situations but that doesn't make it the ideal model for building a developmental culture.

    Will it travel?
    Using 360° instruments across cultures is fraught with problems. In some cultures direct reports will not readily give negative feedback about their manager, even anonymously. In one situation I was about to give the feedback on a Benchmarks questionnaire to a Malaysian manager. Unusually, his report showed that responses had been received from 5 superiors, 5 direct reports but no peers! It turned out that he was a high flyer who was younger, and less experienced than other managers at the same organisational level. They naturally (in that culture) regarded themselves as his superiors rather than his peers. What is more, they rated him low on all competencies - after all he was young and inexperienced, how could he perform well? In contrast, his direct reports rated him very highly on all competencies.
    The competency model itself may not travel well. Compare, for example, what passes for good decision-making in the US and Japan.
    Finally, even in cultures that are closer to one another there may be quite different attitudes to rating scales. Americans often ignore the lowest rating on the scale, whilst in Europe it is the highest rating that is scarcely used.

    What happens next?
    In principle a 360° appraisal serves as the basis for a focused development plan. In practice it too often results in information overload. Just as important as the feedback data itself is a useful process for development planning; making sense of the feedback, identifying and prioritising development needs and finding the support and the resources to implement the development plan.

    Are you ready for the administrative effort?
    When you consider that for 20 managers you can easily have 200+ bits of paper to manage, you can see the potential for a hugely time-consuming and complex administrative effort. Remember too that the responses are anonymous and confidential so you will need to keep track through some system of coding. Typically, questionnaires cannot be scored unless there are a minimum number of responses, so you will have the task of sending out reminders.
    Some suppliers have started to address the administrative issue, but usually this relies on electronic distribution of questionnaires so that administrative functions can be automated. Not all suppliers are willing to do this.

    In summary:

  • Make sure that the instrument you choose is relevant to your needs
  • Don't make it a substitute for ongoing, face-to-face feedback
  • Be sensitive to cultural issues
  • Back it up with good development planning support
  • Simplify the administration as much as possible

    Preparing Documents for Global Use

    Companies continue to globalise through expansion, mergers or joint ventures. Many managers are faced with the challenge of developing processes and policies for international use. This article offers some practical tips for producing such documents.

    One size doesn't fit all
    You may find it difficult to aim your document at a level that suits everyone. In one of our recent projects a senior headquarters manager commented 'I don't need all this - just give me a couple of flow charts and some notes.' However, in the far-flung reaches of your business there may be managers who have considerably less experience, less support and more limited English. The same document produced an enthusiastic response from a manager in China. 'I didn't have to re-read a single sentence!' he commented.
    One solution is to include some overview pages backed by step-by-step detail. This helps different readers to find the level that is right for them.

    Two-way traffic
    It may not be practical to travel round the world to present a new process. Often, the document you produce will be the only available reference. It must be clear enough for users to understand and implement the process. Try to anticipate the questions that may arise. Better still, ask a sample of intended users to review your first draft. Reinforce the aspects that were not clear. If possible build in some two-way communication. After you distribute the document, set up a Question and Answer forum on a Web site or on a Tele-conference. Welcome questions as a natural part of the implementation. They do not imply criticism of the document.

    Give me a sign!
    Within a long and complex document provide clear signposts telling the reader where she is in relation to the process/document as a whole. Some of the ideas for doing this are:

  • Follow the classic advice 'Tell them what you are going to tell them, Tell them, then Tell them what you have told them'. For example, you could start each main section with an Introduction and Overview and end it with a Key Points Summary.
  • Use a Key Steps Overview and have the number and title of the relevant Key Step visible at the top of each page. Alternatively use a simple flow-chart graphic and use it as an icon on each page, highlighting the relevant step.
  • Use graphics and icons. For example, you could adopt an icon for each function or person who plays a part in a process - line manager, employee, HR department etc. So it is always clear who does what.
  • Use information techniques such as Info Mapping. This approach divides information into clear chunks, each with a label. The page is then easy to scan, and the labels provide a natural key point summary.
  • If you distribute the document electronically, make use of key words, hyperlinks, bookmarks and clear navigation tools.

    Mind Your Language!
    People whose first language is not English may need to understand and implement the process. Remember that the effort of concentration, and the potential for misunderstanding will be greater than when they are reading in their own language.

  • Check the 'readability' of your document. Many word processing packages include a tool to do this. It will show you the average length of your words, sentences and paragraphs. Look for ways to shorten them.
  • Some tools give you a score. Aim for a score of 60-70 on the Flesch Reading Ease scale, or 7-8 on the Flesch-Kincaid Grade scale. (This article scores 7.7)
  • Check through your document for jargon or culturally specific references.
  • Make the language as direct as possible. Compare ' forms for your use can be found in Section B of this binder' with the simpler 'use the form in Section B of this binder'

    Size matters!
    Remember that US paper sizes are different from those used elsewhere. A4 is longer and narrower than the US Legal paper. Electronic files are unlikely to print out well at the other end. Paper documents and binders will not fit into standard filing systems or shelves. A4 paper is not readily available in the US, just as US Legal paper is not easy to find elsewhere. You may need to re-format your document for US/non-US use.

    If all these tips sound self-evident, then I would guess that you are either an expert or a novice. Getting it right is far from simple. The above tips are based on several projects for a global healthcare company.

    Published on HBC Web-site 10/1999

  • Back to Top

    Feedback on this site is welcome. Whether you have compliments, complaints, ideas or requests, simply type your comments into the feedback box and click SUBMIT. Thank you!

    Copyright (c) 1999-2002 Helen Bouchami Consulting. All rights reserved. Material on this site may be reproduced in any medium for non-commercial purposes provided that it is attributed to HBC, and this web-site address quoted.