The knotty problem of adjudication - by Stephen Roberts

The composer Stephen Roberts has plenty of experience of being in the box. Here he gives his thoughts on adjudication problems.

I recently wrote about my experiences adjudicating at the NABBA Championships and inadvertently found touching on a subject which has recently been the centre of attention.

So I thought I might follow up my NABBA report with a halfpenny worth of my thoughts on adjudication in general.

I have been privileged to act as an adjudicator in many fields, not only for brass bands and solo contests, but also as a moderator and examiner at music colleges and universities, BBC Young Musician, school festivals and competitions, etc.

I have also been on the receiving end of these competitions as performer and conductor, so am a poacher turned gamekeeper, to coin a phrase! It seems that as one is deemed more venerable in old age, then one is called in more often to judge the efforts of others. Greying locks lend an air of impartial wisdom and gravitas!

So here are some thoughts, which I hope may stimulate yet more debate.

There are essentially two main types of contest. One where candidates are subjected to examination and the other where candidates volunteer to enter a competition of some kind. To a degree both types of contest are entered into in order to further the aspirations of the competitor.

In the final analysis the results are at the mercy of the adjudicator(s) who must make a decision. In one sense this is similar to the role of the politician who has to take decisions, whether popular or not, because that is his job.

I believe we are far more likely to respect the decisions of politicians who take a measured and impartial view, even if that decision may go against us. Where a politician is seen to be partial, because of party allegiances or other preformed agendas, there is bound to be accusation of bias.

In adjudication of examinations there are often strict criteria laid down by the organising bodies. These are designed to limit subjective adjudication and obviate any cause for complaint from the candidate and protect the adjudicator's integrity. When the examiner consults the laid down criteria, he knows he is examining as instructed by the examining body and has that yardstick to lean on.

Strict criteria are laid down by all universities and music colleges, as well as the Associated Board and similar examination boards. In a way it is ‘the law'. To a degree these ‘laws' are a reflection of the attitudes and background of the individuals who constitute them. They are a corporate measured response to the needs of the society, which adopts their practice. If you enter the examination, you do so with the knowledge that you must fulfil certain fixed criteria to do well.

In music festivals and many band contests, there are often no criteria laid down. The adjudication relies on the experience and integrity of a person or persons who are known and respected in their field. Because of this, there is bound to be room for manoeuvre since the chosen adjudicator must create their own best set of fair criteria. In a university situation all assessments have to be second marked, sometimes ‘blindly' so that the second marker has to assess without reference to the first marker.

This is not hard to do since both markers assess according to the criteria as laid down in an official document. Thus, in a recital programme, if a performer manifests, say, a well-projected sound, and this is an essential part of the marking criteria to gain a 2 ii, then this should be included in the examiner's write-up. All assessments may then be audited and the inclusion of key words or phrases greatly assists the auditing process.

If you read my report on adjudicating at NABBA you will have noticed that each judge had to give marks for specific areas and then add those marks together for a final tally. This is more in keeping with the type of adjudicating carried out for examinations.

In British brass band contests there are usually no formal criteria laid down and that is not necessarily a bad thing since they are not examinations as such. Actually most band enthusiasts would say they are much more important than that! But the lack of formal criteria does leave room for subjectivity and that subjectivity is inevitably going to emanate from the adjudicator's musical experience and background. That is not a criticism of any adjudicator. We all form our opinions in this way – we CAN only be a natural part of our own experiences because that is what makes us.

However, it is fair to say that adjudicators in brass band circles tend to be chosen because of their track record and acknowledged experience in this somewhat specialized field. Thus their set of criteria will usually match the expectations of those taking part in the contest and, although these criteria may not formally laid down, their degree of objectivity is tacitly accepted by the majority and the adjudicator will do his best to administer in a way that reflects the expectations of that majority.

To this end there is no doubt in my mind that closed adjudication assists the fairness of the process. Justice is blind and weighs the balances with impartiality.

I still agree with the notion of closed adjudication, since I have experienced all forms of adjudication and can state with certainty that one's assessment is quite different when one cannot see who is playing. This is not to say that I am swayed by WHO is playing, but that the visual element adds another branch of aesthetics. Performing music is a visual, as well as aural experience.

We know this in brass band contests because it is classed as ‘entertainment'. A conductor who is visually exhilarating can subliminally enhance anyone's enjoyment of a piece without necessarily doing so aurally. The visual dimension adds another branch of subjective judgement, which can affect our response to a performance. In fact I know this as a performer as well.

If I give a confident impression as a player I am far more likely to get away with imperfections than if I adopt a diffident or apologetic stance. Actually you could apply that maxim to just about any aspect of human endeavour! (I remember a recording session where the conductor thought I might play a wrong note simply because my instrument was not lacquered!)

Now when it comes to adjudicating less experienced performers I believe the moderator also has a duty to make constructive criticism by observing the good, as well as the bad aspects of a performance. He has a duty to make didactic observations. In such situations, open adjudication may be of benefit, since posture, breathing, stance, etc. may be remarked upon for improvement.

At the very top level, however, such observations may be less appropriate since the level is so high. The top bands all have well-projected tone, good intonation, tight ensemble, reliable and virtuosi soloists and conductors who are musically adept and experienced. Sometimes the winning performance is ahead by one split note.

Or is it?

How do you weigh up a performance that is scintillating, but prone to splits, with one that is immaculate, but less exciting? Even with strictly laid down criteria, you might still end up with a tie. But can you say one is better than another? It's swings and roundabouts, apples and pears and all those other clichés.

Personally I would go for musicality over fastidiousness, but to make a qualitative judgment one has to weigh each against the other. (The ideal is a performance that combines both, of course, and when that happens the adjudicators are relieved, because their job is clear-cut.)

However I believe one should never allow personal preferences to colour objectivity when assessing performances.

Consider all this furore over Wagner's turns in Rienzi. If someone plays them the ‘wrong' way, then so what? If it is stipulated in the rules or foreword that it MUST be played in one way or another – then fair enough (perhaps the arranger should have written it out to eliminate doubt). But, if a judge is foolish enough to have an agenda about it, without telling the bands, then that is not objective adjudicating and unacceptable.

You cannot have adjudicators having some ‘secret agenda' so that bands are performing in a lottery instead of a fair contest. I am not suggesting that anyone did do this, by the way, but merely using it as a potential example of unfairness. What is more important is the overall performance. Consider a beautiful face. Its beauty is not lessened by a slightly crooked smile, any more than a first rate performance of Rienzi is lessened by turns going the ‘wrong' way. You might say it gives it individuality, character, personality, spirit and so forth. Certainly I, as a human being, value these traits more than the bureaucratic niceties of notational ambiguities.

A performance is, ultimately, judged as a whole and a winning performance is greater than the sum of its parts.

There are other examples of possible subjectivity, which might be considered unacceptable.

Consider tempo. Two bands can take a section at exactly the same absolute tempo, but one may sound scrambled and rushed, whilst the other may sound fluent and exciting. For an adjudicator to have a preconceived idea about the tempo and penalise both bands because the tempo is ‘incorrect', is patently erroneous, or, at very least, pedantic.

What matter are the expressive content and appropriateness of mood and how these apply to the performance taken as a whole. Strict criteria may dictate that a tempo should be within certain tolerances. But consider, say, Holst's tempi in his recordings of the Planets. They are invariably MUCH faster than any other recordings made by many great conductors since, and his tempi were also fast in live performances, according to Imogen Holst.

Who is ‘correct'? Clearly there is no such thing as a ‘correct' tempo, even if the composer himself may prefer one to another.

Some adjudicators may have betes-noires over details such as bass pedalling, vibrato, weight of tone, stylistic preference and so forth. In my view this type of pedantry has no place in a music contest. Adjudication should be open-minded, and competent and confident enough to accept that there are many ways to skin a cat.

That is not to say that any performer should be arrogant enough to take liberties with a score. Where bass pedalling detracts from a score, it is clearly wrong. But the main point is that the judging process should be about the performances and not about the adjudicator. The adjudicator's preferences should be irrelevant.

All of which brings us back to the knotty problem of judging criteria. How prescriptive should they be? The very imposition of criteria is a subjective process in itself. Who decides the preferences and how many should there be? Should, say, a fine sound score more points than fine ensemble? Who is going to make that value judgment? Even when these criteria are laid down it is still up to the individual judge to interpret them. Furthermore, where the room for individual interpretation is so proscribed that the adjudicator is merely ticking boxes, then the situation becomes meaningless.

Finally I should like to touch on the notion of individual, as opposed to committee type adjudication. At NABBA each judge is independent, and because the judging follows guidelines, this works very well. When I arrived at the adjudicators' meeting I was worried that I would not be able to fairly assess the first band's performance, as I had no yardstick to go on and was told that comparing performances was forbidden.

For all I knew the first band might have sounded quite mediocre, but been the best band there. The fact that I had to apply marks to five SPECIFIC CRITERIA meant that I did not have to apply a general, or comparative, yardstick mark for the first band and could award an absolute mark. The five main criteria were also given sub-headings for consideration to assist the moderation process. These included key words, such as blend, ensemble awareness, balance etc. These guidelines tend to reflect the marking criteria of university and college examinations, as outlined above.

Where the independent judges are given fair-minded and enlightened guidelines to apply objective marks to given criteria, then I am in favour of independent adjudication. With one set test piece, however, this could all too easily result in a tie for first place.

This is where the committee scores, since discussion can ensure that there are no ties. The trouble with committees is, as we all know, they tend to be dominated by personality and the chemistry of any committee is determined by its members and their respective personalities.

Suffice it to say that every adjudicator I have met sees it as his or her duty to do a good job, to the best of their ability. Every performance is listened to with equal care and it is does not really matter whether adjudicators discuss, or not, as long as there is a consensus as to how to apply judging criteria. A committee of judges agrees to its own set of criteria in the same way as individuals apply marks to given criteria.

The application of value judgements is set by a process of experience, tradition, social acceptance and knowledge.

In addition many adjudicators attend special courses to ensure these values. What is imperative is that the judging is SEEN to be equitable and that is why I believe the publication of some guidelines for marking criteria would be a good public relations exercise on the part of contest boards.

This is particularly relevant to the area finals where there is usually only one judge. If the judge can point to the fact that this band did not fulfil expectations in this or that specific area, then everyone knows where they stand. The audit trail is simple and nobody can logically argue with it. However, as I have discussed above, the process of applying specific marking criteria is not clear-cut.

To conclude, I would suggest that each contest has a set of marking criteria in the same style as those adopted by universities and music colleges for recitals. These should be tailored to suit each contest. In this way adjudicators could be confident in their impartiality and candidates could be clear about their objectives.

Stephen Roberts

^back to top