EU career candidates often wonder who the "assessors" are in their EPSO Assessment Centre and whether they really are able to provide an impartial, professional evaluation of their performance at the EPSO exam.
To gain more insight into how the assessors are trained, I myself (András Baneth) followed an intensive course run by the same company EPSO commissions to run its own Assessment Centre in Brussels (and exceptionally, in Luxembourg).
At the "assessor training course" I learned some great insights about the assessment centres, the way evaluators are trained and the code of conduct the members of a selection board, set up by EPSO or any other organisation, should follow. To help your preparation, I have collected here a few ideas that will hopefully be useful.
1. Assessment Centres have at least 5 "unique selling points"
Find and identify suitable candidates : if you have ever tried to recruit someone for a job, you certainly know how hard it is to find the most suitable candidate for the given role. Even after the pre-selection tests, there are hundreds of EU candidates in the competition who must be carefully evaluated.
Graphology? It seems like a fun assessment method but its ability to predict who will be an effective EU official is close to zero. Previous work references, a simple CV-screening or "classic" (unstructured) interviews are somewhat better...but still far from perfect. So there must be a really effective way to select and recruit, and that is the Assessment Centre.
Assessment Centres were created because they deliver many safeguards and checks that other methods are not able to. This includes using multiple criteria to assess the candidates, multiple observers (assessors) to ensure homogeneity and lack of bias, peer review of assessor’s findings and assessors’ challenge of each other’s view to make the judgments legally and professionally water-tight. The aim is to learn about candidates in a holistic way instead of just focusing on their organizational skills or analytical abilities.
2. Is being objective about the EPSO candidates possible?
Who or what is being observed? Who or what is being observed? By the nature of the EPSO Assessment Centre exercises, it is candidates’ behaviour that is being judged, which makes it prone to subjectivity from the assessors. Therefore a special methodology must be used to avoid this and to ensure a streamlined methodology.
Intimate knowledge of the EPSO procedure : The assessors will need to be intimately familiar with the entire EU selection procedure and the process of evaluating candidates. They need to know the exact criteria based on which they are scoring them (more on this below). They need to know the schedule of the assessment day so if a candidate overruns his or her time in the Oral Presentation, or doesn’t arrive on time to the Group Exercise, this may be reflected in the scoring. Moreover, rating forms must be uniform for all assessors, and most importantly, the method, including the positive and negative indicators (that is, sub-categories of each EPSO competency) must be crystal clear to each of them.
Cognitive bias and subjectivity : One way an assessor could evaluate a candidate’s ”Working with others” competency would be to simply say ”Well, this candidate seems to cooperate well with her colleagues”. However, the proper way to assess is to judge the performance of the candidate against each indicator of that competency (in the above case, indicators such as ”Demonstrates openness to others’ ideas” or ”Tries to find a compromise in case of conflicting views”). The overall score for a given competency can thus be awarded. The key is to minimise subjective judgments or cognitive bias (ever heard of the ”halo” effect?) and streamline the evaluation procedure as much as possible..
3. What the hell is a ”behavioural statement”?
Observed, registered, done : the essence of the 7+1 competencies that EPSO evaluates is that each of them must be described and detailed by the above mentioned indicators, which are then observed and registered through ”behavioural statements” during the assessment itself: what assessors ”want” to see you do by directly observing your actions in the Assessment Centre.
What cannot fly : if an assessor writes in his or her notes that ”the candidate seems to think high of himself”, or that the ”candidate is aggressive”, this cannot be considered as a behavioural statement. That would need to be something like ”candidate mentioned their qualifications 10 times during the group discussion” or that she ”raised her voice and pointed at other members of the group when talking”.
Qualified assessors will not write down in their original notes that the ”candidate understood that her role was very important”, because this is not linked to a specific action that could be observed directly.
See, hear, OK : the take-away from this is that assessors must be very careful to only write down what they actually see or hear but they cannot infer anything else from it – yet. The conclusions come at the end of the day when descriptive notes about your actions are turned into an evaluation: the dots between your behaviour, as shown by your actions, are connected and each competency (such as Leadership or Learning and development) is scored on this basis. More on this later in this article.
So the elements look pretty much as follows:
4. Exercises and tests are great... but there is a limit!
Competencies - max. 8-10 : at the SHL course I also learned that in any assessment centre (and EPSO’s is no exception either) the number of tested competencies should be limited to 8-10. That is why you are tested on 7+1 competencies in the EU career system (the plus one refers to Leadership which is only tested for Administrator roles) and not, say, the full 12 EPSO competencies which include areas such as “Creating & conceptualising” or “Enterprising & performing”.
Exercises – each to test max. 3-4 competencies : another ‘industry best practice’ is that each test exercise should focus on no more than 3-4 competencies. This in practice means that when you are in the Group Exercise, no more than 3-4 competencies are being evaluated (such as Working with others, Analysis & Problem solving etc.), though in some specialist EPSO exams there could be exceptions from this rule. In any case, assessors should not be requested to focus on so many competencies at the same time while registering your words, actions and non-verbal signs.
Exercises – max. 5 : the last limitation is the number of exercises or tests used in the Assessment Centre. Candidates should not be request to sit more than 5 as this number is sufficient to provide a solid picture of your competencies and personality. Again, EPSO may occasionally have 6 tests for certain specialist roles (such as a second ‘interview’ apart from the usual ‘Structured Interview’), but this is the exception and not the rule. The core test exercises are the Structured Interview, the Oral Presentation, the Group Exercise and the Case Study.
5. I believe it when I see it
Professional assessors like those seconded EU officials who serve on the EU Career selection boards may have four ways of understanding who you are and whether you have the required level of competencies. The ways to identify and apply evidence are the following:
- Inferred behaviour : this is based on certain pieces of evidence such as your CV or the fact that you arrive in a T-shirt to the EPSO competition and certain conclusions are drawn from these. For instance, if this method of gathering evidence about you was used, assessors could infer from your appearance that you don’t care much about the competition and thus reject your candidacy. This method, of course, is not best practice and it is hardly ever used in an assessment centre (it may nevertheless be used in other HR processes).
- Measured behaviour : as the name suggests, this is a scientific way of gathering evidence and it is quite reliable, though it has its limitations given its indirect nature. For instance, the pre-selection tests such as the abstract, verbal or numerical reasoning exams or the situational judgment tests (SJTs) are measured according to a special matrix because the assessors are not sitting behind your neck looking at how you answer the test. Also, the case study or the professional knowledge tests can only be evaluated on the basis of measured behaviour, but no such method can be applied to “measure” your activity in the Group Exercise, so it is checked through a different method.
- Reported behaviour : this is certainly secondary evidence and it has its merits, but it is another indirect method and thus should be treated with care. The Structured Interview is a typical example where you report a certain past behaviour and give details about it to the assessors. Since your word is the only source of evidence (though your body language, your answers to the follow-up questions and other ‘signs’ are also factored in), this needs to be thoroughly tested to make sure there is no “distortion” in it. Also, an endorsement of or a recommendation letter about you can equally serve as a confidence-building tool but as these are also indirect references to your behaviour, they may be biased.
- Displayed behaviour : this is one of the main ways of gathering evidence about you, your competencies and your overall personality in the Assessment Centre. You demonstrate a series of behavioural traits during the Group Exercise or in the Oral Presentation, and the task of the assessors is to carefully take notes and observe your displayed behaviour. This is a very strong source of evidence given that it is terribly hard if not impossible to consistently manipulate or alter your behaviour (you may be able to do so for a while but given the number and duration of the exercises, you will sooner or later act as your real self).
6. Step-by-step assessment method
When facing an Assessment Centre, you may wonder: how do assessors actually evaluate me throughout the day? I was also asking this question before the training course and got a detailed answer eventually.
Step #1 – Observe : first and foremost, all assessors are required to observe your actions and behaviour. This may sound obvious, but it is very easy to go beyond that and immediately attribute characteristics to the candidate (“he is very self-assured” – this is not an observation, this is already an evaluation!). Observation is limited to what the eye can see or the ear hears (or, as a matter of fact, what may be missing: a lack of answer, a lack of acknowledgment, a lack of interaction in a group etc.), it may not go beyond that. A good way to make sure assessors are only observing is to use almost exclusively verbs in their notes and fewer adjectives.
Step #2 - Record : the second step is to record what you have seen. Again, this may sound very easy but if you try to observe a person in a group (why not check a studio discussion on TV or Youtube and try to focus on recording the actions, words and gestures of one participant!), it can be challenging to write and watch at the same time. Even if an assessor may never observe more than 2 persons at the same time, each person must be considered in a social setting and not in isolation (this is true mostly for the Group Exercise of course).
Quoting what a candidate said is just as important, because it can serve as perfect evidence when it comes to the evaluation phase of the assessment procedure. Lastly, assessors make sure they only gather evidence during an Assessment Centre exercise and avoid anything they may have seen during the lunch break or outside the centre in Avenue de Cortenbergh (where EPSO’s headquarters are).
Step #3 – Classification : the third step is classification of the recorded evidence, which is done in a way that assessors will circle or underline or draw arrows from each piece of evidence and link these to specific competencies. This is a delicate exercise because certain pieces of evidence, such as a candidate regularly interrupting another one, can be classified under “Working with others” or “Resilience” as well. Assessors may not do this as each piece of evidence should only count once – you should not be double penalised (or double-rewarded) for the same behaviour. On the other hand, all evidence must be considered in context so if you interrupt someone once, this will not impact your score negatively...unless there is other supporting evidence that you don’t work cooperatively with others.
Step #4 – Evaluate : finally, assessors evaluate all evidence under each competency (e.g. under “Resilience” they may have classified pieces of evidence such as “Candidate was silent and folded his arms for 5 seconds when asked a challenging question in the Oral Presentation” and “Candidate could not back up his claims that option #3 in the Group Exercise was the best one when others in the group challenged his position and he said “I don’t really know what I prefer this but I like it”) and convert this into a score. EPSO usually uses a metric of 1-10 per competency, while for specialist exams, a metric of 1-5 is used per competency.
7. EU integration? Not this time: Assessors’ integration session or How scoring is done
It is vital for qualified assessors to follow a strict methodology to prevent any bias that may harm candidates (or lawsuits!). Therefore, certain rules must be observed, such as the following ones:
- Wait ‘till the end : Assessors will never discuss any particular candidate before the full day is over – otherwise they risk that the other assessor will evaluate the candidate with pre-conceived ideas or even prejudice.
- Semi-anonymity : Candidates are usually scored and evaluated on their first-name basis and a candidate number but assessors don’t know their full name, their CV or any other information which may affect their objective evaluation.
- Group exercise for the Assessors : when all assessors gather, the group dynamics between them must also be handled with care. A dominant assessor and his/her opinion should not affect the scoring of any candidate: if one assessor would give 3 points out of 10 for a given competency, but another one would give 7, they should find a compromise and not have the first assessor’s opinion, due to his/her seniority, experience or other factors, “weigh” more than the second one.
- Scoring challenges : Assessors must be extra careful about the scoring, they mustn’t go to extremes (top score, absolute minimum score, one score of a competency affecting another competency’s score), and they must clarify in advance if fraction points (e.g. 3.5) are allowed.
- Decision-making procedure : How do assessors as a group make decisions? Is it by consensus, by voting, by drawing lots (joking J) – this is a fundamental issue though most scores and candidates are generally agreed on by consensus.
Well, if you see this paragraph, congratulations for reading through the entire article!
This means you are really interested in Assessment Centres, and my guess is you are a candidate in an EPSO exam. If so, make sure to check out our online training webinars, our classroom simulation courses and private coaching services!
And you can of course ask us for free advice, any time by clicking here!