How to run a successful assessment centre in a virtual world

In December last year, People Solutions team members Georgia and Courtenay ran an online webinar to a range of HR managers. The webinar delved deep into assessment centres, and how to successfully run them in a virtual world. We know that COVID-19 has created an extremely challenging environment for businesses, and many have had to adapt their entire business model to survive. The question is, how do you meet hiring outcomes and maintain best practice in a virtual world?

 

Webinar Agenda

• Introduction

• • People Solutions, who we are
  • What is an assessment centre?
  • What are the benefits of an assessment centre?

• Part One: The Challenge

• • What is the problem we are trying to solve?
  • Achieving outcomes and maintaining best practice

• Part two: Case Studies

• • Case Study 1: Entry Level Role – Mining Services
  • Case Study 2: Trainee/Student Role – Emergency Services
  • Case Study 3: Multi-disciplinary Graduate Role – Mining & Chemical Manufacturing

• Part three: Lessons Learnt – Courtenay McGill

• • Limitations
  • Recommendations for the future

• Questions

Watch our webinar, or read the transcription below

 

 

[00:24]

Introductory Slide: People Solutions

Running Successful Assessment Centres in a Virtual World

9^th December 2020

Georgia Nedkoff & Courtenay McGill

 

[00:39]

Georgia: Hi, everyone. Welcome to our webinar today. This is actually our first webinar that we are running. It’d be great to hear your feedback. We look forward to sort of running many more of these webinars as well in the future. My name is Georgia. I’m a registered psychologist and consultant at People Solutions. Next to me is my colleague, Courtney. We didn’t color coordinate our outfits today. They just happened to both be quite bright. So, hope you enjoy. She’s also a registered psychologist and consultant at People Solutions as well. We look forward to sharing our insights with you today. In terms of the structure of the session, we’re not going to be chatting for the whole hour. The talk itself will probably go for about half an hour. Then, we’ll open up to some questions so if anything does pop out throughout the session, feel free to write it in the Q&A box and we can respond to those at the end. Anything that we don’t get to in the session, we’ll sort of write it up and send an email around to everyone as well at the end that will include the recording and the slides as well. You’ll have all that information there and feel free to forward that on to anyone who may have missed the session, or you think would be of benefit hearing any of the stuff that we talk about today.

 

[02:02]

Georgia: I suppose before we jump in who we are, if anyone on the line isn’t familiar with People Solutions as a business.

 

Slide content: People Solutions

• Perth-based Organizational Psychology business with predominantly WA head office customers, as well as national and global customer base
• Strong track record in providing Assessment Centre and Psychometric Assessment Services

Our purpose “Helping People Succeed”

 

Georgia: We are Perth-base, organizational psychology business. We are based in WA a head office is in Subiaco. We do primarily with Western Australian or Australian based organizations as well. We do have a really strong track record of providing assessment centers and assessment solutions as well.

 

[02:36]

Slide Content: Overview 

• Part One: The Challenge – Georgia Nedkoff

• What is the problem we are trying to solve?
• Achieving outcomes and maintaining best practice

• Part two: Case Studies – Georgia Nedkoff & Courtenay McGill

• Case Study 1: Entry Level Role – Mining Services
• Case Study 2: Trainee/Student Role – Emergency Services
• Case Study 3: Multi-disciplinary Graduate Role – Mining & Chemical Manufacturing

• Part three: Lessons Learnt – Courtenay McGill

• Limitations
• Recommendations for the future

 

Georgia: Essentially, the structure of the session today. Initially, I’m going to be just giving a bit of a brief rundown in terms of what assessment centers are for those who may be less familiar. And also look at that challenge, what that challenge was that we’re trying to solve when we develop these online assessment centers. Then, we’ll go into case studies. I’ll be talking to a couple of case studies where we ran online assessment centers and Courtenay will also be mentioning another one that she’s running herself. And then, a last part will be about lessons learnt and sort of limitations and recommendations for the future as well. So, I do hope that you find value in the information that we provide today as well.

 

[03:21]

Slide Content: What is an Assessment Centre?

Multi-Method/Multi Assessor Approach – Behavioral activities – Interview – Psychometric Assessments

 

Georgia:  As an understanding, maybe people with varying levels of understanding of Assessment Centers on the webinar today. Just to give you a bit of an initial introduction, and as a refresher for those who are familiar, what an Assessment Center is. It’s essentially a structured and objective approach to selecting candidates for a role. What that looks like is it to Multi-method and Multi Assessor process? Multiple-Method means that candidates get to participate in more than one activity as a part of the assessments in today. What those activities might look like there could be some Behavioral-based activities. These are typically things like you may have heard of building a tower out of Lego or building a tower out of spaghetti. It’s the same kind of practical team-based hands-on tasks. Behavioral activities may also include things like group discussion activities, where candidates have a bit of a chat and problem solve together through a scenario. It could also be things like one-on-one role plays as well, or any sort of web sample activities too. Another activity could be an Interview. These may be things like behavioral or technical interviews as well, or situational based interviews. And another piece may be sort of Psychometric Assessments, whether these are things that candidates complete on the day, whether it’s something that they complete prior, and the information is discussed on assessments in today. The beauty of it being a Multi-Method Approach is that you get to see the way a candidate behaves in more than one setting as well. Particularly for candidates that may be a little bit more reserved or quiet in those group activities, you get to say how they’re going to be one-on-one in those interview performances or vice versa.

 

[05:16]

 

Georgia: If they’re not too strong at interview, you might see them perform a little better in those team-based activities as well. So, it’s really good from both ends in terms of assessors, being able to understand candidate’s behavior, and performance, but also a candidate having a good experience in terms of the session and feeling like they were able to really show themselves on the day as well. The other piece of the puzzle is that it’s a Multi Assessor Approach. What this suggests is that the candidate doesn’t go through each of these activities and is assessed by the same person, because there would be a conflict of interest or bias. And they’re supposed to assess candidates throughout the whole day. It’s Multi Assessor, which means that it’s more than one person from the organization or consultancy that will be assessing the candidates on the day. These can be HR, Hiring Managers, Psychologists that can assess these candidates throughout the day as well. The beauty of this is that these activities elicit these candidates’ behaviors in more than one setting. So, you feel like you have really good understanding of how the candidates are going to perform at the end of the session as well.

 

[06:64]

Slide Content: The Benefits of Assessment Centres

1. Effective way of predicting candidate’s on-the-job performance
2. Opportunity to sell your company to the candidates
3. Large volumes in short period of time

 

Georgia: Again, just to reiterate those benefits is that it is one of the best predictors of understanding of candidate on the job performance, but not only that, it’s also an opportunity for organizations to sell their company to the candidates as well. From our experience, a lot of candidates come away with really good feedback from assessment centers. They really appreciate the opportunity to not only speak with recruitment or HR, but also speak with Hiring Managers, Line Managers, Operational Staff as well. It also allows us to do a large volume in a really short period of time. You could have, let’s say around 30 candidates in a day for assessment centers and causes more than one person, lots of activities. They are really good ways of getting through a big volume in a very short period of time.

 

[07:24]

Slide Content: Part one: The Challenge

 

COVID-19, border closures and social distancing presented unique challenges for talent acquisition.

Companies still needed to fill roles, and recruitment practices required valid, fair and reliable processes.

The problem we needed to solve:

How do you meet hiring outcomes and maintain best practice in a virtual world?

 

 

Georgia: In terms of what the challenge was or the problem that we’re trying to solve obviously, COVID thrown in a bit of a curve ball for all of us this year due to the border restrictions, social distancing, when we’re in complete lockdown. A lot of companies still needed to hire and fill people for roles, particularly those essential positions as well. A lot of companies didn’t stop and particularly as locked down finished and restriction easing, there was an immediate ramp off in a lot of organizations, recruitment practices. Essentially, how do we make these hiring outcomes whilst maintaining best practice in a virtual setting and in the virtual world as well.

 

[08:22]

Slide Content: Other Considerations

• Technology issues and candidate access
• Potential disruptions
• Adapting activities
• Candidate demographics and attributes

 

Georgia: Some other things that we really needed to consider, some other problems that we recognize that we might face are things around these different candidates. What is their sort of demographics and attributes are they going to have computer proficiency? Are they in a position where they might be adversely impacted by technology, jumping online? Are they working night shift or working away and is it a bit too challenging for them to have reliable internet access as well? Also considering things like technological issues, any sort of disruptions, dropout of internet, also adapting the activities. Those building of spaghetti towers or Lego towers are really great in an in-person assessment center. However, unfortunately, it’s not something that the technology isn’t advanced enough for us to be able to send over pieces of Lego to other candidates. So those practical activities were essentially out the door when it comes to the online assessment centers. We’re mainly focusing on things like psychometric assessments, interviews, and group discussion activities that can be easily replicated online. And also thinking about what level of candidate is that going to be most appropriate for as well?

 

[09:33]

Slide Content: Best Practice

 

Assessment centres based on best practice design principles show significant predictive validity. A best-practice AC design requires:

• Competencies: Developed through a job analyses and reflects critical role factors. Each competency must be measured at least twice throughout the assessment centre.
• Exercises: Activities that elicit competency-specific behavior and are culturally and role-level appropriate.
• Rating: Behavioral anchors that demonstrate a below, meets and above role expectations to allow for objective ratings.
• Scheduling: Ensuring assessors are not assessing the same candidates twice and conflict of interest is removed.
• Assessor Training: Required for all assessors resulting in better quality and quantity of observations.

 

[09:34]

Georgia: I suppose one of the biggest things that we also needed to consider was around best practice. Sometimes when thinking about what best practice is, it’s easier to think about what best practice is. Best practice assessment center is not chucking a bunch of candidates in a room together, asking them to complete a task and then, assessing them on your judgment or inference. It’s not having sort of arbitrary measurements of best performance, know sort of ambiguity in terms of what expectations are. And it’s not things like speed interviewing where candidates are asked to move really quickly from table to table. It’s just not providing candidates with the best opportunity to sort of get to know you and to show their best potential as well. What is Best Practice? Best Practice Assessment Centers, if they’re done really well and they meet all these criteria is a great predictor of on-the-job performance.

 

[10:34]

Georgia: What you’re looking for is, and what we needed to ensure that we replicated was really clear competencies. So, we needed to ensure that each of the candidates were measured against job critical criteria, which for most of our assessment centers were things that were pre-established. They are succinct that we were already sort of measuring in face-to-face assessments and is ahead of time as well. Exercises, they need to elicit these competencies and these job critical behaviors, and they also need to be role level appropriate. Like I mentioned before, having those group discussion activities, if we can only do those online, we need to ensure that the candidates, that the activities are going to be appropriate for that level of candidate as well. Graduate level candidates or those discussion activities are really great for those types of roles as well.

 

[11:33]

Georgia: We need to ensure that our ratings are really clear and objective. Thinking about what does a low performer look like and what does a high performer look like? And what does an average performer look like as well for each of those competencies. Things like scheduling is one of the big tasks, but it is something where assessment centers are going to run smoothly, the candidates and assessors if they do have a very consistent and clear schedule. And also ensuring that assesses on assessing the same candidate more than once. Again, to remove any sort of bias or conflict of interest as well. The last piece is really around the assessor training. Best practice highlights that we’ll assessors need to be trained on the competencies, the exercises, the ratings just so that they’re remaining consistent in terms of their approach and that they have a good understanding in terms of what’s expected of them on the day as well. We have to ensure that all of our assessment centers still aligned with each of these best practice criteria.

 

[12:43]

Slide Content: Candidate Experience

Not only are assessment centres ways for businesses to establish whether candidates are a good fit for the role, they are also used to ‘brand’ and ‘sell’ their organization and for candidates to understand whether the role is the right fit for them.

Using technology in Assessment Centres can result in:

• A reduction of interest/engagement
• The conveyance of an impersonal organizational image

 

How do you maintain best practice and ensure a positive candidate experience when conducting an Assessment Centre online?

 

[12:43]

Georgia: Another thing we also needed to consider was candidate experience. It can be all well and good to have best practice, good routing technology, but you need to ensure that it’s also an enjoyable day for the candidates as well. Not only is it a way for us to recognize if a candidate’s going to be a good fit for the position, but it’s also a way for candidates to understand whether they’re going to be a good fit for your organization. Self-selection is a much cheaper way of sort of removing candidates from a process rather than them being hired and working out down the way that this isn’t going to be the right fit for them as well. It works both ways and candidates also, they really enjoy those face-to-face assessment centers. So, it’s thinking about how we are going to replicate that same sort of informal relaxed environment in a bit more of a formal online setting as well. The use of technology in assessment centers, the research is very much so infancy, but it can result in a bit of a reduction of interest or engagement from candidates as well. What we’re really looking at is, how do we maintain this practice? How do we ensure candidates have a good time whilst also looking at things like technology? Ensuring that these online assessment centers are going to be a really good fit for candidates and meet those hiring targets as well. There’s lots of things that we sort of thought about with developing these online assessment centers.

 

 

[14:17]

Slide Content: Part Two: Case studies

1. Mining Services (entry level roles)
2. Health Services (student/trainee roles)
3. Mining and Chemical Manufacturing (graduate roles)

 

[14:17]

Georgia: Moving onto the second part now, how we sort of worked through this. We’re going to talk through three really quite different case studies. The first one that I’ll talk through is one that we did during lockdown. This is an assessment center that still ran where all of the assessors where at home. The second one, Courtenay will be talking through and this one will be more sort of down over our restrictions easing. And then the third one is one that I’ll be talking through and this one will be more around restrictions of ease, but borders are still closed as well.

 

[14:58]

Slide Content: Case Study One: Entry level Role-Mining Services

This company has used face to face assessment centres for their Entry Level candidates for over a year. Statistical analyses of these assessment centres has shown a decrease in Entry Level Turnover and higher performance during training since using the assessment centres as a part of the recruitment process. Due to business growth, assessment centres had increased to a monthly basis pre-COVID.

COVID challenges:

• Needed to continually meet business and growth demands during complete lockdown
• Candidates still required to be assessed on role critical factors

 

Using the pre-developed competencies for the Entry Level role, we designed additional scenario-based interview questions to help elicit behavioral responses, as well as included situational judgement assessments.

 

[14:58]

Georgia: Our first case study, our first virtual assessment center that we ran was for an entry level role within mining services organization. With this organization, we’ve been running face-to-face assessment centers for over a year. Our competencies were very much well-defined, a lot of things run quite sweetly and they’re also running quite fast as well. This organization increased to monthly basis assessment centers. We were running these months to month and then, COVID happened. Unfortunately, that put a bit of it stopped those face-to-face assessment centers. It’s really the challenge for this organization was to meet the business targets, the growing demand, but also ensure that the candidates were assessed homeless role critical factors. From some statistical analyses that we’ve done on our end, the candidates that had already been going through the assessment center had shown really good performance in this organizations training center. And then also been a noticeably massive reduction in turnover as well. It was really important for us to maintain elements of this assessment center as well. By using those predetermined competencies and the same ratings and assesses, we could at least keep some of that best practice involved as well.

 

[16:33]

Slide Content: Case Study One: Entry Level Role – Mining Services

Considerations:

• Volume of candidates
• Timing: some applicants were on site and/or on night shift, and assessors were currently in full lockdown or based on site.

Solutions:

• Continued utilizing psychometric assessments (abilities & personality) though included the addition of situational judgment assessments pertaining to safety and FIFO Fit.
• Due to the above considerations, an ‘asynchronous’ assessment centre process was used.
• Developed an additional scenario-based situational judgement interview to help elicit some behaviors that may have been difficult to measure without an interactive activity.
• Interview and psychometric data was collated and discussed in an objective wash-up.

 

 

[16:33]

Georgia: Things that we also need to think about was the massive volume of candidates and also the timing. A lot of these guys, where on sites, they do night shift so finding a time where everyone was able to sort of log in at the same time and complete a lot of virtual assessments that wasn’t going to be viable for this demographic as well. There were also queries in terms of the candidate’s technological proficiency or computer proficiency as well. Again, this is where using a live streaming platform wasn’t going to be a good fit for these candidates and also given the circumstance as well with all the assessors being in locked down, a lot of the candidates working night shift. So, what we did as a part of the process, we continued to use psychometric assessments. We used the ability assessments and a personality profile. And then, we also included some situational judgment type questionnaires. These were particularly around safety and FIFO Fit that were identified as role critical factors as well. And then, what we asked is that the candidates completed these assessments prior to the next stage of the assessment center. Essentially, what this assessment center looked like is a bit of an asynchronous approach in that. There were lots of different components that were completed over several sittings in a several days, rather than all of it in one day that you might be used to from a traditional assessment center. They completed psychometric assessments first, then we sort of moved on to our interview. This was our behavioral-based interview questions that we already completed prior with our face-to-face assessment centers. And then, what we also did was develop some scenario-based interview questions.

 

[18:32]

Georgia: The benefit of using the scenario-based interview questions was because we weren’t able to replicate the practical activity. We still needed to measure things like communication and teamwork. These questions sort of presented a hypothetical scenario for candidates, where they did have to interact with others or solve a problem. And they essentially had to say, how they would go about interacting with this individual or working through these issues as well. You were able to sort of understand how they would do with obtained based activities and what their communication style is like as well. All of this information was collected once the candidates had completed these components. The psychometric data, as well as the interview data, and then all of that information was captured in the spreadsheet and was discussed over a Microsoft teams meeting with each of the Assessors particularly those Line Managers because they really enjoyed being a policy assessment center. It was always really helpful having one of those guys on board to discuss the candidate’s performance as well.

 

[19:41]

Slide Content: Case Study One: Entry Level Role – Mining Services

Results:

• Able to meet hiring requirements
• Flexible process for candidates
• Assessors were able to be involved in the discussion of the candidate’s interview and psychometric data results
• The scenario-based interview questions and additional psychometrics assessments were retained due to the value-add.

 

 

Georgia: Essentially, during this time during COVID, this organization was still able to meet hiring requirements whilst presenting quite a flexible assessment center option for candidates as well. Also, the assessors were still able to be involved in the process, which was really important for them. In terms of those additional components that we added to the online assessment center, in terms of the additional psychometric assessments and scenario-based interview questions, these are still used in this organization’s face-to-face assessments to this site as well because they feel the value in using those measures. We’ve also replicated this sort of virtual asynchronous approach to this organization’s apprentices during restrictions. We’re also going to be looking to implement it for overseas graduate roles as well, whether it’s that massive time difference. And we need to provide that more flexible asynchronous approach as well. I’m going to pass you over to Courtenay now and she’s going to talk about the next case study.

 

[20:50]

Slide Content: Case Study Two: Trainee/Student Role – Health Services

This health services organization uses assessment centres as part of an annual recruitment drive.

Over 1000 applicants annually, around 1/3 attend assessment centres.

COVID challenges:

• Needed to meet annual recruitment requirements
• Able to run in-person ACs in Perth, but interstate applicants who had made through to this stage were unable to travel

 

We ran a combination of in-person ACs for local applicants (with extra social distancing & hygiene measures), plus Virtual ACs for interstate applicants on the last two days.

 

 

[20:50]

Courtenay: Thanks, Georgia. This next case I study is with our health services organization. We’ve been running assessment centers with them for a couple of years now. They have a big annual recruitment drive for a student role that attracts over a thousand applicants each year. And they use assessment centers as part of that recruitment process to whittle down the applicants. As we’re coming into all the COVID restrictions and locked down, we ended up postponing the assessment centers. Initially, there were some concerns we wouldn’t be able to run them at all. And then luckily as locked down opened up in WA and restrictions started easing, we were actually able to run in-person assessment centers. The concern because they need to make these annual recruitment requirements to meet operational needs. We definitely had to still assess these candidates in some manner. Luckily, we were able to run in-person assessment centers in Perth, but obviously there was still the border closures. And now we’re interstate applicants who had made it through to this stage of recruitment, but they were unable to travel to Perth as I went to the previous years for in-person assessment centers, and they had to be assessed at this stage because they made it so far already in the process. What we ended up doing was actually doing a bit of a combined model where we ran in-person assessment centers for the local applicants, obviously with extra safety measures. And then, we created virtual assessment centers for those interstate applicants on the last couple of days of assessment centers.

 

[22:25]

Slide Content: Case Study Two: Trainee/Student Role – Health Services

Considerations:

• Session structure, activities, and assessment process had to be the same for in-person and VAC
• Technological requirements and issues

Solutions:

• Group practical activity converted to discussion activity. Also conducted two one-on-one activities
• Kept the same day plan and activities
• Used Microsoft Teams – different meeting links for each activity
• Assessors and facilitators ran practice sessions ahead of time
• All assessors in the same location
• Built extra time into the session to account for tech issues and internet dropping out
• Candidate instructions for activities hosted on Dropbox, with links sent at the start of activities

 

 

Courtenay: We had to make sure because we were doing this combined model that the session structure, activities, process assesses everything had to be as similar or the same between the in-person and virtual assessment center, so that we didn’t end up adversely impacting or disadvantaging those interstate applicants in any way. And as with the other assessment centers, we had to be aware of all the technological requirements and issues that could come up. In terms of some of the things that we did, we usually do quite a hands-on practical activity for this assessment center. So, we converted it to a discussion activity for this assessment center this year. That also helped with social distancing and hygiene requirements because candidates weren’t handling the same materials in person, and it allowed us to use a format that could be done virtually. Candidates also participated in different one-on-one activities with assessors as well. We were able to keep identical day plans and activities between virtual and in-person. And we had all of our assessors in the same location. We actually ran the sessions, virtual sessions via Microsoft teams. That way this actually looked in-person was all of the candidates and assessors logged into one meeting link on teams for a briefing and an introduction session at the start of the day. They were introduced to the different assessors and given a rundown of what would happen. Then there was sent different meeting links for each activity they will be participating in. Each candidate and assessor within their individual day plan with all the different activities, what times and the link that they needed to click on to. After the briefing, half of the candidate, eight in total, so four were split into one meeting link for a group discussion activity and the other four went into the individual activities. And then I swapped over because we had all the assessors in the same location. Once we finished up at the end of the session, we were able to run an in-person wash-up session where we actually discussed each candidate performance one at a time, but all in the same location. At the end of all of the activities and assessments, we had all of the candidates log in to the same meeting link again for a Q&A session. We try to create that bit of informal discussion, a chance for the candidates to talk to assessors and ask any questions that they might have, because we know from experience, that’s such a valuable part of the process for them. Any instructions for activities, we hosted on Dropbox and sent links through teams at the start of each activity as well. We built in extra time to the sessions to account for tech issues.

 

 

[25:10]

Slide Content: Case Study Two: Trainee/Student Role – Health Services

Results:

• Ran five virtual sessions over three days
• Able to overcome tech issues during activities so that all candidates could participate (activities/timer paused during drop-outs to avoid disadvantaging candidates)
• Able to observe and assess same behaviors over MS Teams that we had seen in person in the group and individual activities
• Interaction/communication between candidates in the virtual group activity were more natural than expected and easy to assess

 

 

[25:10]

Courtenay: We ended up running five virtual sessions over two and a half or three days. Obviously, we did have tech issues, but we let everybody know ahead of time that if anything happened, we pause time as pause activities, get people back online, get things happening to avoid disadvantaging anybody. We actually found that it was quite easy to observe and assess the same behaviors over Microsoft teams that we had seen in person in both the group and individual activities. And the interactions and communication between the candidates in the virtual group activity was quite natural. We actually had the last COVID week ended up getting themselves, giving each other virtual high five, which was quite funny for us to say, but it just showed sort of how natural that interaction ended up being.

 

 

[25:59]

Slide Content: Case Study Two – part 2: Senior role – Health Services

• Following the success of the student VACs, this organization use the same process for internal recruitment into a senior role
• Experienced applicants, applying for pool positions (annual pool)
• Applicants based remotely across WA – hard to get them all to Perth at the same time
• Conducted VAC over Teams – group activity plus two individual activities
• Small sessions (4 candidates/per session – 1 stream)
• All assessors in same location in Perth
• Candidates instructed to test meeting links and internet ahead of time
• Format was more convenient for candidates (able to participate without travelling) and easier for recruiters to coordinate

 

[25:59]

Courtenay: And this format was so successful that for this particular organization, that they ended up using it a couple of months later for another recruitment process. They had a senior position in the organization that they have a pool for each year. But the applicants for this pool are based remotely all across WA and it was hard to get them to Perth at the same time. We still had restrictions happening. We ended up doing the same thing, conducting a virtual assessment center over Microsoft teams with the same group or different group activity with the same format, we group activity and individual activities. We had smaller sessions, which actually made it much easier to manage. Just four candidates per session. And again, all the assessors in the same location as well with the same format sending out meeting and Dropbox links to candidates ahead of time. We actually found the format was quite convenient for the candidates because they were able to participate without having to travel to Perth. Without having to make the time for someone that would have been a good half day drive just to get in, they would’ve lost a couple of days getting to, and from assessment centers and participating. This way they’re able to chill in from home or from work. It was easier for them as well for the recruiters to coordinate that process. We’ll head back to Georgia for the last one.

 

[27:21]

Slide Content: Case Study Three: Multidisciplinary Graduate Roles- Mining & Chemical Manufacturing

2020 saw an opportunity for this company, with an expansion of Graduate vacancies forecasts for 2021, as well as the introduction of a new Graduate training program.

COVID challenges:

• Needed to meet Graduate Program 2021 vacancies
• With sites and locations across Australia, the border restrictions presented a unique challenge.

 

Two assessment centres were run, one face-to-face for western operations, and one online for eastern operations. For this, activities and an assessment processes was designed that could elicit behavioural competencies consistently across both an online and face-to-face setting.

 

Georgia: Alright. Our very last case study is online assessment center that we ran for a mining and chemical manufacturing organization for multi-disciplinary graduate roles. I think this organization, they had a year of growth this year and also introduced a new graduate training program. It was really important for them to be able to roll out these new programs that we’re really excited about. To roll out a program, you need graduates, and you need someone that’s going to be a good fit for the organization. Essentially, we need to fill those vacancies whilst, for local candidates so those based in Perth, we were able to run a face-to-face assessment center. But this organization had sites and locations across Australia as well. Therefore, their candidate pool was Australia-wide as well. And with border restrictions, we weren’t sort of host an assessment center in the state. What we did was also do an online virtual assessment center, as well as a face-to-face assessment center. For Western operations, we ran one face-to-face assessment center for the day, looking at multiple mostly on graduate roles and then, we ran a virtual assessment center. We really needed to ensure similar lead to the case study that Courtenay just spoke about. We need to make sure that the activities were able to be consistently run face-to-face and online as well. What the activities were mainly two group discussion activities. As I mentioned before, group discussion activities, they’re really great for graduate roles, particularly, a lot of organizations look at things like critical thinking, problem solving, communication, teamwork, which can be elicited from those group discussion activities. These guys also completed psychometric assessments prior to the assessment centers, which were able to be done online. They also had a behavioral interview question as well. We really need to make sure that the activities weren’t technical based because there was lots of different sort of disciplines coming into the assessment center. You don’t want to disadvantage one discipline over the other, particularly, and instead just looking at those behaviors. Those things around sort of problem-solving, teamwork, communication, and that type of thing.

 

 

[30:00]

Slide Content: Case Study Three: Multidisciplinary Graduate Roles – Mining &Chemical Manufacturing

Considerations:

• Session structure, activities, and assessment process had to be the same for in-person and VAC
• Multidisciplinary Graduate roles-activities were non-technical
• Technological requirements and issues

Solutions:

• Two group discussion activities were developed that are appropriate for a graduate level.
• The same day plan and activities were used, however, a smaller number of candidates attended the online assessment centre, to allow and easier flow.
• Used Zoom- one stream with breakout rooms for interviews
• Assessors were trained ahead of time on technology an online expectations.
• Built extra time into the session to account for tech issues and breaks
• Candidate instructions for activities hosted on Dropbox
• Assessors turned off microphones and cameras for group activities

 

[30:00]

Georgia: Again, similarly to Courtenay, we also needed to consider things like technical requirements and issues. Because these guys were graduates, they’re quite familiar with online conferencing systems and platforms. A lot of the university degree sort of towards the end had to be online webinars and things like that as well. These guys had a pretty good understanding of technology and platforms probably more so than some of us as well. Essentially, like I mentioned before, our solutions were having one face-to-face session, one online session with activities that are able to be replicated across the settings as well. To assess the training, we had one face-to-face for the face-to-face assessment center, but then we ensured that we had a specific online assessment training for those that were going to be assessing during the online assessment center. This is because of assessing and training them on the technology that they were going to be using, but also explaining to assesses and training them on how to look for behaviors online.

 

[31:05]

Georgia: Obviously, you can only see someone’s sort of front half of their face. You can’t see full body language. Really sort of identifying to assess is what you’re going to be looking for and having those different expectations face-to-face as you are online. We replicated the same day plan face-to-face as the online assessment center. However, similarly to Courtenay that we reduced the number of candidates. We had four candidates per session, and this made it much of an easier flow for the assessors, but also for the candidates as well. Because we had a smaller number of candidates, they felt a lot more comfortable asking the assessment questions and having a bit more of an informal chat with them. A lot of the time when you’re in webinars and you see lots of different faces pop up, you’re less likely to sort of ask a question because you feel like someone else might ask the same question or you might be more inclined to sort of take a step back and allow other people to take the lead as well. I think we’ve both noticed a benefit in having smaller number of candidates in an online assessment center as well. For this one, we used Zoom as an online platform and this worked super well, super easy. You send out one meeting link to the candidate that’s used for the introduction presentation for the group activities. During the group activities, assessors were just asked to turn off their cameras and microphones so that the candidates could just focus on each other as well during that time. And then for the interview, I was facilitating the day and I popped everyone into breakout rooms with their assessors and for the interviews because there was one or two technical questions as well. We also would have an HR Representative and a Line Manager in those interviews. We also ensure that we’ve just built in a little bit of extra time to allow for technological issues and breaks. You can’t forget breaks even when you’re online. People still need to have a quick water break. Similarly, to the previous one, the instructions for activities were hosted on Dropbox. What Courtenay and I did is we use Dropbox for business, which allowed us to restrict the access to those files as well, just to make sure candidates were able to download them.

 

 

[33:37]

Slide Content: Case Study Three: Multidisciplinary Graduate Roles – Mining &Chemical Manufacturing

Results:

• Ran a full day face-to-face and full day virtual assessment centre
• Assessors and candidates were able to join the virtual assessment entre from all around Australia
• Able to observe and assess same behaviours over Zoom that we had seen in person in the group activities.
• Interactions/communication between candidates made the session more personal as each candidate was able to ask questions of the assessors during the Q&A session rather than seeming like a webinar!

 

[33:37]

Georgia: Essentially, the results we were able to successfully run assessment centers face-to-face and virtual. The beauty of the virtual assessment center was that the candidates were from across Australia and the assessors were from across Australia as well. Some of them had never met each other within this organization as well. It’s kind of a really nice, I don’t know, humbling way to sort of connect with people from across WA in Australia as well and giving them the same opportunity in those face-to-face assessment centers. The interactions and communications with candidates and assessors were actually really relaxed. I think that’s where it helps to have a bit of an ice breaker at the start, a bit of an initial presentation, Q&A session and also keeping those groups to quite small. You don’t want to be assessing up to 30 candidates in a session because then it won’t be that relaxed environment as well. I’m going to pass you over to Courtenay. Here’s a photo of her in action at the virtual assessment center. She’s going to now talk through the lessons learned.

 

[34:57]

Slide Content: Part three: Lessons Learned

Lessons Learned: Challenges and Limitations

• Technological challenges (internet dropping out, video/sound lagging during activities, trouble loading instruction documents)
• Candidates need access to appropriate technology (laptop, good internet)
• Potential for disruptions (other people at home, etc.)
• No opportunities for informal interactions between assessors and candidates
• Limited in the types of activities you can run
• Not appropriate for all types of roles or candidates

 

[34:57]

Courtenay: Obviously, there were going to be some challenges and limitations with running assessments and this format. So, tech issues are the biggest one and most obvious internet dropping out, lagging and sound and video during group activities or trouble loading documents, bandwidth issues. We really had to keep this front of mind, just be aware to build in that extra time. Allow for people to have those issues and to overcome them and to have some solutions on hand as well. It’s really important that candidates have access to the appropriate technology. Both the laptop, good internet, all of those sorts of things which was why we did the asynchronous assessment centers for the first one, because they weren’t going to have that necessarily. There is a potential for disruption of having other people at home. Luckily neither of us experienced that with any of our candidates, but that potential is there. There’s not the same opportunity for informal interactions between assessing candidates like you would have when you’re all in person in the same building. We tried to replicate with the intros and Q&A sessions, but it’s not going to be the same as in person. And that’s something that we know from feedback that candidates really love from in-person assessment centers. You are limited in the types of activities you can run. As Georgia said, it’s hard to build a Lego tower over Zoom. And they sort of assessment centers are not going to be appropriate for all types of roles or candidates.

 

[36:21]

Slide Content: Lessons learned: Benefits of Virtual Assessment Centres

• Easy for candidates to be punctual
• Time efficient
• More accessible for interstate or remote candidates (access to a wider candidate pool)
• Candidates can participate from the comfort of home
• Format can be tailored to accommodate different types of candidates
• Allowed organizations to meet hiring targets during COVID

 

[36:21]

Courtenay: But obviously, there was some real benefits of doing this as well. Because candidates could participate from the comfort of their own home, it was easy for them to be punctual for this assessment center. They didn’t need to fight traffic or find parking. We actually found them, even though we do extra time for tech issues to be very time efficient to run. It was more accessible for interstate or remote candidates to participate, which actually gave our clients access to a much wider candidate pool across Australia than they would have had otherwise. And running assessment centers in this virtual format does allow you to really tailor them to accommodate different types of candidates and roles as well. And the biggest benefit was that it allowed organizations to make these hiring targets during COVID.

 

 

[37:09]

Slide Content: Lessons learned: Recommendations for future Virtual Assessment Centres

Maintaining best practice:

• Competencies
• Exercises
• Ratings
• Scheduling
• Assessor Training

VAC design

• When deciding which approach to take, consider the candidates (role level, technological proficiency, access to technology)
• Practical group activities very difficult to do well/at all online – discussion activities work better (consider literacy/ESL)
• Build in buffer time for tech issues – there will be some!!
• Build in time for breaks

 

[37:09]

Courtenay: In terms of recommendations for future assessment centers and this is purely based on our experiences. Obviously, maintaining best practice was really important. We had those competencies that have been predetermined through job analysis ahead of time, but it’s really important to do that going forward when you’re creating these from scratch as well. And then, we designed the activities and chose activities to elicit the behaviors relevant to those competencies. We had the writing sheets that we’ve used previously, and we just made sure to adapt these with new behavioral anchors to reflect the new activities we’ve developed. In terms of scheduling, we made sure that we’ve had to one assess that per candidate throughout the day so that people weren’t assessed by the same person twice and just tried to remove any potential conflicts of interest as well. And we still kept up with that assessor training including training assesses on a technology that they would be using as well as it was really important to have those training assessors.  In terms of designing virtual assessment centers, we’re really considering the candidates with the most important things or looking at their role level, how technologically proficient they were, the access to technology. Looking at what types of activities, obviously practical activities are very difficult to do online. Discussion activities work better, but then you need to consider literacy levels language. If English is second language, making sure that candidates aren’t being disadvantaged unfairly. Building in time for, buffer time for tech issues, because there always will be and building in time for breaks. And also looking, as Georgia mentioned that the size of the session. We found that smaller sessions with less candidates, one much easier to manage design wise than other sessions.

 

[38:54]

Slide Content: Lesson learned: Recommendations for future Virtual Assessment Centres

Other tips:

• Use a facilitator
• Understand that VACs are different to in-person ACs – adjust your expectations of candidates accordingly

Platforms:

• There are some specific VAC platforms available
  • Can be pricey
  • More difficult to tailor to the level we needed
• Can use video conferencing platforms (e.g. MS Teams and Zoom)
  • Breakout rooms vs separate meeting links
  • Different administrator/access permissions
  • Security restrictions around Zoom

 

[38:54]

Courtenay:  Turn to a few other tips. We found it was really beneficial to have someone who is a dedicated facilitator, just to make sure that everybody was where they needed to be, to follow up with any candidates who are having tech issues and just to manage the whole process. And also, just understand that virtual assessment centers are different to in-person assessment centers and to adjust your expectations of candidate’s. Things like you’ve got people that seem like they’re talking over the top of each other in a group activity, it could be lagging with the video or the sound. Just keeping those sorts of things in mind. In terms of platforms, we did look at different virtual assessment center, specific platforms because there are a few out there. We did find that for us though, they were a bit pricey. And also, it was quite difficult to tailor them to the level we needed. We wanted to tailor things like day plans, competencies, activities, assessor allocation and it’s quite difficult to do that to the level we needed on these platforms. However, we found that it was quite easy to use existing video conferencing platforms like Teams and Zoom to run assessment centers. There were some differences between them. In Zoom, we could send people, have one link to the assessment center and send people to breakout rooms. Whereas in Teams, it was separate meeting links to every activity. In Zoom, it was easier for us as external facilitators to have permission to run it and to access the meetings as opposed to Teams, we needed permission to enter each of the meetings ourselves as well from someone internally. However, there are organizations that do have the security restrictions that prevent them from using Zoom. And so, Teams is quite beneficial for those organization.

 

[40:37]

Slide Content: Final thoughts

• Increasing demand for virtual recruitment solutions
• COVID-19 forced us to explore and trial ways to run assessment centres in a virtual world
• Allowed our customers to meet recruitment requirements and allowed candidates to access recruitment opportunities without the need to travel
• Limited research so far around effectiveness of conducting VAC versus in-person AC (and differences in candidate experience)
• VACs are going to be part of the future of recruitment – it’s important that we do them well

 

[40:37]

Courtenay: And just a few final thoughts before we wrap up. We’ve noticed even before this, that there was an increasing demand for virtual recruitment solutions. And this would only be increasing leading up to entering COVID and COVID has really forced us to explore ways to run assessment centers and recruit in this virtual world. It did allow our customers to make recruitment requirements and allow candidates to access recruitment opportunities without needing to travel. There’s limited research so far because this is such a new area around both the effectiveness and the candidate experience in the virtual versus in-person assessment centers. And there’s a lot more work that’s going to be needed here to help us understand what best practice in this virtual world is. We know that virtual assessment centers are going to be part of the future of recruitment. It’s important that we continue to do them well and continue looking at ways that we can keep improving them. That’s all for our presentation. We’ll have a quick look at the Q&A box and see if we have any questions. If anybody does have any questions, feel free to add them to either the Q&A or to the chat box. Let’s see what’s up and have a look.

 

Georgia: I suppose just adding on to what Courtenay said, although research still needs to be done, it’s a very new area in terms of understanding best practice. But anecdotally, so feedback that we’ve received from candidates and assessors is that it was a really good experience that I enjoyed the process. The candidates felt that it was fair. Obviously, more research needs to be done in terms of understanding, whether these candidates go on to the top performers as well. And whether it still as good a predictor of job performance as the face-to-face assessment centers. But anecdotally from our experience, it has been really beneficial rather than sort of just going for it to be alone.

 

Courtenay: We do have a question in there. “Do we normally include exercises for the mining company that switched these out for additional assessments and kept everything else this time?”

 

Georgia: Yes. If we think about the first mining organization case study, essentially yes. In our face-to-face assessment centers, we usually had a practical activity, which actually is really appropriate for those more hands-on level roles and also a group discussion activity. In terms of when we then went to the virtual assessment center, it was much too challenging for us to take up a time where all of the assessors who were in full lockdown could sort of join into the assessment center as well as candidates who were typically already working on site and on night shift as well. What we did was we removed the group activities. And what we did instead was developed some scenario-based interview questions, which help to reset those competencies that may be better measuring those group activities. Like I mentioned, things like teamwork, how would they respond in a situation where someone’s done something? We also maintained the psychometric assessments and included this scenario-based interview, sorry, scenario-based psychometric assessments as well. Things we have a FIFO Fit assessment and a safety assessment test. We were able to understand the candidates’ behaviors, and knowledge in those areas as well. We kept the behavioral interview as well. Those interviews can either be done live or pre-recorded as well. Typically, we recommend running interviews live where you can, because then you’re able to prompt and probe the candidate and build up that sort of more personal organization brand.

 

Georgia: However, obviously due to timings, people being on site, sometimes it can be too much of a challenge to run those live interviews. If you do have a pre-recorded interview, just make it really clear to the candidates that a person will be assessing those responses rather than sort of computer. It wouldn’t be scored by a robot or anything like that. That way it brings in more of that personal touch, personal organization, brand and trust. If it is those prerecorded interviews, ensure that you have those prompting and probing questions as well, just to really sort of contribute the same amount of detail as each other as well. I hope that answers your question. Anything else, did anyone have any other questions?

 

Courtenay: Otherwise, feel free to get in touch with us, send us an email or get in touch on LinkedIn, if you’ve got any other questions or want to ask anything else about our experiences running back to the assessment centers and we’re happy to have a chat and answer those later on.

 

Georgia: Yes. As I mentioned at the start, we’ll send you all our recording of this presentation, as well as the slides. Feel free to get in contact with us, whether it’s over LinkedIn. We’ll get that office call. If you do have any further questions at all, we’re also going to be sending through a bit of a feedback form or a survey, like I mentioned before, this is our first webinar. We’d love to hear how you found it. Is it the appropriate amount of detail for you and was it helpful? And also, is there anything else that you’d like to hear from us, any other topics that you’d like to get covered as well? We’d love to hear that from you as well. Cool. That seems to be it. Thank you so much for your time everyone, for taking time out of your lunch. We look forward to sort of speaking with you soon. Thanks.

 

Related blogs: Running a virtual assessment centre, our guide to effective process.