Ronan O’Loughlin talks about the issues surrounding the 2021 FAE interim assessments and the measures being taken by the Institute for future online exams. 

As many of you know, on Saturday, 24 April, FAE students experienced significant technical issues with our online exam platform Cirrus during their AAFRP interim assessment. I want to share with you what happened and what this means for you, our students. 

First, on behalf of the whole Institute team, I am truly sorry for the concern and uncertainty that these issues have caused. My team and I share your frustration, and I apologise for this unforeseen event, which is, of course, at odds with our ambition to serve students’ needs. 

What happened on the day? 

1,450 FAE students were admitted onto the exam platform in small groups. Towards the end of the onboarding window, students started to see an error message on their exam page in Cirrus. Cirrus has confirmed that the problem was related to server capacity, which performed in a way that was outside the many scenarios we had tested. This resulted in students losing connection to the assessment and experiencing difficulties and delays in reconnecting. Unfortunately, these issues prevented many people from completing the exam. In addition, we cannot be certain that the technical issues did not impact the scripts that were submitted. We began an investigation that same day under our risk management processes. At the time of writing, we are still completing the full ‘root cause analysis’ to understand why the issue occurred and prevent any future recurrence.  

As soon as we became aware of Saturday’s issues, we contacted all FAE students and advised them that the exam had been stopped. However, we know that some students were unable to access our usual one-to-one assistance on the day due to the numbers impacted by the issue. We very much regret the additional frustration that this caused for students.

Students have asked if the problems that arose during the recent AAFRP interim assessment have occurred before, and I can confirm that this was the first time this has arisen for Cirrus, a company that hosts online exams worldwide. Naturally, they share our urgency in investigating and resolving this issue. 

As you may be aware, Cirrus hosts our exams while ProctorU ‘onboards’ students and invigilates the exams. During the early days of our online examinations last year, I know that some students encountered individual connectivity and technical difficulties relating to online invigilation, which we resolved.

Since COVID-19 emerged, the Institute’s priority has been to ensure that students can continue their studies and their path to qualification, despite the pandemic. These types of transformation projects are by nature challenging, and we have greatly appreciated the collaboration of students, firms and employers in making this happen. 

Where to from here? 

The AAFRP should have accounted for 15% of the FAE Core grade, with the remaining 85% represented by August’s main Core exam. When we realised that we had to stop the AAFRP exams, we knew we needed to give students clarity quickly and provide a solution that treated all impacted FAE students equally. As you know, it was decided that we would put aside the AAFRP assessment on this occasion so that this year, the main FAE Core sitting in August will represent 100% of the marks available. The weighting of financial reporting in the Core paper will remain at 25%, in line with the competency statement. While the format of the Core exam is different from the AAFRP, I want to assure you that studies up to now will still be relevant to the main exam.

I have been asked why it is not possible to simply re-sit the assessment. There are several reasons for this. Your welfare and successful progression are the priority. We did not wish to insert another significant exam milestone in the run-up to the main FAE exams, which, as always, include the AAFRP material in their scope. In addition, we need to ensure a robust exam platform for all students ahead of the upcoming main exams beginning with CAP1 on 24 May. 

Since 24 April, we have been consulting and listening to our stakeholders. We have heard from students, both directly and with the assistance of CASSI, who undertook a valuable survey. We have also spoken to firms, employers, the UK and Irish regulators (who oversee us), and our governance structures such as Education Board and Oversight Board and exam committees. We know that trust in our online exams has been impacted, I want to share with you some of the things we are doing. 

• First, we are working closely with Cirrus to diagnose and fix the issues we encountered. We will test these extensively over the coming weeks. To allow us to focus on this fully, we have rescheduled CAP1 practice onboarding. 
• Second, we are reflecting on how we communicate with students and employers, and we are committed to providing you with timely, accessible updates.
• Third, our examining team will consider if the average student performance in financial reporting at the end of the main exam in August is in line with recent years, and a detailed analysis of recent student performance will inform this. In this way, we will seek assurance that students have not been disadvantaged due to recent issues.

Since we first introduced e-assessments in 2019, over 20,000 individual online exams have successfully taken place, making the Institute one of only a handful of chartered bodies around the world to be able to continue exams throughout the pandemic. While this does not in any way diminish the significant upheaval of the last few days, it will hopefully give you cause for confidence in our continued commitment to a successful exam season over the coming months. 

My team is here for you at any time, on FAEExam@charteredaccountants.ie. The CA Support team is also available to offer confidential one-to-one listening support, and you can find their dedicated site and contact details on our website.

Ronan O’Loughlin is Director of Education and Training at Chartered Accountants Ireland.