Audit quality
Annual report 2016/17

Who we are
The Auditor General, the Accounts Commission and Audit Scotland work together to deliver public audit in Scotland:
•	Audit Scotland is governed by a board, consisting of the Auditor General, the chair of the Accounts Commission, a non-executive board chair, and two non-executive members appointed by the Scottish Commission for Public Audit, a commission of the Scottish Parliament.
•	The Auditor General is an independent crown appointment, made on the recommendation of the Scottish Parliament, to audit the Scottish Government, NHS and other bodies and report to Parliament on their financial health and performance.
•	The Accounts Commission is an independent public body appointed by Scottish ministers to hold local government to account. The Controller of Audit is an independent post established by statute, with powers to report directly to the Commission on the audit of local government.

About us 
Our vision is to be a world-class audit organisation that improves the use of public money.
Through our work for the Auditor General and the Accounts Commission, we provide independent assurance to the people of Scotland that public money is spent properly and provides value. We aim to achieve this by:
•	carrying out relevant and timely audits of the way the public sector manages and spends money
•	reporting our findings and conclusions in public
•	identifying risks, making clear and relevant recommendations. 
Key messages
1.	Audit Scotland teams and the appointed firms have effective audit quality arrangements.
2.	Our appraisal work has shown that high-quality audits are being delivered.
3.	We are revising how we appraise and report on audit quality.
4.	We know from international comparisons with other audit agencies and institutions that we are starting from a good base.
5.	We will develop our internal information systems to support improved audit quality indicators.
6.	We are focussing our audits on the right areas.
Foreword
Quality is at the core of what we do and is the foundation for building consistency and confidence across all of our audit work. We ensure that auditors carry out high-quality audit work with robust quality arrangements that lead to improvements and build stakeholder confidence.
We manage the appointment of financial auditors on behalf of the Auditor General and the Accounts Commission. And we also make sure that the appointed auditors and Audit Scotland staff working on performance audits and Best Value audits are doing high-quality audit work.
Audit quality is more important than ever. The Scottish Commission for Public Audit (SCPA), a commission of the Scottish Parliament that approves our budget, recognises our commitment to maintaining the quality of audit in the context of the significant cost reductions that we obtained in the last audit procurement exercise. We are enhancing our quality appraisal arrangements and putting in additional resource to ensure that the competitive fees that we obtained in the procurement round will continue to deliver high-quality audits.
Although there is currently no common approach to measuring audit quality, increasing consideration is being given around the world to the quality of an audit and what audit quality means, how it could be defined and how it could be measured. We have been appraising audit quality using a variety of measures for a number of years. However we recognise that we can improve. That is why we have invested in a project to review the audit quality appraisal arrangements..
This report on audit quality has been prepared to meet the principles of the Statutory Auditors (Transparency) Instrument 2008. Although the requirements of the instrument do not apply to our work, we choose to produce our own report to comply with best practice, and provide explicit assurance to the Auditor General and the Accounts Commission on the quality of audit work commissioned and performed on their behalf.
Audit Scotland has designed and implemented effective audit quality arrangements to ensure high-quality audit work for the period 1 April 2016 to 31 March 2017. The results of our internal and external work on audit quality provide assurance that audit work carried out for the Auditor General and the Accounts Commission is high quality and also highlight areas where we may benefit from improvement.

Introduction
Our audit quality framework is based on the International Auditing and Assurance Standards Board’s (IAASB) Framework for Audit Quality. The framework provides the foundation for achieving consistency in audit quality and applies to all audit work.

In this report, we set out the inputs, the processes, and the outputs that we have in place for all our audit work. We describe in the Appendix our corporate structure, our finances, and who we audit.
Inputs
Values, Ethics and Attitudes
Independence
Auditors’ independence and freedom from influence from the bodies they audit is an essential element of high-quality audit. Scottish public sector bodies do not have the power to appoint their own auditors. The Auditor General and Accounts Commission appoint auditors to carry out the statutory financial audits of Scottish public sector bodies. These auditors are a mixture of Audit Scotland staff and some of the top UK audit firms. This independent appointment process is a unique feature of public sector audit and helps to ensure the auditors’ independence from the bodies they audit. Appointed auditors apply their professional expertise to conduct their audits and report their findings in public.
Audit Scotland staff carry out performance and Best Value audits. 
We consult widely to set our work programme, but the statutory responsibilities for the programme rest solely with the Auditor General and Accounts Commission.
Responsibilities
The Audit Scotland Board is responsible for the exercise of all functions of Audit Scotland. It exercises its powers and authorities through a Scheme of Delegation, which it determines and approves.
Audit Scotland’s management team oversees the day-to-day operations of Audit Scotland’s work, with each member taking responsibility for a specific area. Further details on the composition of the board and management team are set out in the Appendix.
Fiona Kordiak, the Director of Audit Services Group (ASG) is responsible for Audit Scotland’s audit quality arrangements for financial audits. Fiona is supported by the Professional Standards and Quality Improvement Group which is managed by a full time senior manager and led by an Assistant Director.
Each appointed firm has its own arrangements for ensuring audit quality. Their arrangements vary but are all consistent with International Standard of Quality Control 1. Further information about their audit quality arrangements can be found in their Transparency reports.
                  
Fraser McKinlay, the Director of Performance Audit and Best Value and Controller of Audit is responsible for the audit quality arrangements for performance audits and Best Value audits. Fraser is supported by the Performance Audit and Best Value Quality Group.
Russell Frith, the Assistant Auditor General is responsible for coordinating the quality appraisal of all the audit work carried out for the Auditor General and Accounts Commission. Russell is supported by the Audit Strategy team and a corporate quality group with membership from the Professional Standards and Quality Improvement Group and the Performance Audit and Best Value Quality Group.
Ethics
We have a rigorous process to ensure that our auditors comply with the Financial Reporting Council’s Ethical Standard to avoid any actual or perceived conflicts of interest. We consider potential conflicts of interest when making recommendations for appointment, and also when approving non-audit work requests by the firms.
We require all our staff to complete and submit an annual Fit and Proper form. Staff are personally responsible for declaring any potential conflicts of interests. As part of this process they must confirm they have read relevant policy documents and are up to date with their professional training. We use their responses to put in place any necessary safeguards, such as removing someone from an audit.
We carried out a procurement exercise on behalf of the Auditor General and the Accounts Commission during 2016 to select auditors to appoint to all our audits. This process included rotating auditors, primarily between firms and Audit Scotland auditors. Where the audit firm or in house provider was not rotated, the teams on the audit were rotated based on the Ethical Standard’s long association requirements.
Non-audit work
Audit Scotland staff do not conduct any non-audit work for bodies which we audit. When secondment opportunities arise, these are considered by the Ethics Partner to ensure that they do not create any conflicts of interest.
The appointed firms carry out some non-audit work for the bodies they audit. Firms explain how the work complies with the FRC’s Ethical Standard and what safeguards they plan to put in place before obtaining clearance to do the work from Audit Strategy. A small number of requests for non-audit work during 2016/17 were approved, none of which created an actual or perceived threat to the independence of the auditor.
Knowledge, Experience and Time
Our most important assets are our staff. Having the right staff, allocating the right time to audits and providing the right training are critical to delivering high-quality audit work.
We have introduced more collaborative working for Best Value audit work under the new Code of Audit Practice. All appointed auditors are now more clearly covering their Best Value responsibility through the annual audit. Auditors are working collaboratively with Audit Scotland staff on the periodic Best Value Assurance reports.
Staff survey
We conducted a staff survey this year to better understand how our staff see us as a place to work. We used three questions that are reported by all the major UK audit firms. We also include the results of the firms’ own staff surveys that they report in their Transparency reports to show the views held across all auditors involved in Scottish public sector audit. These questions provide an insight into how our own staff and those of the appointed firms view how well they are supported to provide high-quality audit work.

| Audit Scotland | Grant Thornton | Deloitte | KPMG | PWC^1

I am encouraged to perform a high-quality audit

92% positive
6% neutral
2% negative

92% positive
4% neutral
4% negative

86% positive
11% neutral
3% negative

86% positive

90% positive

The time and resources available to me enables the delivery of a high-quality audit

63% positive
17% neutral
20% negative

33% positive
15% neutral
52% negative

40% positive
23% neutral
37% negative

85% positive

57% positive

The training and development I receive enables the delivery of a quality audit

62% positive
21% neutral
17% negative

65% positive
14% neutral
21% negative

64% positive
23% neutral
13% negative

67% positive

74% positive

Note: 1. PwC reported the result as a number out of 5. We converted this to a percentage to increase comparability with other firms’ results.

The results show that there is a strong culture of support for performing high-quality audit across all auditors. The time available to auditors across the profession shows a mixed picture. Staff are broadly positive about the adequacy of training but we recognise that there is more that we can do, and this is being picked up as part of the review of audit quality arrangements.
Qualifications
We ensure that our staff are appropriately qualified. 85% of our staff working in financial audit are either CCAB or CIMA qualified or in training for a CCAB qualification. A further 12% have other accounting or IT qualifications.
Our staff working on performance audits and Best Value audits have a variety of audit and research-related qualifications, all of which go towards supporting the delivery of high-quality audit work. 22% of staff within this business group have a CCAB qualification, which supports our move to a more flexible ‘One organisation’ way of working.
Training
Audit Scotland runs one of the largest public sector accountancy training schemes in Scotland. We have 35 trainees and staff working towards CIPFA and ICAS qualifications. This year we recruited eight financial trainees and eight trainees successfully completed their ICAS training to become qualified accountants. In June 2017, the programme will be opened again to a new generation of trainees that will help shape the growing future of Audit Scotland.
Over the course of the year, Audit Scotland recorded 89 discrete learning events, totalling almost 1,367 training places at a cost of £267k. This represents an average spend of £972 per employee. The average number of hours of recorded training per person is 34 hours, although this does not include other training activity undertaken by staff through their work or in their own time. We will be working over the coming year to obtain more comprehensive training data.
Audit Scotland’s events included our annual audit conference (200 attendees), technical training workshops on Capital Accounting, Management Commentary, and specialist areas such as NHS audits and pension fund audits. We also delivered personal development skills training including Thinking on your Feet and Client Relationship Management. The appointed firms each host a variety of training events for their staff including annual mandatory technical training, subject specific training and webinars.
We provide additional training opportunities through a programme of knowledge cafes. In the last year we have hosted sessions on a wide variety of topics from Government Expenditure and Revenue Scotland statistics to TED talk sessions on the Surprising Habits of Original Thinkers by Adam Grant.
We are working to increase collaborative training to maximise the offering to all staff involved in Scottish public sector audit. We invite staff from appointed firms to attend training that our technical experts deliver, and cooperatively deliver some training with experts from appointed firms.
We make sure that we put staff on audits with the right skills. Our financial auditors have experience and training in sectors and specialist areas, eg local government pension funds, registered charities, and the NHS. Our performance and Best Value auditors often have experience of working in particular areas or specific research skills. We provide internal training for staff who are new to a sector/specialism. We support all auditors involved in public sector audit with our experienced technical advisers in Audit Strategy. Audit Strategy produces accounting and auditing guidance, and provides responses to technical accounting and auditing queries.
Process
Audit process and audit quality arrangements
Organisation-wide audit quality arrangements
All parts of Audit Scotland and the appointed firms have distinct responsibilities for the main strands of audit quality.
Our Corporate Plan: 2017/18 update confirms our commitment to operate to the highest ethical and professional standards and our aim to maximise the value of the audit and support improvement. This commitment is central to our Quality Framework, which sets out the following principles:
•	Quality is the responsibility of everyone in Audit Scotland.
•	Our approach to quality will drive continuous improvement across Audit Scotland.
•	Quality will be assessed in terms of outputs, outcomes and impact, as well as inputs and processes.
We are reviewing our quality arrangements.
The Audit Strategy Group is independent of auditors. It is responsible for overseeing audit quality arrangements of all audit work and providing assurance to the Auditor General for Scotland and the Accounts Commission in accordance with the framework approved by the board.
Auditors are responsible for establishing the arrangements for their activities, including the processes for getting it right first time, and reviews of audit work. Relevant documents are available to our staff and also to the firms.
Our audit quality arrangements and those of the firms are focused on making continuous improvements to the way we do audit work by understanding why errors are made and sharing good practice. We regularly update our methodologies, and provide learning and development opportunities to our staff.
There are many ways to describe the arrangements to ensure audit quality. They all share a small number of critical features shown in the diagram below. We describe the arrangements that we have in place for financial audit work and for performance audits and Best Value audits under each feature.
Financial audit
All financial audits, whether undertaken by Audit Scotland staff or appointed firms share certain common features to ensure audit quality. They are designed to comply with International Standards on Auditing (ISA) and the International Standard on Quality Control (ISQC1), and the Code of Audit Practice 2016 jointly approved by the Auditor General and Accounts Commission.
A new Code of Audit Practice 2016, published in 2016, applies from the 2016/17 audits. This report covers work on the 2015/16 audits, so the 2011 Code applies to that work. The new Code raises the expectations on auditors, in particular the wider scope of public audit. This is most notable in the requirements to provide conclusions on the audit dimensions (financial sustainability, financial management, value for money, and governance and transparency). The wider reporting that comes with Scottish public sector audit differentiates our work from private sector audit, and requires greater judgement from the auditor. It also puts more useful information in the public domain about the audit process and about performance of the bodies being audited.
Manuals and methodologies
All staff working on financial audits must comply with their manuals and methodologies and record all work on their documentation system. Manuals and methodologies set out how to conduct the audit to ensure high compliance with auditing standards. Manuals, methodologies and documentation systems are regularly reviewed and updated to ensure they reflect changes in standards and continue to meet business requirements.
Review of work
More experienced members of the audit team review all the audit work of less experienced staff. This ensures that the work achieves its objectives, judgements are appropriate, and work complies with the audit methodology and auditing standards. The appointed auditor then supplements this with a further review of key areas of the audit.
Hot reviews
Auditors apply a variety of review approaches based on different criteria to review audit work before the audit opinion is signed. Overall judgements and conclusions, and the wording of the independent auditor’s report are reviewed on a proportion of audits based on audit size, risk or likelihood of a modified opinion. Auditors’ approaches to specific issues are reviewed across the population where a specific matter is considered to be high risk or to ensure that training has been successfully reflected in audit work.
Cold reviews
Senior and appropriately experienced colleagues that have not been involved in the audits carry out a sample of reviews after audits have reported. These differ from hot reviews in that they cannot change the work in the event of an error. Cold reviews go into greater depth than hot reviews. The samples reviewed ensure that all auditors that sign an opinion are reviewed at least once in a three-year cycle and cover all sectors and sizes of audit. The results are reported back to the teams that conducted the audit. The results also help to identify wider training needs and where audit manuals and methodologies can be improved.
The independence of the reviews is strengthened within Audit Scotland by having reciprocal arrangements with other members of the UK public audit forum and by firms using staff from other UK offices. We have supplemented this with a programme of external review of Audit Scotland audits by the Institute of Chartered Accountants in Scotland (ICAS).
The appointed firms are subject to external cold reviews through regulation by their accountancy institutes. For the larger firms, this involves the Financial Reporting Council (FRC) conducting a sample of reviews which they report publicly. For the smaller firms this involves ICAS conducting periodic reviews.
Both hot and cold reviews provide an opportunity for auditors to identify why errors are made. Auditors carry out root cause analysis to better understand what led auditors to make errors. This helps to identify what changes will best be able to prevent any recurrence of the errors.
Developments in audit quality in Audit Scotland
We put in place new arrangements for Audit Scotland teams for relying on auditors’ work where bodies used shared services for 2015/16 audits. This meant that we were prepared for the introduction of the first Integrated Joint Board (IJB) audits and also addressed one of the findings from previous quality reviews. In IJBs, the council and health board host all the IJB’s systems including general ledger, payroll and other payments systems. We further developed the arrangements ahead of the 2016/17 audits when the rest of the IJBs became operational, and will review them after the conclusion of the 2016/17 audits of IJBs.
We responded to quality findings by providing mandatory focused training on a key audit planning tool before the 2016/17 audits started. We plan to carry out a hot review of the effectiveness of that supplementary training in 2017. Our one day annual audit conference is not exclusively focused on technical training. We are now providing more technical training in smaller groups so that we can use case studies and increase opportunities for questions and answers.
We continue to use the results of quality reviews to influence the programme of training for new starts and all financial audit staff. We recently commissioned ICAS to provide focused training on common areas of ISA non-compliance to provide staff with up-to-date knowledge on current issues.
Performance audits & Best Value audits
We managed performance audits, Best Value audits, Community Planning Partnership audits, statutory reports (S102 or S22), sector overview reports, and reports in the ‘How councils work’ series during 2016/17 under a common set of audit quality arrangements.
We carry out performance audit and Best Value audits for the Auditor General, the Accounts Commission and the Controller of Audit. All are involved at the planning and reporting stages of the work to provide input to ensure that the audits are high quality and meet their needs.
An important dimension of audit quality is the relevance and timeliness of any audit work. Focusing on the right issues, on the right organisations ensures that we add value and support our World Class Audit ambitions of making a difference through our work. The introduction of our new strategic approach to programme development is an important component of taking forward this agenda.
Our five-year rolling work programme is based on our analysis and understanding of the key risks facing the public sector in Scotland. Our strategic approach to programme development helps us to be flexible, so that we can respond to any emerging or changing risks and adapt the programme accordingly. During 2016/17, we strengthened our approach by engaging with a wider group of stakeholders on an on-going basis about issues that they consider should feature in the programme and how they can inform specific audits.
Our stakeholders (audited bodies, national bodies such as COSLA and SOLACE, the Scottish Parliament) have told us that our programme development activity is delivering a work programme that is focusing on relevant and important issues of strategic concern across the public sector.
Manuals and methodologies
All Audit Scotland staff use the Audit Management Framework (AMF) to carry out performance audit and Best Value audit work. The AMF covers each key stage of an audit from project selection, audit design, fieldwork and reporting through to post-audit review and impact assessment. We supplement the AMF with our performance audit manual and performance audit standards that comply with INTOSAI standards.
Review of work
The manager of the audit reviews all the work of the team. The Assistant Director then carries out a quality review before it is further reviewed by the Controller of Audit/Director of Performance Audit and Best Value and the Auditor General for Scotland and/or the Accounts Commission.
Hot reviews
We review all significant pieces of audit work (performance audits, overview reports, Best Value audits) using experienced colleagues who have not been involved in the audit before the work is completed. This gives us assurance that the judgements in the report are appropriate and are supported by appropriate audit evidence. It also contributes to shared learning and ownership of quality throughout the group. 
Cold reviews
All audit teams carry out a post-project review to identify potential improvements to the audit process for future work. These are supplemented with ‘reviews of project reviews’ to identify common themes and learning points that are then shared with all audit staff to support system-wide improvement.
We obtain additional independent assurance over the quality of our reports through reciprocal peer reviews with our UK partner audit agencies (National Audit Office, Northern Ireland Audit Office and Wales Audit Office).
Developments in audit quality
The Accounts Commission has approved a new approach to Best Value audit. We have prepared a separate Best Value audit guide that is available to all auditors involved in Best Value audit work, which will apply for reports that are published during 2017/18.
We identified last year that we needed to introduce a system of cold reviews of a sample of audits to independently assess compliance with the AMF. We piloted two cold reviews in the last year and then reviewed how the process went.
We found that the process worked well overall, but that we could make refinements to improve it. Before we use the cold review process on 2017/18 reports, we will:
•	focus the questions more tightly on compliance with the AMF
•	eliminate duplication between the hot reviews and post project reviews
•	strengthen the focus on independently assessing the range and quality of audit evidence underpinning key audit judgements.
Outputs
Audit Quality Indicators
We set out below the output measures that we used this year to appraise audit quality. Many of these are common to the indicators agreed by the Professional Reputation Group (a forum for the six largest firms: BDO, Deloitte, EY, Grant Thornton, KPMG and PwC).
We publish below the results of audit quality reviews based on our Audit Scotland work and each appointed firm’s transparency reports.
We report information on staff surveys and investment in Audit Scotland staff in the Knowledge, Experience and Time section of this report. We were not subject to any external investigations, and none of our appointed firms were subject to external investigations in relation to their public sector work.
Reporting
Timeliness
Financial audit work on 2015/16 accounts
Audit Strategy sets deadlines for auditors to ensure that their financial audits are completed in a timely manner. The deadlines vary by sector. Audit opinions and annual audit report deadlines are 30 June for health, 30 September for local government, 31 October for central government and 31 December for further education. The accounts of all bodies other than local government have to be laid in the Scottish Parliament. Legislation requires the accounts to be laid within nine months of the financial year end. This is 31 December for health and central government and 30 April for further education.
Auditors are expected to do all they can to meet the deadlines, but we recognise that in some circumstances, events beyond the control of the auditor can cause the deadlines to be missed. Auditors’ performance in meeting the deadlines for 2015/16 audits is set out below.

| Local government | Health | Central government | Further education

Audit opinion on time

106/106
100%

23/23
100%

70/76
92%

21/21
100%

Annual audit report on time

105/106
99%

22/23
96%

68/76
89%

21/21
100%

Accounts sent for laying on time

N/A

23/23
100%

76/76
100%

21/21
100%

Opinions for six central government bodies were signed after the Audit Strategy deadline. All of these were complete and laid in parliament by the statutory deadline. Three bodies were unable to schedule meetings to sign the accounts by the deadline despite the auditors being able to do so. Two bodies were insufficiently prepared at the start of the audit leading to delays throughout the audit. In one of those bodies this was exacerbated by the number of changes that were required to the annual accounts and the extended audit work that was needed to conclude the audit. The sixth body was new and experienced a number of accounting and management difficulties.
Ten annual audit reports were late. The auditors of the six bodies with late opinions issued annual audit reports after the opinion resulting in them also being late. The other two central government annual audit reports were late because of delays to responses by the bodies. The two local government and health audit reports that were late were completed and reported to the audit committees on time but finalised after the deadline.
Completion of principal financial audit outputs over time
Auditors’ performance in delivering outputs over time is shown in the graphs below. This shows a high level of compliance in delivering the audit opinion on time. The completion and delivery of annual audit reports (AAR) has been steadily climbing towards 100 per cent and auditors have successfully ensured that 100 per cent of accounts were ready to be laid in parliament on time for three of the last four years.
We brought forward the AAR deadline by a month for 2015/16 audits so that the opinion and AAR would be issued concurrently. Auditors maintained their high standard of performance despite the shortened time available to them.
Modification of audit opinions
Modified audit reports are issued where an auditor concludes that the accounts contain material misstatements, where significant expenditure has been incurred in breach of rules, or where reporting requirements have not been met.

| Local government | NHS | Central government | Further education
Modified opinion | 3 | 1 | 2 | 1
All audits | 106 | 23 | 76 | 21
Percentage unmodified opinions | 97% | 96% | 97% | 95%

The three modified local government opinions related to local authorities not achieving a statutory duty for significant trading operations to break even. The modified college opinion was because of a failure to include required information on the pensions of senior staff. The NHS and one of the central government modified opinions referred to inadequate records being kept throughout the year on property, plant and equipment. The auditor was ultimately able to conclude that the final year end balances were materially correct. The other modified central government opinion was of the European Agricultural Fund Account to reflect uncertainty about the accuracy of disclosures.
Publication of performance audit and other reports
The number of performance and other reports fluctuates over time. We have a five-year rolling work programme that covers a range of public sector bodies and services, and is framed around nine key risks which inform all of Audit Scotland’s policy work and monitoring activity. The work programme for 2017/18 includes 18 planned national reports and six Best Value Assurance Reports.
The Auditor General and Controller of Audit write statutory reports under Section 22 of the Public Finance and Accountability (Scotland) Act 2000 and Section 102 of the Local Government (Scotland) Act 1973 respectively where they identify matters that require reporting.

| 2016/17 | 2015/16 | 2014/15 | 2013/14 | 2012/13 | 2011/12
National reports | 16 | 11 | 20 | 13 | 16 | 17
Best Value | 4 | 6 | 4 | 6 | 8 | 12
S22/S102 | 7 | 9 | 6 | 3 | 4 | 5
Other | 2 | 2 | 2 | 3 | 4 | 3
Total | 26 | 28 | 32 | 25 | 32 | 37

Progress on delivery of the work programme is routinely reported to the Accounts Commission Performance Audit Committee and the Auditor General throughout the year. Any changes to planned publication dates are agreed with the Auditor General and/or the Accounts Commission study sponsors/Chair of the Accounts Commission.
Improvement feedback
Each year Audit Strategy reviews a sample of annual audit reports to ensure that they are of good quality and comply with the requirements of the Code of Audit Practice. As part of this we identify examples of good practice to share with auditors.
The review of 2015/16 annual audit reports confirmed that the quality of reporting remains high with a number of improvements. Indeed, the best reports contain many of the features required under the new, more demanding Code of Audit Practice. This provides assurance that auditors are reporting in the spirit of the Code and working together to improve the quality of reporting as expectations increase over time. Examples of good practice were identified and shared with auditors to help support continuous improvement in reporting.
Quality reviews
The following section sets out the findings from the review work that auditors have carried out over the last year.
Hot reviews
Audit Scotland teams conducted seven reviews covering 6 per cent of 2015/16 financial audits (eight reviews covering 8 per cent of 2014/15 audits) all of which were satisfactorily concluded. These hot reviews were designed to meet the scope of Engagement Quality Control Reviews (EQCR) as set out in ISQC1. Deloitte conducted reviews on all its audits. The other firms did not conduct any EQCR scope reviews, although Grant Thornton conducted quality compliance reviews on two of their 2015/16 audits looking at ten key quality metrics which change over the year.
We reviewed all 16 performance audit and four Best Value reports published in 2016/17 using colleagues not involved in the work. The reviews helped to shape the final reports and confirmed that the judgements were appropriate and supported by evidence.
Cold reviews
Internal reviews
Audit Scotland and each firm review a sample of completed financial audits to ensure that they are being delivered to the expected quality. The reviews are carried out by staff not involved in the audit. Each firm has their own method of scoring their reviews so they are not directly comparable. However, Audit Scotland has committed to using the Financial Reporting Council’s grading from the 2016/17 audits to improve comparability and provide more meaningful assurance.
Each firm ensures that each engagement lead’s work is reviewed at least every two or three years. In the case of the firms, the scoring from quality control reviews forms a part of the performance assessment of the engagement lead and through that influences their remuneration.
The number of quality control reviews has been increasing over the last five years:

Audit year | 2015/16^1 | 2014/15 | 2013/14 | 2012/13 | 2011/12
Audit Scotland | 12 | 10 | 8 | 9 | 8
Scott-Moncrieff | 1 | 2 | 1 | 2 | 1
KPMG | 1 | 1 | 1 | 1 | 1
Grant Thornton | - | 2 | - | - | 1
Deloitte | 2 | - | - | - | -
PWC | - | 1 | 1 | 1 | -
Wylie + Bisset | - | - | 2 | 1 | -
Henderson Loggie | - | - | 1 | - | 1
Total | 16 | 16 | 14 | 14 | 12
Percentage | 8%^2 | 9% | 8% | 7% | 6%

Notes: 1  PwC, Wylie + Bisset and Henderson Loggie no longer carry out audits following the 2016/17 audit tender. 2. Of those auditors carrying out reviews.

The overall conclusion was that the results of the quality monitoring activity provided assurance that audit work carried out on 2015/16 financial audits was of a good standard, supported key judgements and resulted in the correct audit opinions being issued.
The majority of Audit Scotland’s ISA breaches identified related to ISA 230 (audit documentation) and ISA 500 (audit evidence). This is not unexpected and is consistent with industry experience. The breaches identified were not of sufficient severity to affect the audit opinions.
Our Audit Scotland teams revised internal reporting on cold reviews to improve the balance of reports. The reports now highlight instances of good practice to spread useful innovations across Audit Scotland, and link any findings more closely to ISAs.
One of the 12 cold reviews on 2015/16 audits was carried out by the Northern Ireland Audit Office as part of reciprocal arrangements to increase the independence of these reviews.
Audit Scotland staff piloted cold reviews on one performance audit and one Best Value audit during 2016/17 representing 10 per cent of our principal reports. We found a strong culture of learning and improvement in the two audits, alongside a commitment within the audit teams to delivering high-quality audit work. We were able to place a high level of reliance on audit teams’ own post project reviews which had already identified areas where either the AMF had not been fully applied or where improvements could be made for future audits as part of their post-project review discussions.
The single most significant area of improvement across the two reviews was the need to improve the evidencing of some key quality review stages on working drafts of audit reports. We have now introduced a standard format to record key quality assurance approvals which is now incorporated within the PABV AMF.
External reviews
The Financial Reporting Council (FRC) conducts independent external reviews of the quality of financial audits subject to their regulatory oversight. This includes bodies listed on the main market of the London Stock Exchange but does not cover the vast majority of public sector audits. The reports of the appointed firms are available from the FRC. 
The appointed firms that are not regulated by the FRC are subject to oversight from ICAS which carries out periodic reviews.
Audit Strategy’s review of firms’ arrangements for audit quality confirms that they use common systems to ensure high-quality audit work so although the audits sampled do not include Scottish public sector audits, the results are a useful indication of the quality of the firms’ work.
The diagram shows the aggregate results for PwC, KPMG, Deloitte and Grant Thornton for audits of 2011/12 to 2014/15. These percentages compare favourably to the other firms inspected by the FRC, who only had 60 per cent of audits described as good with limited improvements and 11 per cent requiring significant improvement. The most recently published data on ICAS reviews relate to 2015/16 audits. These are not directly comparable to the FRC categories, but 70 per cent were judged as not requiring follow up and the remaining 30 per cent only requiring some follow up.
The results show that the firms we use have more favourable review findings than other regulated firms and so are among the best performing in their profession. ICAS regulated firms perform nearly as well as our own appointed auditors.
The FRC is setting a target that by 2019, no more than 10 per cent of audits will require more than limited improvements. The external reviews of public sector work do not yet use the FRC’s categories. We are in the early stages of procuring external review of Scottish public sector audits and we will identify an appropriate target for all Scottish public sector audits.
The ICAS quality monitoring team completed its third twice-yearly assessment of Audit Scotland financial audits in 2016, covering the 2014/15 audit year. The scope of their work was to:
•	inspect a sample of six completed audits to assess the quality of audit work and compliance with the ISAs and Ethical Standards
•	review progress against the findings contained within the report issued in relation to the outcomes from their second assessment of 2012/13 audits.
The ICAS review made a number of recommendations that we have now put in place. Many are about how existing audit procedures are applied and the adequacy of recording. Consistency in applying audit procedures will improve by establishing an even greater focus on staff training this year.
We received feedback from our UK partner audit agencies on two performance audit reports during 2016/17 representing a further 10 per cent of our principal reports. Both had clear structures, were well balanced, were written in accessible language and provided authoritative summaries of the key issues covered by the audits. Improvements could be made by further reducing the limited use of jargon and increasing the use of plain language, and improving the presentation of material in the reports.
Surveys
Audit Strategy conducted regular surveys asking bodies for their views on audit quality over the course of the five year appointment. The surveys were sent to two sectors each year for the first four years of the appointment period. The table shows the percentage of respondents who agreed with the question, “Did the audit deliver a high-quality service?” The years in the table represent the financial statements audit work about which respondents were asked.

Sector | 2014/15 | 2013/14 | 2012/13 | 2011/12
Local government | 95 | - | 97 | -
NHS | - | 87 | - | 100
Central government | 100 | - | 94 | -
Further education | - | 100 | - | 88

The surveys were sent to the auditors to address any concerns expressed by the audited body. Audit Strategy discussed poor survey results with the auditor to understand both sides of the relationship and to find out what the auditor proposed to do to improve relationships.
This year we took a different approach and met with a smaller sample of 11 directors of finance and audit committee chairs from all sectors. This gave us an opportunity to get richer information about bodies’ views of their auditors. We scheduled the interviews this year to be able to ask about the handover of audits following the new audit appointments. It was particularly encouraging to hear that the handover between auditors was generally smooth and that the new auditors had made a good start in developing an understanding of their new audits.
Overall, the views were positive about the quality of auditors’ work. One body expressed a more ambiguous view about the quality of the auditor’s work, but did recognise that there had been difficulties from both sides. All bodies agreed that their auditors understood their body well, and most were satisfied with the quality of financial statements work and the contribution made by external audit. 
Audit Scotland commissioned an independent survey of MSPs in 2016 to gather evidence on their levels of awareness of our work and our effectiveness as Scotland’s public sector auditor. The survey also provided some evidence on the early impact of our new parliamentary engagement strategy.
The key findings from the survey were that:
•	our standing among MSP has improved since 2013
•	this is regardless of party, their region and committee role
•	almost all MSPs interviewed are familiar with Audit Scotland (95%)
•	the vast majority think Audit Scotland is effective in its key role (86%)
•	the vast majority are favourable towards Audit Scotland (91%)
•	key committees are overwhelmingly supportive of Audit Scotland (72-100%)
•	almost all MSPs interviewed have had some form of contact with Audit Scotland in the past year (76%)
•	contact with key committees has been well managed (72-100%).

Review of audit quality arrangements
We are undertaking a review of audit quality arrangements to improve audit quality across all of our work to help us achieve our ambition of becoming world class. We aim to enhance our measurement of audit quality and provide a stronger evidence base through greater use of KPIs and application of the Financial Reporting Council's widely recognised audit file scoring system.
In reviewing and revising our existing quality framework our goal is to:
•	provide assurance to the Auditor General, the Accounts Commission and Audit Scotland Board about the quality of all types of audit work carried out on their behalf
•	support continuous improvement in the value for money of audit work.
The key deliverable within the review is to produce a revised Audit Quality Framework that:
•	is applicable to all audit work
•	provides assurance on audit quality
•	includes performance indicators for quality
•	establishes new internal and external reporting arrangements
•	leads to an enhanced annual Audit Quality Report.
Progress to date includes:
•	Commissioning a report on international comparisons of the approach to audit quality, quality monitoring and arrangements, performance indicators and transparency reporting.
•	Developing a Draft Quality Framework that will improve organisational governance arrangements and new KPIs that will be subject to wide consultation.
•	Reviewing options for securing external assurance across all audit work.
The main development areas that are in progress include:
•	Integrating quality arrangements into the new approach to the audit of Best Value.
•	Extending quality arrangements to cover the wider scope audit required by the new Code of Audit Practice.
•	Sourcing external assurance over all audit work.
•	Further developing performance indicators to measure audit quality.
•	Adopting scoring assessment criteria to facilitate greater comparison and identify areas for development and improvement.
We aim to have all the new arrangements in place for 2017/18 audits, and to apply as much as possible for 2016/17 audits.
Appendix
How we are structured
Audit Scotland is a body established by the Public Finance and Accountability (Scotland) Act 2000. We provide the Auditor General and the Accounts Commission with the services they need.
The Audit Scotland Board is responsible for all Audit Scotland’s functions. It exercises its powers and authorities through its Scheme of Delegation. The board consists of the Auditor General, the chair of the Accounts Commission, and a non-executive board chair and two non-executive members appointed by the Scottish Commission for Public Audit (SCPA), a commission of the Scottish Parliament.
The board members during 2016/17 were as follows:

Ian 
Leitch CBE
Chair of the board

Caroline Gardner
Auditor General and Accountable Officer for Audit Scotland

Ronnie 
Hinds
Acting chair of the Accounts Commission
(from March 2017)^1

Heather Logan
Independent 
non-executive member

Professor Russel Griggs OBE
Independent 
non-executive member

Note: 1. In March 2017, Douglas Sinclair retired as chair of the Accounts Commission due to ill health. Ronnie Hinds was confirmed as the acting chair of the Accounts Commission and joined the board in March 2017.

Ian Leitch CBE was an assistant chief executive at the Scottish Parliament before his retirement, and before that was in local government. Ian joined the board on 1 April 2014 and became chair in October 2015.
Caroline Gardner is the Auditor General and the Accountable Officer for Audit Scotland. She was appointed by the Crown for a single fixed-term of eight years from 1 July 2012.
Ronnie Hinds is the acting chair of the Accounts Commission from 9 March 2017. He is a former chief executive of Fife Council, and chairs the Local Government Boundary Commission for Scotland.
Heather Logan was a member of the Scottish Government's audit and risk committee and chair of the corporate services audit and risk committee. Heather joined the board in October 2015 and is currently chair of the Audit Committee.
Russel Griggs has had a number of non-executive positions in the private, public and third sector. Russel joined the board in October 2015.
Douglas Sinclair was chair of the Accounts Commission until his retirement on 8 March 2017. He was the Former Deputy Chair of the Accounts Commission. Douglas held the position of chief executive in district, regional and unitary councils in Scotland, as well as being former chief executive of COSLA.
The board has two standing committees:
The Audit Committee oversees the arrangements for internal control, risk and corporate governance, internal and external audit, the annual accounts and the Standing Orders, Financial Regulations and Scheme of Delegation.
The Remuneration and Human Resources Committee oversees the arrangements for salaries and terms and conditions of staff and other HR matters.
Our governance arrangements provide effective governance of Audit Scotland while preserving the independence of the Auditor General and the Accounts Commission in their statutory roles.
All audit judgements are made independently by the Auditor General, the Accounts Commission, the Controller of Audit or appointed auditors.
A management team oversees our day-to-day operations. The members at 31 March 2017 were as follows:

Diane McGiffen
Chief Operating Officer

Russell 
Frith
Assistant Auditor General

Fraser McKinlay 
Director of Performance Audit and Best Value and Controller of Audit

Fiona
Kordiak
Director of Audit Services

The management team also includes 
Caroline Gardner

Caroline Gardner, Auditor General for Scotland and Accountable Officer.
Diane McGiffen, Chief Operating Officer, chairs management team meetings and is responsible for our internal operations and business performance.
Russell Frith, Assistant Auditor General, is responsible for the external strategic and development work of financial audit, auditing and accounting standards, technical support and quality assurance. He acts as Audit Scotland’s Ethics Partner.
Fraser McKinlay, Director of Performance Audit and Best Value and Controller of Audit, manages a team that is responsible for carrying out performance and Best Value audits.
Fiona Kordiak, Director of Audit Services, is responsible for the Audit Scotland financial audit services we provide to the health service, central and local government. She acts as our Ethical Compliance Partner.
Our finances
We meet our costs from a combination of audit charges to public bodies and funding from the Scottish Parliament through the annual Budget Act. We set audit fees to recover the costs of local audit work and, where appropriate, a contribution to other audit and support costs. We reviewed our approach to funding and fee setting this year to ensure that it meets our business needs and is clear and understandable to audited bodies and stakeholders.
Our spending and use of resources are examined by the SCPA. This is a statutory body comprising five MSPs that reports to Parliament and appoints external auditors for Audit Scotland.
Income and expenditure 2016/17 (£ millions)

Scottish Parliament: £7.4 million

Net finance expenditure: -£0.6 million
Operating income: £18.6 million

Fees and expenses paid to external firms: £5.0 million

People costs: £16.3 million

Other operating expenditure: £4.1 million

2016/17 revenue resources : £25.4 million

Who we audit
Audit Scotland is a statutory body set up in April 2000 under the Public Finance and Accountability (Scotland) Act 2000. We help the Auditor General for Scotland and the Accounts Commission check that organisations spending public money use it properly, efficiently and effectively. We do this by carrying out financial, performance and Best Value audits across the public sector in Scotland. In carrying out this work we seek to be transparent in our governance and operations and to produce consistent, high-quality audits.
Public audit in Scotland covered 226 public organisations in 2015/16 and includes the Scottish Government, the NHS in Scotland and local authorities. These organisations spend over £40 billion of public money every year. The Auditor General audits or appoints the external auditor of most public bodies in Scotland, except for local authorities and integration joint boards where the Accounts Commission is responsible for securing audits.
76: Central government bodies and Scottish Parliament (including police, fire, Scottish Water)

23: NHS bodies

32: Councils

74: Joint boards and committees (including 30 health integration boards)

21: Further education colleges

226: Public bodies audited