Global Model Practice Survey 2011

Global Model Practice Survey 2011 Growth through Model Control

Financial Services Industry 2011

Contents

Foreword 4 Executive Summary 5 Introduction 7 Section 1. Respondents 10 Section 2. Model Governance 12 Section 3. Model Development 15 Section 4. Model Validation 20 Section 5. Vendor Models 27 Section 6. Internal Audit 29 Section 7. The Supervisor 31 Section 8. Regulatory Highlights 33 Conclusion 35 Appendix — Key Criteria for Sound Model Practice 37

Growth through Model Control Global Model Practice Survey 2011

3

Foreword

Dear colleague, We are proud to present the results of the second edition of the Global Model Practice Survey (GMPS). The survey was conducted during the second half of 2011 and focused on models in the area of risk, pricing and valuation used by institutions in the financial services industry. We would like to take this opportunity to thank all survey respondents for their participation in the GMPS. This year’s edition focused on the model life cycle and its practitioners.

Models are at the heart of institutions within the financial services industry. They can be powerful tools in assessing risk and improving decision making. The model life cycle aims to control and reduce model risk. A definition of model risk that is often applied is that it reflects the risk that a model insufficiently approaches reality. Examples of distinct stages in the model cycle are model development, model validation and model implementation. Sound model practice constitutes adequate execution of each separate stage of the cycle. We believe that the cycle requires a holistic design that is embedded in the roots of the organisation.

Since the financial crisis, a wave of regulatory reforms has forced financial institutions to focus on improving the control of their model risk. Banks today are still facing challenges in implementing Basel II, with the implementation of Basel III already imminent. For investment managers, UCITS IV became effective in July 2011, while European insurers will have to comply with the Solvency II framework as of January 2013. These regulatory frameworks require certain models to be independently validated, which may be new to a number of institutions. In previous years, the expectations and scrutiny of regulators have driven institutions to increase accountability and their efforts in providing transparency to their stakeholders. The enhancement of the model governance framework as well as the improvement of data quality is expected to remain on the agenda of many institutions in the coming years.

Based on the survey results our team of modelling experts has designed a comprehensive set of key criteria relevant for sound model practice. For your convenience, we have converted these criteria into a leading practice checklist.

We believe that the survey results presented in this report and the checklist will provide you with useful information on the state of model practice in financial institutions and expectations for the future.

Yours sincerely, Twan Kilkens Managing Partner Deloitte Financial Risk Management The Netherlands

4

Executive summary

Models are at the heart of institutions within the financial services industry. They can be powerful tools in assessing risk and improving decision making. The financial turmoil of recent years has had a significant impact on model practice within financial institutions.

The financial crisis taught financial institutions that simply relying on the outcome of models is a risk itself.

Therefore, understanding the risks, their limitations and the economic environment is critical when using models. The importance of a well maintained model governance framework and the transparent accountability of roles and responsibilities are emphasised by regulators and supervisors in all regions around the world. Among others, the BCBS, the OCC and EIOPA1 have given recommendations on the development, validation, implementation and use of models. Additionally, regular validation of models plays a critical role in mitigating model risk.

Deloitte’s Global Model Practice Survey (GMPS) provides insight into the various aspects of model practice which are covered by financial institutions. The survey results also reveal to what degree attention is dedicated to current on-going trends. The survey was conducted during the second half of 2011 and had the participation of 89 financial institutions from all around the world.

The key findings are listed below. Potential solutions are included in section Conclusion.

Rejecting models is challenging for model validation The survey results show that the main challenges for model validation departments are retrieving data from the source systems, planning and resourcing (i.e., recruitment and retention). Remarkably, rejecting models is also a challenge for model validators. This finding is supported by the observation that 14 per cent of the model validation departments have never rejected a regulatory model.

Model practice policies could be improved There are possibilities to improve the quality of policies that are part of the model governance framework.

1

The Basel Committee on Banking Supervision (BCBS), the Office

of the Comptroller of the Currency (OCC) and European Insurance

Up to half of the respondents indicate that one or more of the identified policies do not exist or are of poor quality. The largest improvements can be made in the areas of expert judgment, model implementation and data quality policies. Model practice policies are not always given the appropriate level of priority. We believe that concise and clear policies that have arisen through support of the stakeholders may lead to consistent models, created through an approach that meets the pre-defined internal leading practice.

The ownership of the model governance framework is not always safeguarded The ownership of the model governance framework is not safeguarded by all financial institutions in an unambiguous way: 20 per cent of respondents indicate that multiple committees have ownership of the framework. We think that assigning the responsibility of the governance framework to multiple stakeholders may lead to not making the best decisions. Furthermore, 20 per cent of respondents indicate that ownership is assigned to either the model development department or the model validation department. While these departments have strong knowledge of the technical aspects of the models, we believe assigning ownership to a stakeholder that is part of the process may not lead to a transparent model governance framework, since the responsibilities of these stakeholders are described in the framework itself.

The model governance framework is considered to be complex Most survey respondents experience their model governance framework as being complex. Clear oversight of all models within the organisation, including their status, need to be obtained. Installing a model inventory may provide a solution to gain a more detailed insight in the complexity of the model environment.

Modellers spend much time on data collection and preparation Professionals involved with the development of models spend a relatively significant amount of time on retrieving, preparing and cleansing data. We believe spending much time on data may be an undesirable situation as model developers have the largest added value when they can apply their skills and knowledge to the actual modelling.

and Occupational Pensions Authority (EIOPA) Growth through Model Control Global Model Practice Survey 2011

5

New model requirements imposed by regulators Recently, regulatory authorities have been encouraging financial institutions to improve the governance of risk related models and to give model validation a more prominent role in the risk management function. A regular cycle of model validation is required. Therefore it could be desirable for financial institutions to more thoroughly evaluate the performance and stability of their risk related models, and verify compliance with regulatory requirements. Regulatory authorities also prescribe an active involvement of senior management, ensuring systems operate properly and resources are available.

Activities in order to enhance model practice are expected to increase In the coming years the activities for the model development department and the model validation department are expected to increase. Moreover, regulatory authorities are expected to initiate new and more stringent requirements on model performance.

6

Given that financial institutions already face capacity constraints, it may be a challenge in the coming years to complete required validation processes and model improvements on time, let alone research and innovate new methodologies.

Investment managers face more stringent regulation The regulatory landscape for investment managers faces a similar degree of reform as banks and insurers.

In Europe, becoming compliant to UCITS IV will be an important focus for the coming years. Although the exact implications of the new regulations are not clear yet, it is evident that it will have a considerable impact on investment management model practice as governance and accountability will have to be improved and models will have to be validated on a regular basis.

Investment managers will have to catch up with banks and insurers in the coming years, particularly with regard to model validation and model governance.

Introduction

Areas of weakness that require attention include failures to measure the acceptable risk level‌

Models are at the heart of financial institutions. The use of models is widespread as they can be powerful tools for decision support, scenario analyses and determining the value of assets and liabilities. Models are also used for identifying, measuring, monitoring and mitigating risks. Regulatory requirements such as the use test in the Basel and Solvency directives force financial institutions to understand and apply the results of their models in the daily risk management processes.

Both internal as well as external stakeholders rely on information resulting from models. Understanding the risks and limitations of these models is of vital importance as decisions based on models that are incorrect or misused can have severe consequences.

Financial institutions may mitigate these consequences by means of an effective management of model risk.

An often applied definition of model risk is that it reflects the risk that a model insufficiently approaches reality. A solid model governance structure and periodic validation of the models could reduce this risk.

Model life cycle In the model management process nine phases can be distinguished. These phases together form the so-called model life cycle. In the first phase the stakeholders agree upon technical procedures and the project plan for initiating a new model. A clear statement of the purpose and use of the model must be made, which should be consistent with the policy of the institution and the regulation in force.

During the second phase the model is developed.

This requires a full understanding of the underlying theories and assumptions underlying the model. Testing the performance of the model and the sensitivity of the results forms an integral part of the model development process. In the third phase of the model life cycle an independent validation of the model takes place. During this phase the soundness of the model is verified as well as the extent of compliance with objectives, regulations and business practices. The

fourth phase consists of the approval of the model by a responsible committee. When the model is approved, it will be embedded in the organisation during the fifth phase of the model life cycle and in the sixth phase the model will be put into operation. The seventh phase consists of regularly monitoring the input and output of the model. During the eighth phase the model performance is challenged and, if applicable, improvements to the model are proposed. Finally, the ninth phase consists of periodically performed independent validations of the model.

Models and the financial crisis The international financial and economic crisis considerably changed the landscape of the financial services industry and has resulted in greater emphasis on prudent risk management. The financial crisis revealed that major events, that were estimated to be extremely rare, could suddenly appear to be realistic market movements. Many financial institutions were not prepared to absorb large losses, which resulted in several collapses and widespread government support.

The size, impact and timing of the problems on financial markets and the real economy came as a surprise for most practitioners, regulators and academics. Although it should be noted that proper risk management cannot always prevent large losses, it has become clear that it should be given higher priority within financial institutions. In the Senior Supervisors Group (SSG) paper Observations on Risk Management Practices during the Recent Market Turbulence a number of risk management practices were identified that enabled certain financial institutions to withstand market stresses during the early financial crisis better than others2. These practices include effective firm-wide risk identification, a consistent application of independent and rigorous valuation practices across the institution and informative and responsive risk measurement. In the successive report Risk Management Lessons from the Global Banking Crisis of 2008, the SSG reviews funding and liquidity issues in the financial crisis and mentions deficiencies in governance, risk management and internal controls.3 Areas of weakness that require attention include failure 2

Observations on Risk Management Practices during the Recent

Market Turbulence, Senior Supervisors Group, March 2008.

3

Risk Management Lessons from the Global Banking Crisis of

2008, Senior Supervisors Group, October 2009.

Growth through Model Control Global Model Practice Survey 2011

7

Existing regulatory structures are currently being transformed and regulatory authorities are coordinating their reforms. to measure the acceptable risk level, inadequate technological infrastructures that impede effective risk identification and management, and prioritising risk taking at the cost of risk management. Furthermore, in this report, the SSG concludes on the results of a survey of financial institutions, including self-assessments.

Supervisors are mentioned to believe “that a full and on-going commitment to risk control by management, as well as the dedication of considerable resources toward developing the necessary information technology infrastructure, will be required‌â€? to ensure alignment between actual and recommended practice in a robust and sustainable way.

Regulatory response to the crisis As a result of the financial crisis, regulators are increasingly paying attention to topics such as the performance of models, capital requirements and accounting methods. Existing regulatory structures are currently being transformed and regulatory authorities are coordinating their reforms. On a global level new regulatory frameworks are being implemented; for banks Basel II is already in effect, with Basel III becoming effective starting in 2013, and for insurers Solvency II will be effective as of 2013.

Also on a local level existing regulatory frameworks are being revised. Examples are the Dodd-Frank Wall Street Reform and Consumer Protection Act in the United States (Dodd-Frank Act), affecting all federal financial regulatory agencies and the entire U.S. financial services industry. In the United Kingdom, plans are being made to reform the existing regulatory framework with prudential regulation of banks to be separated from oversight of consumer protection and market conduct. Other countries are implementing equivalents of regulatory frameworks. In South Africa for example, insurers have to comply with the Solvency Assessment and Management (SAM) framework, which is currently being enhanced significantly. Not all

8

implications of the regulatory restructuring are clear yet, but what is clear is that certain financial institutions need to invest considerable amounts of time and resources to comply with the stringent rules.

Regulation and models Recently, regulatory authorities have been encouraging financial institutions to improve the governance of risk related models and to give model validation a more prominent role in the risk management function, including regular validation of risk models by an independent department.

One of the results of the financial crisis was the European Commission creating the European System of Financial Supervisors (ESFS) in 2009. The ESFS replaced three existing Committees of Supervisors with three new authorities.4 These newly created authorities not only execute supervisory tasks, but also have regulatory power, including the ability to overrule national regulators if these fail to properly regulate the local financial institutions.

The Basel Committee on Banking Supervision (BCBS) published initial guidance on corporate governance in 1999, with revised Principles in 2006 and 2010.5 Under the Basel II Directive, financial institutions have to demonstrate to their supervisors that the internal validation process is able to assess and control the performance of credit, market and operational risk models. With Basel III coming into force, the requirements on regulatory capital and the reporting of information to the supervisor and the market will expand.

4

The European Banking Authority (EBA) replaced the Committee

of European Banking Supervisors (CEBS), the European Insurance and Occupational Pensions Authority (EIOPA) replaced the Committee of European Insurance and Occupational Pensions Supervisors (CEIOPS) and the European Securities and Markets Authority (ESMA) replaced the Committee of European Securities Regulators (CESR).

5

Enhancing Corporate Governance for Banking Organisations,

Basel Committee on Banking Supervision, September 1999 and February 2006, and Principles for enhancing corporate governance, Basel Committee on Banking Supervision, October 2010.

In the United States the Dodd-Frank Act is expected to have a major impact on the risk management function of banks in the coming years. The effects of the Dodd-Frank Act are also likely to have an impact outside the United States. As a consequence of the Dodd-Frank Act, the Office of the Comptroller of the Currency (OCC) is designated as the supervisory and regulatory authority for national banks and the federal branches and agencies of foreign banks. In Supervisory Guidance on Model Risk Management the OCC describes the key aspects of effective model risk management.6 Under the Solvency II Directive, insurers are required to have a model governance framework which should contain a regular cycle of model validation. This implies that these institutions should evaluate the performance and stability of internal models, the sensitivity of results to changing assumptions and the appropriateness of input data. They should also verify compliance with regulatory capital requirements. Furthermore, the Directive prescribes active involvement of senior management, ensuring systems operate properly and resources are available. In October 2011 the International Association of Insurance Supervisors introduced core principles, which are expected to have a considerable impact on global insurance regulation.

When using historical data, one should be aware of changing volatility and correlation patterns during periods of financial turmoil. Therefore, the provision and management of reliable data may need to be an integral part of enterprise risk management. A model that was validated yesterday may not be valid today. Furthermore, stringent regulations will force financial institutions to further professionalise their model governance structures. Although these changes require substantial investments in systems and people, organisations that are able to adapt quickest to the post-crisis financial environment could obtain strategic advantages by being better equipped to identify opportunities and manage downside risks effectively.

The second edition of the GMPS examines the progress of model practice within financial institutions with regards to the above mentioned challenges. Topics covered are model governance, development and validation, including outsourcing, vendor models, and changes in the approaches of internal audit departments as well as the supervisor. The impact of bank and insurance regulations on these topics is described throughout the survey. The role of model practice regulations in the investment management industry is described separately in section Regulatory Highlights.

For European investment managers, the European Security and Markets Authority (ESMA) 10-788 guidelines became effective in July 2011. These guidelines prescribe Undertakings for Collective Investment in Transferable Securities (UCITS) to have their Value-at-Risk (VaR) models validated by an independent party at the initial development or when introducing significant changes to existing models. The ESMA plays an increasingly important role as it fosters supervisory convergence amongst regulators and financial industries.

Lessons learnt Post crisis lessons learnt in the area of risk modelling include the need for effective firm-wide risk identification and the measurement of the acceptable risk level. Risk identification and measurement includes backtesting the outcomes of models on a regular basis.

6

Supervisory Guidance On Model Risk Management, Office of the

Comptroller of the Currency, April 2011.

Growth through Model Control Global Model Practice Survey 2011

9

1. Respondents

The GMPS includes responses of 89 financial institutions worldwide. This section provides a high level overview of the characteristics of these respondents.

Figure 1: Geographical distribution of respondents

4%

Europe

8%

Africa

Geography As shown in Figure 1, the institutions participating in the GMPS represent most of the major geographic regions in the world. About half of the participating financial institutions employ global activities, as these respondents indicate to be active on multiple continents.

Industry About three quarters of the respondents are employed by financial institutions that are active in the banking industry (see Figure 2). The insurance and investment management industries are represented by 19 and 5 per cent of the respondents respectively. As only four respondents are employed in the investment management industry it is not possible to draw solid conclusions for this industry based on the results of the survey.

38%

15%

Oceania Asia Middle America

16%

North America 19%

The figure is based on the office country of the respondents.

Note: Some graphs displayed in the survey do not add up to 100% due to rounding.

Figure 2: Respondents by industry 5%

19%

Bank Insurance

Department Most respondents fulfil a role at the risk management, model validation or model development department.

Around one-third of the respondents are employed in a risk management department, whereas 9 per cent of the respondents fulfil the role of Chief Risk Officer.

Model developers and model validators are similarly represented as both groups constitute around 20 per cent of the total number of respondents (see Figure 3).

Investment Management

76%

Figure 3: Role of respondents

3%

Risk Management

3% 3%

Model Validation

7% 36%

9%

Model Development/ Maintenance Chief Risk Officer Other Board level (other)

18% 21%

Senior Management Business Unit Internal Audit

10

Asset size The financial institutions participating in the survey have a variety of sizes. Around 40 per cent of the participating banks have an asset size smaller than 50 billion USD. The two other groups of about 30 per cent have asset sizes between 50 and 500 billion USD and larger than 500 billion USD respectively (see Figure 4).

For insurers the amount of gross written premiums is used as a measure of size. Half of the participating insurers have gross written premiums smaller than 1 billion USD, whereas the other two-quarters have gross written premiums between 1 and 10 billion USD and larger than 10 billion USD respectively (see Figure 5).

Figure 4: Asset size of banking respondents

Smaller than 50 billion USD

30% 41%

Between 50 and 500 billion USD Larger than 500 billion USD

29%

Figure 5: Amount of gross written premiums of insurance respondents

25%

Smaller than 1 billion USD 50%

Between 1 and 10 billion USD Larger than 10 billion USD

25%

Growth through Model Control Global Model Practice Survey 2011

11

2. Model Governance

94 per cent of all respondents experience their model environment to be complex to a moderate extent.

Model governance frameworks could provide support and structure to the risk management function, by stating policies for risk management activities, assigning roles and responsibilities and providing implementation as well as evaluation procedures. Proper governance frameworks and transparent model environments result in an adequate and transparent organisational structure with clear allocation and segregation of responsibilities.

Unambiguous accountability for each model could mitigate inefficient processes resulting in consistent and rigorous practices across the organisation. Furthermore, the existence of written and implemented policies could make solid risk management, internal auditing and outsourcing possible.

The BCBS and OCC both state that model governance should be provided at the highest level by the Board of Directors and senior management.7 They should establish a bank-wide approach to model risk management. For the insurance industry the International Association of Insurance Supervisors (IAIS) has introduced its Core Principles in October 2011. This will likely introduce greater regulatory convergence between different countries. The Principles state that insurers should establish and implement a corporate governance framework which provides sound and prudent management and adequately recognises and protects the interests of policyholders.

Model environment considered complex In the BCBS paper Principles for enhancing corporate governance, it has been recommended that senior management have adequate oversight of their governance framework and establish a management

7

Supervisory Guidance On Model Risk Management, Office of the

Comptroller of the Currency, April 2011.

12

structure that promotes accountability.8 One of the key factors in obtaining this oversight and accountability is a transparent model environment. The GMPS results show that almost all respondents experience their model environment (i.e., policies, data sourcing, system architecture and model dependencies) to be complex to a moderate extent (see Figure 6). This complexity could be aggravated by mergers and acquisitions. As such, for many financial institutions, the challenge in coming years might be to reduce this complexity.

Figure 6: Do you consider your model environment to be complex?

6%

Yes 36%

58%

Moderately No

Model policies can be improved Factors constituting a solid model governance policy framework include policies which provide structure, guidance and transparency for internal stakeholders.

Policies are not always given the appropriate level of priority. However, concise and clear policies that have arisen through support of the stakeholders (e.g., model developers, end-users, Board of Directors) could efficiently and effectively be embedded in an organisation. This may lead to consistent models created through an approach that meets the pre-defined internal leading practice.

The survey results indicate there are possibilities to improve the quality, or even the existence, of policies in several areas. As can be seen in Figure 7, up to half of the respondents indicate that one or more of the identified policies do not exist or are of poor quality. The Figure also demonstrates that the largest improvements can be made in the areas of expert

8

Principles for enhancing corporate governance, Basel Committee

on Banking Supervision, October 2010.

introductions. A model inventory may be an important tool to provide a clear oversight of the status of all models within the organisation. The maintenance of the inventory could be supported by establishing solid governance and designing processes for updating. A properly maintained model inventory will likely increase the control of financial institutions over the models in use and enable them to assess the impact of changes on the model life cycle.

“The board should give special attention to the quality, completeness and accuracy of the data it relies on to make risk decisions.”

Figure 8 shows that 30 per cent of the respondents do not have a model inventory in place. At an industry level one-quarter of the banking respondents and half of the respondents from insurers indicate having no model inventory. For these respondents it could be difficult to illuminate their model environment which, as described earlier, is generally considered to be complex.

Principles for enhancing corporate governance of the BCBS judgment, model implementation and data quality policies. The latter is of particular interest as model developers and users indicate that they spend most time on data quality as will be discussed later in the report.

Regulators also stress the importance of data quality within the governance framework. The Principles for enhancing corporate governance (BCBS) states that “the board should give special attention to the quality, completeness and accuracy of the data it relies on to make risk decisions.” The first step to take to achieve this would be to have a data policy in place of sufficient quality.

Figure 8: Do you have a model inventory?

30%

Yes No

70%

Widespread use of model inventory Financial institutions employ a large variety of models.

Over time the model environment changes, caused by model mergers, enhancements, phase outs and Figure 7: How do you perceive the quality of the following policies?

Expert Judgment Policy

43%

Data Quality Policy

30%

Model Implementation Policy

31%

Significant Change Policy

28%

Model Development Policy

20%

Model Monitoring Policy

40%

8%

25%

40%

13%

38%

15%

38%

20%

14%

42%

24%

11%

43%

25%

49%

26%

18% 17% 14%

20%

Model Risk Policy

9%

22%

3%

Model Governance Policy

11%

13%

41%

35%

Model Validation Policy

13%

8%

43%

36%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

Policy does not exist Poor Reasonable Good

100%

Growth through Model Control Global Model Practice Survey 2011

13

this percentage is smaller with around one-quarter of the respondents mentioning the CRO to be the owner of the framework (not shown). In the OCC paper Supervisory Guidance On Model Risk Management it is emphasised that the Board of Directors should have clear oversight of risk management. Appointing a CRO, or equivalent senior risk officer, as the model governance framework owner can help clarify accountability and create clear oversight. In the Principles of the IAIS the role of the CRO is also viewed as an important function holder for effective risk management within insurers.

“The business unit using the models and the model risk management unit have to approve whether a model is fit for purpose.�

Bank, United States For respondents which possess a model inventory, the main focus is on the status of the model (e.g., under development, implemented or retired) and the purpose and products for which the model is used. Capturing the status of supervisory approval and the time frame during which a model is expected to remain valid could be improved, as at least half of the respondents indicate that these aspects are at most partly captured (see Figure 9).

Transparent model governance ownership not always safeguarded The survey results show that in 40 per cent of the participating financial institutions, the Chief Risk Officer (CRO) is the owner of the model governance framework (Figure 10). For respondents from insurers,

Roughly one out of five respondents indicates that multiple committees have ownership of the model governance framework. This could be a cause for concern as this may result in no particular stakeholder taking ultimate responsibility for the framework and the accountability of stakeholders not being transparent.

Furthermore, the ownership of the model governance framework is indicated to be at the model development or validation department by about 20 per cent of the respondents. While these departments have strong knowledge of the technical aspects of the models, we believe assigning ownership to a stakeholder that is part of the process may not lead to a transparent model governance framework, since the responsibilities of these stakeholders are described in the framework itself.

Figure 9: To what extent are the following aspects captured for each model by your

Figure 10: Who is the owner of the model governance

model inventory?

framework?

Time frame during which a model is expected to remain valid

32%

Status supervisory approval

40%

31%

Description of extent to which the model functions adequately

13%

Dates of completed and planned validation activities

11%

Status of the model

10%

27%

18%

8%

Chief Risk Officer

9%

52%

Multiple committees 40%

27%

60%

Single committee 29%

60%

Other

13% 24%

Purpose and products for 5% which the model is used 0%

66%

29% 10%

20%

18%

66% 30%

40%

Not at all

14

Head(s) Model Development

13%

50%

Partly

60%

Fully

70%

80%

90%

100%

Head Model Validation

3. Model Development

Model developers have the largest added value when they can apply their skills and knowledge to the actual modelling.

As mentioned in the description of the model life cycle in the introduction, model development involves more than just the actual implementation of a model in the current environment. The process of model development incorporates a clear statement of the purpose and use of the model, which should be consistent with the policy of the institution and regulation in force. Sound development of models includes a full understanding of the underlying theories and assumptions which could be supported by appropriate research and a view to leading practice.

Testing the various components, the performance of the model and the sensitivity of results forms an integral part of the model development process.

Input data plays a critical role, as the recent financial crisis emphasised that the absence of appropriate model inputs could lead to the underestimation of risks. Furthermore, proper documentation of all methodological decisions may enable the model to be reproduced.

Time spent on data The development of robust models involves many activities, which all require a balanced allocation of time. Based on the survey results shown in Figure 11, model developers spend a significant proportion of their time collecting data and controlling the quality.

Less time is spent on the actual modelling. Model developers may have the largest added value when they can apply their skills and knowledge to the actual modelling process. Spending a disproportionate amount of time on the collection and quality control of data prevents model developers from researching and implementing methodological enhancements that reduce model risk and errors. The predominance of data collection and preparation in the allocation of time for model developers could be reduced by delegating tasks to departments that supply data to the model developers and users. It could also be beneficial for model development departments to collaborate with IT departments on data quality requirements instead of personally retrieving and amending the data. This collaboration could be accomplished by means of, for example, a Data Delivery Agreement between both departments. Moreover, regulatory guidelines require financial institutions to define a clear data quality policy. In the consultation paper CEIOPS’ Advice for Level 2 Implementing Measures on Solvency II, CEIOPS for example recommends that institutions clearly define a data quality and update policy, hereby ensuring that their own concept of data quality fulfils the criteria of

Figure 11: To what extent does your organisation spend time on each of the following activities for a model? Please order activities from least time spend (1) to most time spend (7).

Specification of model objectives Stakeholder management Verification of compliance with internal and external guidelines Documentation Identification of the relevant model Model testing and selection Data collection and preparation 1

2

3

4

5

6

Growth through Model Control Global Model Practice Survey 2011

7

15

The absence of proper model documentation reduces the effectiveness of model risk management. accuracy (i.e., degree of confidence), completeness (i.e., comprehensive information in databases) and appropriateness (i.e., data without biases).9 The survey results indicate that model development departments spend the least time on specifying model objectives and verifying compliance with guidelines.

Although these activities in themselves may not require much time, they may still be crucial to the success of later phases of the model life cycle. Furthermore, respondents indicate that model development departments do not spend a large amount of time writing model documentation. Accurately documenting the development process, use and governance of models is clearly important. The OCC paper Supervisory Guidance on Model Risk Management mentions several benefits of the presence of sound model documentation.10 It allows for the continuation of operations and helps parties, some of which might be unfamiliar with the model, to examine the working and underlying assumptions. Model documentation allows validators, internal auditors and supervisors to assess the soundness of the model. The absence of proper model documentation reduces the effectiveness of model risk management. The Guidelines on the implementation, validation and assessment of Advanced Measurement (AMA) and Internal Ratings Based (IRB) Approaches of the CEBS, mention several requirements on the documentation of rating systems

9

Furthermore, the above mentioned OCC paper indicates that properly managed knowledge management systems allow financial institutions to benefit from information advantages. It seems that model developers do not prioritise the documentation of risk models. In order to prevent too much time being allocated to documentation at the cost of actual development and improvement of models, one could for example delegate the development and maintenance of documentation to professionals more specialised in these activities.

Standardisation of processes can be improved Standardising processes may be another way to improve the quality and effectiveness of model development. A standardised process may improve efficiency and uniformity within an organisation. As shown in Figures 12 and 13, compared to respondents from insurers, respondents working at banks indicate that on average more processes are standardised.

However, both industries indicate that processes could be improved for the use of coding standards in programming and the acceptance criteria for data quality, or that these processes may still need to be implemented. As model developers allocate relatively more time on the collection and quality improvement of data, and data quality acceptance processes are generally not standardised, improvements in efficiency could be possible. Accordingly, more time could be made available for writing model documentation, the actual modelling of methodologies that reduce model risk and errors, and innovation.

CEIOPS’ Advice for Level 2 Implementing Measures on Solvency

II: Articles 120 to 126, Tests and Standards for Internal Model Approval, CEIOPS e.V., October 2009.

10

Supervisory Guidance On Model Risk Management, Office of

the Comptroller of the Currency, April 2011.

16

and operational risk measurement systems.11 Essential elements include a list of all internal documents held by the applicant, maps of the models used in the AMA and IRB approaches and general descriptions of each of the models.

11

Guidelines on the implementation, validation and assessment of

Advanced Measurement (AMA) and Internal Ratings Based (IRB) Approaches, Committee of European Banking Supervisors (CEBS), April 2006.

“The biggest change in model development over the next five years is the rebuilding of models for use in new IT infrastructure, allowing wider use of better customer data.”

Bank, United Kingdom

Figure 12: Is the model development process standardised in the following areas? — Banking

The use of coding standards in programming

37%

9%

54%

No Yes, but not in a formal way Yes, by means of policies

Acceptance criteria for data quality

21%

Guidelines on model development

57%

7%

Acceptance criteria for developing and monitoring a model Guidelines on model documentation

22%

52%

40%

10%

48%

42%

10%

43%

46%

20%

0%

40%

60%

80%

100%

Figure 13: Is the model development process standardised in the following areas? — Insurance

The use of coding standards in programming

81%

No

19%

Yes, but not in a formal way Yes, by means of policies

Acceptance criteria for data quality

35%

65%

Acceptance criteria for developing and monitoring a model

35%

59%

Guidelines on model documentation

41%

47%

12%

29%

59%

12%

Guidelines on model development

0%

20%

40%

6%

60%

80%

100%

Growth through Model Control Global Model Practice Survey 2011

17

Increase of regulations and activities Survey respondents appear to be quite unambiguous in their anticipation on annual spending and the direction of changes in the model development area. As shown in Figure 14, over the next three years, the annual spending on model development is on average forecast to increase across all industries. The biggest changes are expected to occur in regulation and the increase in activities in general, as depicted in Figure 15. An increase in regulation to comply with is interrelated with an increase in activities in general. The increase in activities subsequently requires an expansion of annual spending.

Figure 14: Over the next three years, how do you think your annual spending on model development will change (in % compared to last year)?

Decrease by 20% or more

Decrease by 5% to 20%

Stay at approximately the current level Increase by 5% to 20%

Increase by 20% or more

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Figure 15: What do you expect to be the biggest change(s) in the Model Development area over the next five years?

Reduction in activities Migration to a cloud environment Off-shoring of standard activities Other

Increase in activities

Regulation

0%

10%

20%

30%

40%

50%

60%

70%

Note: Percentages total to more than 100% because respondents could make multiple selections.

18

80%

90%

100%

Outsourcing expected to increase Outsourcing is the process of entrusting certain business functions to an external service provider.

External resources may provide added knowledge and improvements in efficiency. As shown in Figure 16, about two thirds of the respondents indicate that outsourcing of the development of models happens to an extent. Almost one-quarter of the financial institutions included in the survey even mentions that “25% or more� of the total number of models requiring validation are outsourced.

Figure 16: To what extent does your organisation outsource the development or improvement of models (in % of all models that require validation)?

Not at all

21% 38%

1 to 5% 6 to 10%

11%

10 to 25% 8%

As depicted in Figure 17, the main reasons for outsourcing model development activities are capacity constraints and a lack of knowledge. The answers given are roughly the same for banks and insurers.

These outcomes indicate that institutions within the financial services industry experience difficulties in hiring a sufficiently large number of skilled professionals which is not necessarily related to the amount of funds available to attract a skilled workforce.

25% or more 22%

Earlier we noted that respondents expect an increase in activities for model development departments.

Without changes in the annual spending on risk management, this will put more pressure on financial institutions to comply with more stringent regulations in the near future. More stringent regulation generally involves changes in model use and reporting at various departments within financial institutions. Embedding these changes, whilst already facing capacity constraints, will be challenging for the institutions.

Outsourcing activities could be an appropriate solution to reduce pressure on the internal risk management organisation, so we would expect this to increase over the coming years.

Figure 17: What is the reason for outsourcing model development activities?

Mandatory outsourcing

Other

Lack of knowledge

Capacity constraints

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Note: Percentages total to more than 100% because respondents could make multiple selections.

Growth through Model Control Global Model Practice Survey 2011

19

4. Model Validation

Model validation is the set of tools and processes used to verify the soundness of models and the extent of compliance with objectives, regulations and business practices. Model validation involves a quantitative as well as qualitative interpretation of the design, underlying assumptions and output of the models.

Model validations are periodically repeated processes which ensure that important changes in activities or the external environment do not lead to inappropriate model output. Validation activities may need to involve an examination of the methodology, input of data and experts, governance, IT implementation and the sensitivity and plausibility of the results. Ideally, based on the descriptions in the model documentation, it should be possible for a skilled independent person to replicate the model. We believe that model owners may be expected to demonstrate and convince the model validator that a model is fit for its intended use.

Effective model validation contributes to the reduction of risk for financial institutions by identifying errors and areas for improvement or even redevelopment. We emphasise that there is not one universal validation process as validations need to be tailored to the specific risks and activities underlying the model in question.

Validation function not always present Figures 18 and 19 show that not all financial institutions possess independent model validation departments. As opposed to banks (19 per cent), the majority of insurers (65 per cent) indicate having no independent model validation function at this stage.

Figure 19: Does your organisation have an independent Model Validation function? — Insurance

35% 65%

Yes No

Regulatory authorities have recently been encouraging financial institutions to give model validation a more prominent role in the risk management function, including regular validations of models by an independent department. Based on the 2004 Basel II Framework, the Committee of European Banking Supervisors (CEBS)12 published guidelines on the implementation, validation and assessment of Advanced Measurement (AMA) and Internal Ratings Based (IRB) Approaches.13 As such, currently banks have been demonstrating to their supervisors over several years that their internal validation processes are capable of assessing and controlling the performance of their risk models. With the ESMA 10-788 and Solvency II guidelines becoming effective, UCITS and insurers are also required to employ a regular cycle of model validation which is less well evolved.

Figure 18: Does your organisation have an independent Model Validation function? — Banking

19% Yes No 81% 12

In January 2011 the CEBS was succeeded by the European

Banking Authority (EBA), which took over all the existing and on-going tasks and responsibilities.

13

Guidelines on the implementation, validation and assessment of

Advanced Measurement (AMA) and Internal Ratings Based (IRB) Approaches, Committee of European Banking Supervisors (CEBS), April 2006.

20

“Insurance and reinsurance undertakings shall have a regular cycle of model validation...”

Advice for Level 2 Implementing Measures on Solvency II of CEIOPS The Committee of European Insurance and Occupational Pensions Supervisors (CEIOPS), which provided technical support to the European Commission in developing Solvency II, stated guidelines on model practice for insurers.14 The 2009 CEIOPS consultation paper CEIOPS’ Advice for Level 2 Implementing Measures on Solvency II includes articles about tests and standards for internal model approval. The paper makes explicit the requirement to accommodate an independent model validation function by stating that “Insurance and reinsurance undertakings shall have a regular cycle of model validation….” CEIOPS underlines the value of model validation by putting forward that, besides improving the institution’s own risk management, this helps to ensure that the level of regulatory capital is not materially misstated and accordingly that the level of policy holder protection will not decrease. Considering the requirements of the Solvency II framework, insurers might need to increase the prioritisation of the design of sound model validation policies in the coming years.

Whilst validation of some models is compulsory, we recommend financial institutions to opt for the validation of material non-compulsory models as well.

Within such an approach financial institutions assess the possible impact of these non-compulsory models on business processes. By means of validating the models it may be ensured that important decisions are made based on sound model outcomes.

The contrast in the maturity of model validation between banks and insurers also manifests itself in the average size of present model validation departments.

Respondents indicate that, if present, the size of the

14

In January 2011 the CEIOPS was succeeded by the European

Insurance and Occupational Pensions Authority (EIOPA), which took over all the existing and on-going tasks and responsibilities.

model validation department is larger at banks than at insurers, as shown in Figure 20 and 21.

Figure 20: What is the approximate size of the Model Validation function in terms of FTE? — Banking

38%

37%

Smaller than 5 5 to 10 Larger than 10

25%

Figure 21: What is the approximate size of the Model Validation function in terms of FTE? — Insurance

50%

50%

Smaller than 5 5 to 10

Model validation function more mature at banks We previously concluded that there are clear differences in the maturity of model validation practice between banks and insurers. Financial institutions that invest more resources in validating models may be able to execute more profound validation trajectories. The OCC paper Supervisory Guidance on Model Risk Management mentions that the rigor and sophistication of model validations should correspond to the financial institutions overall use, complexity and materiality of its models, and the size and complexity of its operations. As can be seen in Figure 22 and 23, respondents working at insurers indicate the majority (83 per cent) of model validations take one to four weeks to complete, whereas for banks, an average model validation requires more time on average.

Clearly, insurers allocate fewer resources to model validations, exhibited by shorter time spans. These

Growth through Model Control Global Model Practice Survey 2011

21

shorter time spans could be due to less thorough analyses or less complex risk models at insurers relative to banks.

that, compared to banks, respondents from insurers challenge the quality and representativeness of external data to a lesser extent.

Model validation focus areas Figure 24 shows the extent to which model validation departments challenge the soundness of various focus areas. Methodology, assumptions and regulatory compliance are indicated to receive significant attention across all industries. On the contrary the soundness of IT and systems (i.e., quality of coding and implementation) and model governance (i.e., roles and responsibilities) are challenged less.

Figure 22: On average, how much time does the validation of a model take in weeks? — Banking

7%

21% 1 to 2 weeks 3 to 4 weeks

40%

5 to 8 weeks 9 or more weeks

32%

In the paper Guidelines on the implementation, validation and assessment of Advanced Measurement (AMA) and Internal Ratings Based (IRB) Approaches CEBS underlines the importance of a sound IT infrastructure with appropriate controls and a continuous availability and maintenance of databases.

Based on the survey results, more time could be devoted to the assessment of the soundness of IT and systems during a validation.

Figure 23: On average, how much time does the validation of a model take in weeks? — Insurance

16%

17%

The quality of IT and systems directly influences the availability of reliable data. The consultation paper CEIOPS’ Advice for Level 2 Implementing Measures on Solvency II (CEIOPS) mentions that the model validation process “shall include an assessment of the accuracy, completeness and appropriateness of the data used by the internal model.” The survey results indicate

1 to 2 weeks 3 to 4 weeks 9 or more weeks 67%

Figure 24: Does the model validation function challenge (the soundness of) each of the following areas?

Methodology

95%

5%

Yes

Assumptions

92%

8%

No

Compliance with regulatory requirements

88%

13%

Model Performance

84%

16%

Expert opinion

83%

17%

Internal data

83%

17%

Documentation

81%

19%

External data

73%

27%

Use Test

69%

31%

Model governance

61%

39%

IT / Systems

50%

50%

0%

22

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Figure 26: On average, how many models are rejected

14 per cent of the respondents indicate that they never rejected a regulatory model.

Model validation challenges The results shown in Figure 25 indicate that, in general, all aspects of the model validation process are considered relatively challenging. The most challenging aspects are mentioned to be resourcing, (i.e., recruitment and retention) retrieving data from the source systems and planning. The observation that some 25 per cent of the respondents mention that it is, to a large extent challenging for the model validation department to reject a model, is remarkable.

This challenge becomes particularly evident given that 14 per cent of the respondents indicate that they have never rejected a regulatory model (see Figure 26).

by the model validation function (as a percentage of all models that require to be validated)?

14%

14% 0% 1-5%

24%

5%-10% 48%

10% or more

We believe it could be possible that the model validation department occasionally lacks the authority to reject models, which may be caused by internal political pressures. When considering models for which validation is compulsory, model validation teams should be able and willing to enforce a rejection where required.

25 per cent of the respondents indicate that for the model validation department it is challenging to reject a model Figure 25: To what extent are the following model validation aspects challenging?

Consistency of validation between team members

28%

45%

20%

Testing the model performance

19%

53%

23%

Consistency of validation process over time

17%

56%

25%

Rejecting a model

17%

50%

27%

Planning

8%

50%

36%

Retrieving the data from the source

17%

39%

36%

13%

31%

48%

systems Resourcing (recruitment and retention) 0%

10%

20%

30%

40%

50%

60%

70%

80%

To no extent To a moderate extent To a large extent

90%

100%

Note: Percentages total to less than 100% because category ‘n/a’ is not shown.

Growth through Model Control Global Model Practice Survey 2011

23

Another possible explanation for the low number of model rejections could be that the size of model validation departments is smaller than the model development departments. Because of the smaller size, model validators might not be as specialised as model developers, but will likely have a broader knowledge. This could make it more difficult to reject a model, as model developers will have more specialised knowledge on the models.

Furthermore, model developers and model validators, who both work within departments which are part of the risk management function, have common interests to utilise models that are compliant with the regulatory framework. Consequently, it might be possible that they coordinate during model development from an early phase in the process. One risk associated with this approach is that it might potentially impair the independence of the two departments.

Internal political pressures relating to model rejections and potential appearance of impairment of independence between model development and model validation could be mitigated by periodically outsourcing model validations to external service providers.

Finally, the survey results indicate that although most financial institutions have their validation processes standardised by means of defined guidelines, fewer institutions employ acceptance criteria for the approval of these models. The standardisation of these acceptance criteria may strengthen the position of model validators with respect to model rejections.

Causes for rejections As shown in Figure 27, rejections of models as a result of validation activities are due to shortcomings in various dimensions. The most important reasons identified include model performance, methodology, assumptions and internal data. Insurers more often mention documentation, expert opinions and use tests as causes for rejection. Over 20 per cent of the respondents did not identify regulatory requirements as a cause for rejection. This is remarkable because an important feature of models is the need to comply with regulation. From the initial phase until implementation model developers may need to remain aware that the results of the model will not be allowed to be used if the supervisor does not approve the model. Therefore, in an ideal situation, it should never be the case that model validation concludes that the model does not comply with regulation.

Figure 27: In case a model is rejected by the model validation department, which of the following dimensions of model validation most frequently occur as a cause of rejection?

Model governance IT / Systems External data Use Test Compliance with regulation Documentation Expert opinion Internal data Assumptions Methodology Model Performance 0%

10%

20%

30%

40%

50%

60%

70%

80%

Note: Percentages total to more than 100% because respondents could make multiple selections.

24

90%

100%

“Third party expert review is occasionally used to supplement or validate our assumptions or approach with regard to specific products or issues.�

Insurer, United States Increase of regulations and activities The expected changes discussed earlier for the model development department are also valid for the model validation department. As shown in Figure 28, respondents anticipate changes in regulations and an increase in general activities. In response to the recent turmoil in financial markets, new and more stringent guidelines on modelling have recently been introduced, ESMA 10-788 for example, or will be introduced in the near future, like Solvency II. The increase in regulations, with more compulsory model validations, could be considered in relation to the expected increase in activities in general. The changing environment could result in an increasing pressure on model validation departments, which have already experienced difficulties with the recruitment and retention of skilled professionals.

Figure 28: What do you expect to be the biggest change(s) in the Model Validation area over the next five years?

Migration to a cloud environment Reduction in activities

Off-shoring of standard activities Other

Increase in activities

Regulation

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Note: Percentages total to more than 100% because respondents could make multiple selections.

Growth through Model Control Global Model Practice Survey 2011

25

Outsourcing expected to increase Figure 29 shows the extent to which financial institutions outsource the validation of models.

Although more than 40 per cent of the respondents indicate that the model validation function is not outsourced, a considerable group refers to external service providers as performers of at least 25 per cent of all compulsory model validations. Equal to the reasoning for model development departments, capacity constraints are indicated to be the main reason (Figure 30). Apparently, outsourcing model validation activities to external service providers could help reduce the pressure on the internal organisation.

Another important reason for outsourcing, frequently mentioned by insurers, is the extended independence of the validation process. Outsourcing could remedy the earlier observed issues of internal political pressures on model validation departments, the frequency of model rejections and supposed non-independence of model validation and model development functions.

Figure 29: To what extent does your organisation outsource the validation of models (in % of all models that require validation)?

22% Not at all 43%

6 to 10% 9%

10 to 25% 25% or more 12% 14%

Figure 30: What is the reason for outsourcing model validation activities?

Mandatory outsourcing

Other

Independence

Capacity constraints

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

Note: Percentages total to more than 100% because respondents could make multiple selections.

26

1 to 5%

100%

5. Vendor Models

“When validating a vendor model, the quality of support, model reputation and testing the stability of the outcomes are of importance.”

Insurer, the Netherlands As described in previous sections of this report, model development and validation departments experience capacity constraints and overall activities are expected to increase in the coming years. Therefore, we expect the use of vendor models to increase. Subsequently we expect the validation of vendor models to become increasingly important in the coming years. The BCBS paper Vendor models for credit risk measurement and management describes that validation of models is crucial when determining whether a model is fit for use.15 Although this paper focuses on credit risk, the validation of vendor models is relevant for other models as well. The OCC states that vendor models should be incorporated into the bank’s broader model risk framework and banks are expected to validate their own use of vendor products.16 Although a vendor model is developed by an external party, an institution which purchases the vendor model is responsible for demonstrating compliance of the model with regulatory requirements. According to the BCBS, vendors should always be able to furnish documentation and reports relating to the validation of their models and the mapping of developmental data used in the model to a client specified portfolio.17

user groups for externally developed models. Models could be better customised when regular consultation with the vendor takes place. In addition, employees could benefit from the knowledge of the vendor.

In this year’s survey, as shown in Figure 31, about two-thirds of the respondents indicate that their financial institutions make use of vendor models. More than half of the respondents indicate that at least 10 per cent of all models in the institutions are vendor models (see Figure 32). The OCC states that appropriate processes should be in place for selecting vendor models. Examples of areas of attention are sensitivity analyses, proper documentation and justification of customised choices.

Figure 31: Does you organisation use vendor models?

31%

Yes No 69%

Figure 32: To what extent does your organisation use vendor models (in % compared to all models used in the organisation)?

4% 32%

23%

Not at all 1 to 5% 6 to 10% 10 to 25%

In the previous edition of the GMPS, 50 per cent of the respondents indicated that they do not participate in

15

25% or more 18% 23%

Vendor models for credit risk measurement and management,

Basel Committee on Banking Supervision, February 2010.

16

Supervisory Guidance On Model Risk Management, Office of

the Comptroller of the Currency, April 2011.

17

Use of Vendor Products in the Basel II IRB Framework, Basel

Committee Newsletter No. 8, March 2006.

Growth through Model Control Global Model Practice Survey 2011

27

Respondents indicate that the most important areas of attention when validating vendor models are the performance of the model using internal data, compliance with regulatory requirements and the appropriateness of the model for the products and risks. The review of the programming code receives less attention. Compared to insurers, banks pay more

attention to most of the questioned validation aspects (see Figure 33 and 34). When validating vendor models, insurers, compared to banks, indicate less attention is paid to benchmarking and regular model validation procedures and more to reviewing the programming code.

Figure 33: When validating a vendor model, do you pay attention to the following areas? — Banking The appropriateness of the model for the products and risks

100%

Model performance on internal data

98%

Choices customizations made to the generic vendor model

95%

Compliance with regulatory requirements

95%

Sensitivity analyses

93%

Regular Model Validation procedures

91%

9%

Benchmarking

86%

14%

The selection process of the vendor model

77%

Review of the programming code / algorithm

47% 0%

10%

20%

Yes 2%

No

5% 5% 7%

23% 53% 30%

40%

50%

60%

70%

80%

90%

100%

Figure 34: When validating a vendor model, do you pay attention to the following areas? — Insurance The appropriateness of the model for the products and risks

93%

Model performance on internal data Choices customizations made to generic vendor model The selection process of the vendor model

86%

14%

86%

14%

79%

21%

Compliance with regulatory requirements

79%

21%

Sensitivity analyses

79%

21%

Regular Model Validation procedures

57%

43%

Review of the programming code / algorithm

57%

Benchmarking

50% 0%

28

7%

10%

20%

43% 50% 30%

40%

50%

60%

70%

80%

90%

Yes No

100%

6. Internal Audit

“Accuracy and completeness of the model inventory should be assessed by internal audit.”

Supervisory Guidance On Model Risk Management of the OCC Role of internal audit Internal audit plays an important role with respect to the internal control mechanisms within a financial institution. Internal audit should ensure effectiveness and efficiency of operations, reliability of financial reporting and compliance with laws and regulations.

Internal audit is also responsible for challenging the risk management processes within a financial institution.

The effectiveness of the risk management processes should be monitored and evaluated on a regular basis.

Furthermore, internal audit generally participates in meetings and discussions with members of the Board of Directors on corporate governance within a financial institution.

The role of the internal audit department with respect to model practice is to ascertain whether the models of the financial institution meet the minimum regulatory requirements. Herewith, internal audit should establish the independence of the model validation process and assess the overall effectiveness of the model risk management framework. In Basel Committee Newsletter No. 4 the BCBS states that “it is important

that a bank’s validation processes and results should be reviewed for integrity by parties within the banking organisation that are independent of those accountable for the design and implementation of the validation process.”18 The internal audit department is responsible for ensuring that validation processes are effective and executed as designed. This implies that it is not the task of the internal audit department to repeat the work of model validation, but instead to evaluate the rigorousness, effectiveness and comprehensiveness of the model risk management framework.

Increased strictness Simultaneous to the prospects for model developers and model validators, the internal audit department will face an increase in stringent regulatory requirements.

In general, as a result of the recent financial crisis, the approach and findings of internal audit are mentioned to be either similar or stricter, as shown in Figure 35.

18

Update on work of the Accord Implementation Group related

to validation under the Basel II Framework, Basel Committee Newsletter No. 4, Basel Committee on Banking Supervision, January 2005.

Figure 35: In which way have the approach and findings of the internal audit department changed as a result of the recent financial crisis?

Have become much less strict Have become slightly less strict Have not changed

Have become slightly more strict Have become much more strict 0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Growth through Model Control Global Model Practice Survey 2011

29

Focus areas Besides the changes in the strictness of the approach followed by internal audit, respondents also signal differences in the areas where the internal audit departments pay attention to. In all industries involved in the GMPS, respondents indicate that internal audit departments pay more attention to the model governance policy framework (see Figure 36). It is notable that about half of the respondents indicate that internal audit does not pay attention to the appropriate use of models. Also less attention is paid to the appropriate development of models and the model inventory. However, in the paper Supervisory Guidance on Model Risk Management the OCC states that “Internal audit should also verify records of model use and validation....” and that “Accuracy and completeness of the model inventory should be assessed”. Furthermore, more than 80 per cent of the respondents that employ vendor models indicate that internal audit does not review the control of these models. Compared to banks, the internal audit departments from insurers were less focused on model practice in general.

Figure 36: Does the internal audit department pay attention to the following areas: Model governance policy framework

78%

Appropriate validation of models

73%

27%

Appropriate implementation of models

72%

28%

Appropriate monitoring of models

68%

33%

Model inventory

59%

41%

Appropriate development of models

59%

41%

Appropriate use of models

47%

53%

Extent of control on vendor models

18% 0%

30

10%

Yes

22%

No

82% 20%

30%

40%

50%

60%

70%

80%

90%

100%

7. The Supervisor

supervisors will be changing. This includes a general increase in strictness in the monitoring of rules, a broadening of the existing tasks to include new capital requirements and the need for international cooperation between supervisors. The implementation of the Dodd-Frank Act in the U.S. for instance, significantly increased regulatory power and broadened the scope of the activities of existing supervisors like the Securities Exchange Commission (SEC) and the OCC and newly created parties like the Financial Stability Oversight Council (FSOC). In 2009 the European Commission expanded the influence of three former committees, CEBS, CEIOPS and CESR, by granting the successive agencies regulatory power.

“The supervisor is expected to put greater focus on the amount of capital. Many more data requests and requiring greater understanding of risk management practices are also expected.�

Bank, United Kingdom Supervisors regularly evaluate and review the compliance of financial institutions with regulatory requirements. They have the power to enforce corrective or supplemental actions. The compliance comprises both qualitative and quantitative requirements and is primarily aimed at assessing financial institutions’ ability to resist the effects of current or potential financial risks.

The survey results, shown in Figure 38 and 39 (see page 32), indicate that financial institutions, especially banks, consider the approach and findings of the supervisor, with regard to the areas mentioned above, as being the same or stricter. None of the respondents indicated that the approach and findings of the supervisor have become less strict.

Focus areas When assessing the quality of the risk management framework, the supervisor is less focused on the appropriateness of internal auditing activities on model practice and also the appropriate use of models (see Figure 37, page 32). It is also notable to mention that supervisors seem to focus less on the appropriate validation and the model governance policy framework, whilst these areas are of importance for the compliance of financial institutions to regulations.

As mentioned earlier, appropriate validation of models and formulating clear policies leads to consistent models created through an approach that meets the pre-defined internal leading practice. An increased focus of the supervisor on model validation and the model governance policy framework could therefore improve the soundness of model practice within a financial institution.

Increasing strictness Interrelated to the changing landscape of the financial services industry, the future tasks and duties of

Growth through Model Control Global Model Practice Survey 2011

31

Figure 37: Does your Supervisor pay attention to the following areas while assessing the quality of your company’s risk management framework?

Appropriate development of models

89%

11%

Appropriate monitoring of models

84%

16%

Extent of control on vendor models

80%

20%

Appropriate implementation of models

80%

20%

Model inventory

80%

20%

Appropriate validation of models

74%

26%

Model governance policy framework

69%

31%

Appropriate use of models

64%

36%

Appropriate internal audit activities on model practice

38%

63%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

Yes No

100%

Figure 38: Have the approach and findings of the Supervisor with regard to the previously mentioned areas changed as a result of the recent financial crisis? — Banking

Have become much less strict

Have become slightly less strict

Have not changed

Have become slightly more strict

Have become much more strict

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Figure 39: Have the approach and findings of the Supervisor with regard to the previously mentioned areas changed as a result of the recent financial crisis? — Insurance Have become much less strict

Have become slightly less strict

Have not changed

Have become slightly more strict

Have become much more strict

0%

32

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

8. Regulatory Highlights

UCITS IV requires having an independent and periodic validation of Value-at-Risk models.

The influence of more stringent regulations on banks and insurers has been discussed extensively in the previous sections. As only four respondents of our survey are employed in the investment management industry, no proper conclusions could be drawn for this industry. However, the regulatory landscape for investment managers faces a similar degree of reforms to that faced by banks and insurers. Therefore, this section focuses on the challenges for investment managers with regards to model practice in the years to come. Some of these reforms stem from G20 initiatives whereas others are locally driven. Although it is not known at this time which of the many regulatory reforms will ultimately be adopted and what the exact effect will be on these institutions, it is clear that the changes will continue to have a significant impact on the investment management industry in the coming years.

Model practice not high on agenda Our survey was completed by only four respondents from the investment management industry. This could be due to the fact that model governance and model validation are not priorities on the agenda of investment managers at the moment. The validation of models has, until recently, not been explicitly required by a regulator. Also, model development is to a large extent outsourced. However, as stringent regulatory rules increase within the investment management industry, the prioritisation of model practice is likely to change in the years to come. Furthermore, clients of investment managers increasingly demand a clear insight how management information is generated (i.e. risk and performance data) and whether the models used are sufficiently sound. The clients demand these insights because of the more stringent regulations (for example Solvency II) and because they would like to be in control of their investments.

Regional differences In Europe, achieving compliance with UCITS regulation will be the main focus for the coming years. The latest of the revisions, UCITS IV, among others requires having an independent and periodic validation of Value-at-Risk (VaR) models (including stress testing) and monitoring leverage. A clear valuation of exposures to counterparties is also required. Besides these UCITS specific topics, improving the governance and overall risk frameworks will be of central importance for all investment managers. The Alternative Investment Fund Managers Directive (AIFMD) is also gaining ground. The AIFMD is expected to increasingly pay attention to the valuation, evaluation and risk measures of alternative investments in the near future. Although the AIFMD only has legal effectiveness within the European Union, it has a potentially global reach. How this Directive is perceived and how jurisdictions in other regions interact will have implications for the investment management industry as a whole.19 In the U.S. the Dodd-Frank Act will not only affect banks and insurers but will also have a significant impact on investment managers. A Chief Compliance Officer (CCO) has to be appointed if not already in place. The CCO will be responsible for managing an organisation’s overall governance framework and compliance program. This framework and program will be subject to regular review by the Securities and Exchange Commission (SEC). Furthermore, the role and accountability of boards and directors will need to be made more transparent.

Compared with Europe and North America, investment management regulation in Asia is perceived to be scattered. The diversity in regulation provides a challenge as regulators in this region take different approaches. The wave of reforms in North America and Europe is likely to influence both Asia and Australia, although the latter will be directly affected by imminent capital and liquidity framework reforms. Some countries will have to adapt less than others: China for example is not expected to undergo major differences as their regulation is more domestic focused.

19

Mastering the AIFMD Challenge| A survey, Deloitte EMEA Asset

Management, Luxembourg 2011.

Growth through Model Control Global Model Practice Survey 2011

33

“In the next five years there will be an increased emphasis on the model validation framework and governance.”

Investment manager, Australia Improvement of model governance Investment managers will be required to improve their governance and have their models independently validated on a regular basis. Similar to banks and insurers, regulators within the investment management industry will encourage greater transparency and accountability. One of the main components of the Dodd-Frank Act is to examine and enhance governance and fiduciary responsibility. In Europe the ESMA 10-788 guidelines has several implications on the governance surrounding models owned by UCITS. Consequently, risk management policies and procedures may need to come into place to enable management of the institution to have a transparent oversight of all risks the organisation is exposed to.

Model validation required The above mentioned ESMA 10-788 guidelines require European investment managers to perform a regular validation of their VaR models. This validation is targeted at mitigating model risk and to increase the validity of models in investment management institutions. A model validation process executed independently from the development process could support ensuring models are conceptually sound and cover all material risks. In the coming years, these validations, to be performed on an on-going basis, may need to be integrated into the organisation’s risk framework. Since model validation has not been high on the agenda to date, this integration will have a considerable impact on UCITS’ internal organisation.

Given the above mentioned wave of regulatory changes, the investment management industry will undergo considerable adjustments in the coming years. Some regions and products will become more attractive. For many institutions the key challenge will

34

be to comply with the new regulations and turn it into a competitive advantage. One of the focus areas of banks and insurers over the past years has been to become compliant with more stringent regulation.

As the regulatory landscape for investment managers currently faces a similar degree of reforms, investment managers will have to catch up with banks and insurers in the coming years, particularly with regard to model validation and governance.

Conclusion

For most practitioners active in the financial services industry the size, impact and timing of the crisis on the financial markets came as a surprise. The use and governance of models as well as the decisions based on the outcome of these models have come under close supervision by regulatory authorities. The financial crisis elucidated that understanding the underlying assumptions and limitations of risk models are crucial to the success and survival of institutions in the financial services industry.

Establishing a sound model governance framework contributes to reducing model risk A starting point to understand the assumptions and limitations of the models could be to put an effective model governance framework in place. Incorporating a transparent organisational structure with clear segregation of responsibilities is an important element of a sound governance framework. Other important features include written and implemented policies and a properly maintained model inventory. Documenting the activities and decisions made during the model development process, compliant with the policies of the institution and regulation in force, could provide a sound basis.

Furthermore, regulators require, or soon will require models to undergo regular validation by an independent model validation department or external party. All these elements contribute to reducing model risk.

Model practice policies could be improved There are possibilities to improve the quality of policies that are part of the model governance framework. Up to half of the respondents indicate that one or more of the identified policies do not exist or are of poor quality. The largest improvements can be made in the areas of expert judgment, model implementation and data quality policies. Model practice policies are not always given the appropriate level of priority. We believe that concise and clear policies that have arisen through support of the stakeholders may lead to consistent models, created through an approach that meets the pre-defined internal leading practice.

New model requirements imposed by regulators Recently, regulatory authorities have been encouraging financial institutions to improve the governance of risk related models and to give model validation a more prominent role in the risk management function. A regular cycle of model validation is required. Therefore

it could be desirable for financial institutions to more thoroughly evaluate the performance and stability of their risk related models, and verify compliance with regulatory capital requirements. Regulatory authorities also prescribe an active involvement of senior management, ensuring systems operate properly and resources are available.

Model governance framework is considered to be fairly complex Currently, the model governance framework is considered to be fairly complex, possibly aggravated by historical mergers and acquisitions. Challenges for the coming years are increasing the consistency and reducing the complexity of the model governance framework. Clear oversight of all models within the organisation, including their status, seems to be a desirable asset. A model inventory provides a solution to model environment complexity.

The ownership of the model governance framework is not always safeguarded Ideally, accountability of the model governance ownership should be placed with a single stakeholder. However, survey results indicate that accountability is not always considered transparent, as 20 per cent of respondents indicate that multiple committees have ownership of the model governance framework. Furthermore, 20 per cent of respondents indicate that ownership of the model governance framework is assigned to the model development or model validation departments.

While these departments have strong knowledge of the technical aspects of the models, we believe assigning ownership to a stakeholder that is part of the process may not lead to a transparent model governance framework, since the responsibilities of these stakeholders are described in the framework itself.

Modellers spend much time on data collection and preparation In order to generate valuable modelling results, input data should be of good quality. The survey results show that model developers spend a relatively significant amount of time on retrieving, preparing and cleansing the data. However, spending much time on data may be an undesirable situation as model developers have the largest added value when they can apply their skills and knowledge to the actual modelling.

Growth through Model Control Global Model Practice Survey 2011

35

Closer collaboration between the model development department and the IT department could provide a solution to the capacity constraints. This collaboration could be formalised by means of a Data Delivery Agreement or a Service Level Agreement between both departments.

The survey results indicate that the data quality policy may be improved; many financial institutions still struggle with this subject. Defining the objectives for a policy could help the stakeholders in the model development process to move consistently and transparently in the same direction.

Processes related to the development and use of models may be standardised to a larger extent, especially with regards to data quality.

Activities in order to enhance model practice are expected to increase In the coming years the activities for the model development department and the model validation department are expected to increase. Moreover, regulatory authorities are expected to initiate new and more stringent requirements on model performance.

Given that financial institutions already face capacity constraints, it may be a challenge in the coming years to complete required validation processes and model improvements on time, let alone research and innovate new methodologies.

Model validation more enhanced at banks compared to insurers The maturity of the model validation function is more enhanced at banks compared to insurers. Model validation departments at banks are larger and on average it takes more time to validate a model. Larger model validation time spans at banks relative to insurers could be due to more thorough analyses or more complex models. A related explanation could be the fact that regulators have been requiring banks to validate their models for years, which is not the case for insurers. However, with Solvency II becoming effective in 2013 insurers will have to validate their models on a regular basis as well.

Rejecting models is challenging for model validation Model validation departments face challenges in the areas of data, planning and resourcing. Remarkably, rejecting models is also a challenge for some model validators. This finding is supported by the observation 36

that 14 per cent of the model validation departments have never rejected a regulatory model. This could have several reasons. Model validation departments could face internal political pressure to accept models as rejections are deemed non-beneficial for the organisation in the short term. Also, it could be the case that model developers have a more specific knowledge than model validators, whose knowledge may be more general. This makes it more difficult to reject sophisticated models. Furthermore, model development and validation departments might be coordinating their activities, which could potentially impair the independence of the latter department. We believe that model owners should convince the model validators that the model is fit for the intended use.

Internal audit faces an increase in regulatory requirements Similar to model development departments and model validation departments, internal audit faces an increase in regulatory requirements. This is confirmed by respondents, as the approach and findings of internal audit are mentioned to be the same or stricter.

Internal audit appears to pay more attention to model governance, and less to the appropriate use and development of models and the model inventory.

In the survey the approach and findings of the supervisor are generally mentioned to be stricter than previously. Since regulation in the financial services industry is going to be more stringent in the coming years, this effect is expected to increase.

Investment managers face more stringent regulation The regulatory landscape for investment managers faces a similar degree of reforms as banks and insurers.

European investments managers will have to comply with UCITS IV, including the compulsory validation of certain VaR models. U.S. investment managers will have to comply with the Dodd-Frank Act which will provide model practice challenges particularly with regards to transparent governance and the development of new reporting models. Although the exact implications of the new regulations are not clear yet, it is evident that it will have a considerable impact on model practice in the investment management industry. Organisations that are able to adapt quickest to the post-crisis financial environment may obtain strategic advances.

Appendix — Key criteria for sound model practice

The GMPS provides insight to what extent the different aspects of model practice are covered by financial institutions and to what degree attention is dedicated to current on-going trends. The survey was conducted during the second half of 2011 and had the participation of 89 financial institutions from all around the world.

The survey results illustrate that respondents face difficulties with enhancing their model practice to a sound standard. Based on this generic observation our modelling team has designed a comprehensive set of key criteria. For your convenience, we have converted these criteria into a leading practice checklist, which may serve as a handle for enhancing your model practice standards. The checklist consists of three focus areas: governance, the design and the input data of models.

Governance • Does the model governance structure define appropriate level of authorities?

• Is independence between validation versus development, implementation and use of models safeguarded?

• Are model developers and validators mainly occupied with their core business (i.e., not too much time spent on data retrieval, data cleansing or other peripheral elements)?

• Are models subject to regular review by internal audit?

• Is there an up-to-date overview of presently used models in your organisation listing their purpose, validation status and supervisory approval?

• Is the knowledge of the development of the model safeguarded (i.e., documentation and human resources)?

• Is the purpose of the model documented?

• Are the assumptions and limitations of the model clear to all stakeholders?

• Besides trying to approach reality with the model, do end-users of the models have a clear understanding of the accuracy and reliability of the model outcome?

Design • Are the models compliant with current and upcoming regulation?

• Are documented procedures in place regarding development, validation, implementation and use of models?

• Can the model be implemented based on the documentation available (without additional instructions from the author of the document)?

• Can the model be enhanced (i.e., due to regulatory changes) in an obvious way?

• Does assurance exist by the risk management or internal audit department at your organisation that the model is fit for the intended use?

• Does the model have an appropriate user interface which ensures users cannot alter the construction of the model?

• Has the model been benchmarked with the results of external models that serve a similar purpose?

• Is the model back tested on a periodical basis?

• Is the model stress tested (i.e., sensitivity tested due to deviating circumstances)?

• Do you have a clear list of requirements for vendor models?

Data • Is sufficient historical data available?

• Does the data used to calibrate or test your models contain exceptional (but plausible) events?

• Are there policies and procedures in place for data management and do these policies include responsibilities and accountability for senior management?

• Is the consistency of source systems controlled in accordance with documented procedures?

• Is data quality regularly reported to your senior management?

• Has the data (source) used by your models been independently validated?

• Have data delivery agreements with data owners been formalised and embedded in your organisation?

• Do you have a clear list of requirements for vendor data?

Growth through Model Control Global Model Practice Survey 2011

37

Contributors

Argentina

Simon R. Walpole

Luxembourg

New Zealand

Claudio Fiorillo

Partner

Xavier Zaegel

Charles Hett

Partner

+85 2 2238 7229

Partner

Partner

+54 11 4320 4018

siwalpole@deloitte.com.hk

+35 24 5145 2748

+64 4470 3866 Ext. 4866

xzaegel@deloitte.lu

charleshett@deloitte.co.nz

cfiorillo@deloitte.com China, Hong Kong S.A.R. Australia

Maria Xuereb

Johnny Yip Lan Yan

Richard Kirkland

Peter Matruglio

Partner

Partner

Partner

Partner

+85 2 2852 1008

+35 24 5145 2489

+64 4470 3711 Ext. 4711

+61 2 9322 5756

marxuereb@deloitte.com.hk

jyiplanyan@deloitte.lu

richardkirkland@deloitte.co.nz

pmatruglio@deloitte.com.au Dubai

Mexico

Norway

Wendy Yip

Fadi Sidani

Miguel Angel Garcia

Henrik Woxholt

Director

Partner

Martinez

Partner

+61 2 9322 5198

+971 4369 8999

Partner

+47 2327 9342

wyip@deloitte.com.au

fsidani@deloitte.com

+52 81 8133 7314

hwoxholt@deloitte.no

miggarcia@deloittemx.com Panama

Belgium

France

Arno De Groote

Herve Phaure

The Netherlands

Bismark Rodriguez Lopez

Director

Partner

Ton Berendsen

Partner

+32 2800 2473

+33 1 5561 2301

Partner

+507 303 4100

adegroote@deloitte.com

hphaure@deloitte.fr

+31 8 8288 4740

brodriguezl@deloitte.com

tberendsen@deloitte.nl Saudi Arabia

Canada

Germany

Mahdi Amri

Thomas Siwik

Twan Kilkens

Aejaz Ahmed

Partner

Partner

Partner

Partner

+1 514 393 6578

+49 211 8772 2147

+31 8 8288 5219

+966 5 4640 7650

mamri@deloitte.ca

tsiwik@deloitte.de

tkilkens@deloitte.nl

aeahmed@deloitte.com

Leon Bloom

Ireland

Michiel Lodewijk

South Africa

Partner

Sinead Kiernan

Director

Pravin Burra

+1 416 601 6244

Director

+31 88 288 3192

Director

lebloom@deloitte.ca

+35 3 1417 2897

mlodewijk@deloitte.nl

+27 1 1209 8118 pburra@deloitte.co.za

sikiernan@deloitte.ie Arjen Pasma

Chile Jaime Barra

Italy

Director

George Cavaleros

Senior Manager

Giorgio Bonanni

+31 88 288 5547

Partner

+56 2729 8682

Partner

apasma@deloitte.nl

+27 2 1427 5730

jbarra@deloitte.com

+39 06 4780 5425 gbonanni@deloitte.it

Director

Jaco van der Merwe

Senior Manager

Japan

+31 88 288 5220

Director

+56 2729 8682

Takashi Miyauchi

eschnezler@deloitte.nl

+27 1 1209 8163

iukrow@deloitte.com

Partner

Ian Ukrow

javandermerwe@deloitte.

+81 3 6213 1162

Pelle van Vlijmen

China

takashi.miyauchi@tohmatsu.

Director

Alvin Chung Hon Ng

co.jp

+31 88 288 5072

Partner +86 10 8520 7333 alvng@deloitte.com.cn 38

gcavaleros@deloitte.co.za Eelco Schnezler

pvanvlijmen@deloitte.nl

co.za

Contributors (cont.)

Andy Rayner

Switzerland

Tunisia

United States

Director

Philipp Keller

Damien Jacquart

Edward T. Hida II, CFA

+27 2 1427 5360

Partner

Director

Partner, Global Leader — Risk

arayner@deloitte.co.za

+41 4 4421 6290

+33 1 5561 6489

& Capital Management

phkeller@deloitte.ch

djacquart@deloitte.com

+1 212 436 4854

Partner

Taiwan

Turkey

+27 1 1209 6104

Benson H. Cheng

Ayse Epikman

Mike McLaughlin

cstretton@deloitte.co.za

Partner

Partner

Partner

+886 2 2545 9988 Ext: 7843

+90 21 2366 6079

+1 312 486 4466

Singapore

bensonhcheng@deloitte.

aepikman@deloitte.com

mikemclaughlin@deloitte.com

Alec Kourloukov

com.tw

ehida@deloitte.com

Catherine Stretton

United Kingdom

Director +65 6538 6166

Thailand

Thomas Clifford

akourloukov@deloitte.com

Suttharug Panya

Senior Manager

Partner

+44 20 7303 6378

South Korea

+66 2676 5700 Ext.5247

tclifford@deloitte.co.uk

Jeong Kee Kim

spanya@deloitte.com

Partner

Tim Thompson

+82 2 6676 3815

Partner

jeonkim@deloitte.com

+44 20 7007 7241 tthompson@deloitte.co.uk

Acknowledgements This report is a result of a team effort that spanned our global Financial Services Industry.

The following individuals performed and coordinated the analysis: Michiel Lodewijk Deloitte Financial Risk Management, The Netherlands Tjeerd Degenaar Deloitte Financial Risk Management, The Netherlands Wouter Kallenberg Deloitte Financial Risk Management, The Netherlands Stefan Boot Deloitte Financial Risk Management, The Netherlands

Growth through Model Control Global Model Practice Survey 2011

39

Deloitte Touche Tohmatsu Limited and DTTL Member Firm Risk & Capital Management Contacts

Argentina

Canada

Germany

Japan

Claudio Fiorillo

Leon Bloom

Jรถrg Engels

Shigeru Furusawa

Partner

Partner

Partner

Partner

+54 11 4320 4018

+1 416 601 6244

+49 211 8772 2376

+81 3 6213 3160

cfiorillo@deloitte.com

lebloom@deloitte.ca

jengels@deloitte.de

shigeru.furusawa@tohmatsu.

Australia

Chile

Thomas Siwik

John Kidd

Pablo Herrera

Partner

Daisuke Kuwabara

Partner

Partner

+49 211 8772 2147

Partner

+61 3 9671 7357

+56 2729 8287

tsiwik@deloitte.de

+81 3 6213 3525

jkidd@deloitte.com.au

paherrera@deloitte.com

co.jp

daisuke.kuwabara@tohmatsu.

Ireland

China

Martin Reilly

Partner

Jason Shigang Li

Partner

Shigeru Omori

+61 2 9322 5756

Partner

+35 3 1417 2212

Partner

pmatruglio@deloitte.com.au

+86 10 8520 7012

mreilly@deloitte.ie

+81 3 6213 3170 shigeru.omori@tohmatsu.

jasonlishigang@deloitte.

Austria

com.cn

India

co.jp

Abhay Gupte

Dominik Damm Partner

Alvin Chung Hon Ng

Senior Director

Tsuyoshi Oyama

+43 15 3700 5400

Partner

+91 22 6681 0600

Partner

ddamm@deloitte.at

+86 10 8520 7333

agupte@deloitte.com

+81 3 6213 1945 tsuyoshi.oyama@tohmatsu.

alvng@deloitte.com.cn Naru Navele

Belgium

co.jp

Arno De Groote

China, Hong Kong S.A.R.

Partner

Director

Maria Xuereb

+1 973 602 6801

Luxembourg

+32 2800 2473

Partner

nnavele@deloitte.com

Laurent Berliner

adegroote@deloitte.com

+85 2 2852 1008 marxuereb@deloitte.com.hk

Brazil

Partner Indonesia

+35 24 5145 2328

Claudia Lauw

lberliner@deloitte.lu

Anselmo Bonservizzi

Denmark

Partner

Partner

Jens Peter Hoeck

+62 21 2992 3100 Ext.33999

Xavier Zaegel

+55 11 5186 6033

Partner

clauw@deloitte.com

Partner

abonservizzi@deloitte.com

+45 3610 3426 jhoeck@deloitte.dk

+35 24 5145 2748 Italy

xzaegel@deloitte.lu

Pierluigi Brienza

Ives Pereira Muller Partner

France

Partner

Malaysia

+55 19 3707 3009

Marc van Caeneghem

+39 06 4780 5412

Steven Lim

imuller@deloitte.com

Partner

pbrienza@deloitte.it

Executive Director +603 7723 6515

+33 1 5561 6588 Rodrigo Mendes Duarte

40

co.jp

Peter Matruglio

mvancaeneghem@deloitte.fr

Paolo Gianturco

Partner

Partner

+55 11 5186 6206

+39 02 8332 3209

rodrigomendes@deloitte.com

pgianturco@deloitte.it

stevenlim@deloitte.com

Deloitte Touche Tohmatsu Limited and DTTL Member Firm Risk & Capital Management Contacts (cont.)

Mexico

Poland

Switzerland

United States

Miguel Angel Garcia

Zbigniew Szczerbetka

Philipp Keller

Edward T. Hida II, CFA

Martinez

Partner

Partner

Partner, Global Leader — Risk

Partner

+48 2 2511 0799

+41 4 4421 6290

& Capital Management

+52 81 8133 7314

zszczerbetka@deloitte.com

phkeller@deloitte.ch

+1 212 436 4854

Singapore

David Streliski

Miguel Hernandez

Tse Gan Thio

Partner

Robert Maxant

Director

Executive Director

+41 2 2747 1900

Partner

+52 55 5080 6295

+65 6216 3158

dstreliski@deloitte.ch

+1 212 436 7046

mihernandez@deloittemx.

tgthio@deloitte.com

ehida@deloitte.com

miggarcia@deloittemx.com

rmaxant@deloitte.com Taiwan

com Spain

Benson H. Cheng

Alok Sinha

The Netherlands

Rafael Campo Bernad

Partner

Principal

Ton Berendsen

Partner

+886 2 2545 9988 Ext. 7843

+1 415 783 5203

Partner

+49 1514 5000 Ext. 1488

bensonhcheng@deloitte.

asinha@deloitte.com

+31 8 8288 4740

rcampobernad@deloitte.es

com.tw

tberendsen@deloitte.nl South Africa

Thailand

Twan Kilkens

Wayne Savage

Suttharug Panya

Partner

Partner

Partner

+31 8 8288 5219

+27 1 1209 8082

+66 2676 5700 Ext.5247

tkilkens@deloitte.nl

dsavage@deloitte.co.za

spanya@deloitte.com

New Zealand

South Korea

United Kingdom

Richard Kirkland

Jung In Lee

William Higgins

Partner

Partner

Lead Partner Risk & Regulation

+64 4470 3711 Ext. 4711

+82 2 6676 1312

+44 20 7303 2936

richardkirkland@deloitte.co.nz

junginlee@deloitte.com

whiggins@deloitte.co.uk

Philippines

Jeong Kee Kim

Vishal Vedi

Diane Yap

Partner

Partner

Partner

+82 2 6676 3815

+44 20 7303 6737

+63 2812 0535 Ext. 9053

jeonkim@deloitte.com

vvedi@deloitte.co.uk

dyap@deloitte.com

Growth through Model Control Global Model Practice Survey 2011

41

42

Models are at the heart of organisations within the financial services industry, since they can be powerful tools in assessing risk and facilitating decision making.

Growth through Model Control Global Model Practice Survey 2011

43

Deloitte refers to one or more of Deloitte Touche Tohmatsu Limited, a UK private company limited by guarantee, and its network of member firms, each of which is a legally separate and independent entity. Please see www.deloitte.com/about for a detailed description of the legal structure of Deloitte Touche Tohmatsu Limited and its member firms.

This publication contains general information only, and none of Deloitte Touche Tohmatsu Limited, its member firms, or their related entities (collectively, the “Deloitte Network”) is, by means of this publication, rendering professional advice or services. Before making any decision or taking any action that may affect your finances or your business, you should consult a qualified professional adviser. No entity in the Deloitte Network shall be responsible for any loss whatsoever sustained by any person who relies on this publication. © 2012 Deloitte The Netherlands

2.601.014

Deloitte provides audit, tax, consulting, and financial advisory services to public and private clients spanning multiple industries. With a globally connected network of member firms in more than 150 countries, Deloitte brings world-class capabilities and high-quality service to clients, delivering the insights they need to address their most complex business challenges. Deloitte’s approximately 182,000 professionals are committed to becoming the standard of excellence.