Stress testing – lessons from the pandemic

The COVID 19 pandemic will inflict a lot of damage on our economy and society. The only upside is to learn something from it and come away stronger. With respect to stress testing, some of these lessons just reinforce things we already knew, in particular a proper understanding of:

• The potential for IFRS 9 to amplify the procyclicality of the banking system,
• The zone of validity of the modelling employed in stress testing, and
• What a bank can do on its own to deal with the inherent uncertainty associated with projecting the range of potential future outcomes

But there is another insight (an “inconvenient truth?”) that is obvious in hindsight but sometimes denied on the basis of the risk of moral hazard. COVID 19 reminds us that there are limitations to what a bank can achieve with its own resources and acting independently of other financial institutions. The task gets even harder when banks are competing, as they are meant to do, with each other and the shadow banking sector and while community trust in the financial system is very low.

This I think is an important lesson from the pandemic. There is a class of scenario that is so deeply systemic that the solutions require cooperation and coordinated responses led by the government or the official family of regulators responsible for the financial system. This is something that banks and supervisors may be reluctant to acknowledge lest it be construed to be expecting or encouraging a bail out, but true none the less.

If you are time poor then stop here. If you have the time and interest then read on for a longer discussion of these issues.

Managing risk and uncertainty

An earlier post discussed the way in which COVID 19 is the first real rest of the IFRS 9 (and CECL) approach to loan loss provisioning introduced in response to some of the lessons learned in the GFC. I argued that

• IFRS 9 by itself will not address the core lesson the GFC taught us about being pre-positioned to deal with uncertainty and
• that the pandemic will hopefully draw attention to the need for more thought on how the Capital Buffer framework can be better designed to absorb the potentially dangerous positive feedback loop that IFRS 9 introduces.

This post looks at what insights the pandemic offers on the use of stress testing to calibrate bank capital requirements. Let me state at the outset that I do not seek to downplay the value of stress testing. To the contrary, I see huge value in stress testing and was happy for it to play a central role in the ICAAP’s that I have been involved with.

That said, I also see stress testing being used in ways that do not respect the inherent limitations of the modelling process. I suspect that the “zone of validity” for this kind of work probably caps out around a 1:25 year scenario; roughly what you might expect for a severe recession impacting a basically sound banking system. Banks obviously need more capital than this but we also need to guard against the risks of using probability to disguise uncertainty (see here, here and here). Honestly acknowledging the limitations of what we can know and incorporating a margin of safety to absorb what we don’t know (“unknown unknowns”) is an intellectually more honest approach rather than pretending (or even worse believing) that a bit of statistical modelling will give us true insight and control.

Historical stress results are also applied in ways that are inconsistent with the current risk profile of the organisation and the external environment in which the bank is operating. I have previously argued that the kinds of really severe historical downturns typically used to calibrate capital buffers are usually associated with conditions where endogenous weaknesses within the banking system are a key element in explaining the extent of the asset price declines and weak recoveries.

This endogenous/exogenous distinction is especially important when using historical episodes to calibrate how much capital banks need to hold. We possibly tend to take flexible exchange rates and sensible policy responses for granted but the severity of some historical scenarios was arguably exacerbated by unhelpful exchange rate coupled with monetary and fiscal policy settings that could not respond to the stress or were prevented from doing so.

This is not to say that these historical episodes are irrelevant or an argument against using very severe downturns to calibrate the resilience of capital buffers in the banking system. The argument that “this time is different” should always be treated with suspicion if not disdain. My argument is simply that distinguishing the exogenous and/or exogenous components of a stress outcome is a better foundation for interpreting stress test outcomes. I would hazard a guess that it is almost essential for reconciling the understandably conservative supervisory approach with internal stress testing which is based on what management typically assume to be essentially soundly managed risk positions but supervisors implicitly assume are not.

The core point in these previous posts remains relevant but the pandemic offers another perspective on stress testing that I want to explore in this post

Some basic principles

Stress testing is generally predicated on the idea that the size of the scenario impact increases in inverse proportion to the likelihood of the scenario. So a 1:25 year scenario should have a larger impact on profit and capital than a 1:10 year and a 1:100 or worse scenario should be larger again. The size of the impact is also likely to increase in a non-linear manner as the probability of the outcome declines.

The response to the pandemic however has the potential to bend that stress testing rule of thumb.

Arguably we are looking at a 1:100 year style event with COVID 19 (at a minimum 1:50 year) but the weight of public resources being thrown at the problem means that the impact on bank capital buffers might not be much worse than a severe recession which is typically thought of as something like a 1:25 scenario. That is the plan in any case; we are throwing everything at the problem to mitigate the impact both on individuals but also on the economic infrastructure that generates employment, spending and profits. The government response matters a lot in these scenarios. Banks acting alone can be strong enough to be part of the solution but capital alone does not fix the underlying issue.

This is something that a thoughtful analysis of stress testing outcomes has probably long recognised but not something you want to put in writing in case it can be misconstrued to imply that the bank is expecting a bail-out.

The pandemic is clearly an exogenous shock for the banking system and the economy as a whole. Unlike the GFC, the banking system has done nothing to contribute to the severity of the impact of the pandemic.       

Supervisors have indicated that this is the time for the capital and liquidity buffers built up in response to the GFC to be used but I don’t see any sign of doubt that Australian banks are strong enough to ride out this shock (This is NOT financial advice). In fact, this time round, the banks seem set to be a big part of the solution and are willing to bear the costs to shareholders that involves. Witness the various programs of forbearance that banks have volunteered to help customers get through the crisis.

The question in stress testing is where to draw the line on what external support can be expected without creating moral hazard. There seems to be a boundary in the loss distribution where an exogenous shock increasingly demands cooperative solutions and some degree of public support is necessary. The issue with moral hazard remains and there should be consequences for banks whose approach to risk has contributed to the problem. At the same time there are limits to what individual self discipline can achieve.

This is not just saying that stress testing analysis should incorporate reliance on a bail-out. It is saying that we need to be realistic and honest about this rather than pretending that the market based economic system can function in these kinds of scenarios without any reliance on government bodies.

Proponents of the big capital thesis (see here) that emphasises large amounts of common equity might feel vindicated by a large external shock but I am not sure that the degree of government support required to deal with this scenario is something that fits neatly with their world view. I also have in mind the 1:200 year benchmark that the RBNZ attempted to employ when calibrating the capital requirements review. This is partly based on my “Zone of Validity” concern with modelling noted above but it is also far from clear that holding more capital would have substantially changed the play book we see being employed to deal with the consequences of the pandemic.

This view should not be confused with arguing that strong capital does not matter. I am 100% behind the idea of banks being held to account against a benchmark of being “Unquestionably Strong”. Holding capital consistent with this benchmark clearly helps reduce the risk to government of any liquidity support offered but it does not really change the fact that liquidity support will be required under some scenarios. The whole idea of Lender of Last Resort (LOLR) was developed during a period when banks were supposedly much better capitalised so liquidity risk seems to be a feature of the system not a bug. And to be clear, the support banks are getting to keep the economy functioning and able to recover deals with liquidity issues not solvency. If that is unacceptable then we need to start looking at more radical alternatives (see here and here)

This is a big topic so it is entirely possible that I am missing something. If so please let me know since the whole point of the post is “lessons learned”. In the interim this is my perspective, honestly but “lightly held” as all opinions should be.

Tony – From the Outside