May 28, 2014

In a previous Perspectives article, we explored how model output has become the currency by which catastrophe risk is transferred and why stability and trust are necessary for this currency to hold value. This time, we'll look at the concept of model flexibility and discuss whether and to what extent flexibility might call into question the value of model output as currency.

From the viewpoint of a model builder, flexibility is a given. On a regular basis, we review new scientific research and data, choose from among various assumptions, learn from recent catastrophes, and take advantage of improved technology to make measured changes to our models. Over time, this flexibility provides the means to derive progressively more realistic representations of risk.

But what about flexibility for the model user? The imperative for companies to truly understand their analytical tools and the output they produce means that they are more engaged with the modeling process than ever before. Some companies want the flexibility to adjust individual model components, to modify the output, or to implement a multi-modeling strategy within a single software platform. What are the implications, and what might the "right" amount of flexibility be?

The Why

The fundamental reason why an insurer or reinsurer would want flexibility—one that modeling companies themselves readily acknowledge—is that models are not perfect, that is, they do not represent reality perfectly. There is ever-present uncertainty inherent throughout the modeling process as well as various sources of loss that we are not yet able to model. Based on claims experience, internal research, or the desire to perform sensitivity testing, a company may want to examine different perspectives or different views of risk than that of the default model. A few examples of changes to model assumptions and parameters include:

—Alternative hurricane landfall frequencies for the U.S. Gulf Coast
—A different weighting of ground motion prediction equations for intraplate regions of Australia
—Customized vulnerability functions for light metal commercial construction in Japan

Companies may also wish to modify losses to reflect their internal view that the model is overstating or understating risk for certain types of events, specific regions, lines of business, construction classes, etc. Another objective might be to capture non-modeled sources of loss—to account for loss adjustment expenses, for example.

Our Position

Our perspective on the matter is simple. We recognize that no single model, AIR's or any other, represents "the answer." Rather, each model is our best formulation based on the culmination of what we currently know, and we believe that it provides the most reliable estimate of potential losses. However, there may be scientifically plausible alternatives to various elements of a given model, and we encourage companies to continually scrutinize and challenge the models they use.

AIR's mission is to empower our clients to truly own their risk. This means we are committed to delivering the flexibility that helps each company use their unique insight and experience in making decisions that make sense for their business. For example, we are rolling out the ability to modify ground-up losses in Touchstone®—a capability that is currently available in CATRADER®. Companies can use these loss adjustment factors to test the sensitivity of net and gross losses to ground-up losses and to account for non-modeled losses in light of their claims data. However, the AIR view of risk will always be preserved and provided as a benchmark alongside the modified view.

Looking ahead, we will provide additional flexibility to adjust AIR's view of hazard and vulnerability. In addition, Touchstone was architected to be an open platform, and we are working with a number of third-party solution providers to enable their views of risk to be run alongside the AIR view. Third-party models and data can provide risk assessment capabilities where detailed probabilistic models do not yet exist, or they can offer alternative perspectives to existing AIR models. We view multi-modeling not only as a scientifically rigorous way to address uncertainty but also as a catalyst for continued innovation and healthy competition in the market.

The Challenges—and Our Responsibility

However, there are challenges ahead as modeling firms look to deliver the flexibility that users are seeking. A core responsibility is to maintain a scientifically credible, internally consistent view of risk. It is relatively easy to know when a model adjustment has gone too far beyond realism (for example, modifying the landfall frequency for Category 4 hurricanes to exceed that of Category 2 hurricanes for a stretch of U.S. coastline), but there are more nuanced considerations when we think about the optimal amount of flexibility to provide within our platform.

Many clients, for example, are surprised to learn that up to half the time we spend on model development is actually spent on model validation. The ultimate goal when we undertake validation is to ensure that—no matter what exposure data are input into the model—the final loss output is robust and defensible. The internal workings of the model represent a stable ecosystem where each component has been carefully balanced to work with the others to produce the best possible estimates of loss potential.

When undertaking a major model update (a decision never taken lightly), the model vendor must open, shake up, and rebalance this ecosystem. Thus, a model user who wishes to "switch out" a component with an alternative view, while leaving the rest of the model unchanged, should be fully aware that such mixing and matching may degrade the reliability and robustness of their final loss estimates.

For example, if one set of damage functions that rightly reflect not just engineering expertise but also loss adjustment practices is replaced by an alternative set that have not been so calibrated, the resulting losses will not make sense. Similarly, if the model's damage functions have been calibrated to one hurricane wind field formulation, model output will not be useful if another wind field formulation is substituted, unless the damage functions, too, are recalibrated.

For companies to own their risk, then, they must own the validation of their modified model output as well, using sufficient data and research that the case can be made to rating agencies. As vendor models are updated, the validation must be redone.

In Closing

In a world continually moving toward open everything, AIR recognizes that flexibility is the new standard, and we are committed to working with our clients—as partners—to figure out how to responsibly deliver the flexibility they need. To be sure, not all companies want or need all the flexibility that we plan to offer. For the ones that do, a key element to ensuring that the implementation of flexibility is beneficial—and not counterproductive—is a commitment to model education. Before modifying a model, the user must have a thorough understanding of the data that went into it, its statistical underpinnings, how its components work together—as well as an intuitive sense for the model as a whole.

The research that goes into our models is world class, and we believe there is no better representation of risk available. That's why, as AIR provides flexibility in our own software platform, the original AIR view of risk will always be provided alongside any modified perspective. However, we also recognize that the randomness and complexities associated with catastrophes cannot be reduced to mathematical certainty. Thus, we want to provide the tools to empower our clients to develop their customized view of risk—responsibly.

In the final analysis, we must go back to the concept of using model results as currency in the risk transfer chain. In real-world financial transactions, the exchange of currency is possible because both parties have a shared understanding of its value. The same is true in catastrophe risk transfer. When modifications are made to the model or model output by one party to the transaction, the confidence that comes with using a trusted currency can be shaken. The modified view of risk needs to be clearly documented and justified when model results are shared with both internal and external stakeholders, including regulators who will undoubtedly ask ever harder questions.

S. Ming Lee  

by Ming Lee
CEO and President
AIR Worldwide


You might also be interested in:



You’re almost done.
We need to confirm your email address.
To complete the registration process, please click the link in the email we just sent you.

Unable to subscribe at this moment. Please try again after some time. Contact us if the issue persists.

The email address  is already subscribed.