Monte Carlo simulation is incredibly popular among advisors today. Most leading industry software (e.g., eMoney, MoneyGuide Pro, RightCapital, Naviplan) will report a ‘probability of success’ metric as the primary focal point for a plan.

Unfortunately, ‘probability of success’ often doesn’t mean what people think it means. I’ve written about a number of these interpretation issues in the past. However, we haven’t had much data to actually rely on in terms of assessing the extent to which advisors understand or misunderstand probability of success measures.

One distinction that is often overlooked is the different levels of risk associated with ‘probability of success’ measures for a one-time plan vs. ongoing planning. While I won’t dive deep into that here (see here for a thorough overview of that issue), the important takeaway is that if advisors understood the difference, then we should see advisors being comfortable with using different thresholds in different planning contexts.

To test that, I ran an experimental survey in October of 2020. Advisors recruited from a Facebook community were given the following prompt and randomly assigned either the “one-time basis” or “ongoing client” treatment groups:

Imagine you are putting together a financial plan for a 65-year-old retiree. Here are some additional details:

• You are meeting with the retiree on a [one-time basis/ongoing client] (i.e., they are not an ongoing client).
• The initial probability of success is very low at the retiree’s preferred spending level.
• You will have to recommend a spending level lower than the client’s ideal spending level to present a prudent spending plan to the client.

Generally speaking, what is the minimum probability of success that you feel would be prudent to recommend in a planning scenario like this? (Please enter a percentage as a whole number between 0 and 100 without a percentage sign.)

I may get around to writing these results up formally at some point, but, in the meantime, here are some summary statistics from this experimental survey:

While directionally some of these results align with what we’d expect if advisors did understand the difference (i.e., one-time plan minimum probability of success levels should be higher), the differences here are not statistically significant.

Another interesting note is that only one advisor (out of 36) in the ongoing planning treatment indicated a willingness to use a 50% probability of success threshold. I’ve written elsewhere about why, for ongoing planning, 50% probability of success is nowhere near as bad as most think. Rather, in an ongoing context, ‘probability of success’ is more akin to placing one’s thumb on the scale either in favor of current income (lower probability of success) or higher legacy values (higher probability of success).

Ultimately, the concept of exactly what ‘probability of success’ means continues to be misunderstood. Not only can the concept be highly confusing to retirees who can’t be faulted for not understanding the nuances of Monte Carlo simulation, but advisors themselves don’t seem to appreciate the differences between the use of Monte Carlo in a one-time plan versus ongoing planning.