commit99df0aa9a4db7c7b63ded62844e8a9e7da5b5ba0parentecb305ec031a3645c3adac414807598552c06818Author:Eamon Caddigan <eamon.caddigan@gmail.com>Date:Sat, 3 Aug 2024 21:56:20 -0700 Add a post about percentage changeDiffstat:

A | content/posts/percent-change/index.md | | | 124 | +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ |

1 file changed, 124 insertions(+), 0 deletions(-)diff --git a/content/posts/percent-change/index.md b/content/posts/percent-change/index.md@@ -0,0 +1,124 @@ +--- +title: "Average percentage change is not the percentage change of the average" +date: 2024-08-03T15:31:38-07:00 +draft: true +katex: true +categories: +- Science +- Data Science +tags: +- Statistics +--- + +When I present data to non-technical stakeholders, I sometimes express +differences (typically changes over time) in terms of _percentage change_. +This indicator has the advantage of being unitless and normalized[^norm], +and is familiar to broad audiences (although some aspects are +counter-intuitive---see below). + +The ratio of the difference between two observations (_x₀_ and _x₁_) and the +starting observation provides the _relative change_: + +$$ \frac{x_1 - x_0}{x_0} $$ + +Multiplying this proportion by 100 yields the percentage change. + +Usually I have more than two observations separated by time---I’ll have _x₀_ +and _x₁_ values for many observational units. Consider the following set of +(made up) test scores: + +| Student | Test 1 | Test 2 | Percentage Change | +|------------:|-------:|-------:|-------------------:| +| 1 | 75.0 | 81.0 | 8.00% | +| 2 | 80.0 | 82.0 | 2.50% | +| 3 | 80.0 | 86.0 | 7.50% | +| 4 | 96.0 | 90.0 | -6.25% | +| 5 | 100.0 | 90.0 | -10.00% | +| **Average** | 86.2 | 85.8 | 0.35% | + +The average score on Test 2 was lower than that for Test 1, so one might +conclude that the students did worse. However, the average percentage change +of students’ scores is positive; in other words, relative to their starting +scores, students typically did better on the second test. This toy example +illustrates an important point: the average of the percentage change (0.35% +here) is not the same thing as the percentage change of the averages +(approximately -0.46% in this case). + +I find the mathematical definition of these two measures illustrative. I’ll +stick with relative changes to keep the equations tidier. + +The average of the relative change is: + +$$ \mathbb{E}\[ \frac{x_1 - x_0}{x_0} \] = \mathbb{E}\[\frac{x_1}{x_0}\] - 1$$ + +On the other hand, the relative change of the averages is: + +$$ \frac{\mathbb{E}[x_1] - \mathbb{E}[x_0]}{\mathbb{E}[x_0]} = +\mathbb{E}[\frac{x_1}{\mathbb{E}[x_0]}] - 1$$ + +One way to think about these equations is that the _average relative +change_ scales each _x₁_ by its associated _x₀_ before calculating the mean +of the scaled value. The _relative change of the averages_ scales each _x₁_ +by the _mean_ of _x₀_, and then takes the mean of these. + +## Which one to use + +Since they measure different things, each measure can be appropriate in +different circumstances. + +If _x₀_ and _x₁_ comprise a sample from a larger population, then the sample +mean of _x₀_ and _x₁_ are reliable estimates of the population mean of these +values. In this case, the percentage change of the averages is probably most +appropriate; it will estimate the percentage change in the population. + +On the other hand, if the observational units represent an entire +population, or when using relative change to compare the behavior of +different measures over the same time period, then calculating +statistics---including the mean---of the relative changes is useful. + +## Why I’m sharing this + +Perhaps this distinction is obvious to most folks[^folks], especially since +there are other transformations that behave similarly[^transform]. As +somebody accustomed to wanting the percentage change of the averages, +I recently saw---in real world data---a fairly large discrepancy between +that and the average percentage change, after deciding that I needed the +latter. Only after checking my code for obvious errors did I confirm that +the math made sense. The experience reminds me of encountering [Simpson’s +Paradox](https://en.wikipedia.org/wiki/Simpson%27s_paradox)[^simpson] in +real data. + +## Other measures of relative change + +I avoid using percentage change as defined here (when I can) because it can +lead to confusion. Among percentage change’s problems is its lack of +“symmetry”; for example, an increase from 4 to 5 represents a 25% change, +while a decrease from 5 to 4 represents a -20% change. This is addressed by +other measures of relative change, such as the “arithmetic mean change” +(where the difference is scaled by the mean of _x₀_ and _x₁_ rather than +_x₀_ alone) or “logarithmic change” (where relative change is represented by +the natural logarithm of the ratio of _x₁_ to _x₀_). Note that even with +these measures, the average of relative changes are still distinct from the +relative change of the averages; however, a symmetrical measure is probably +a better choice when averaging changes. + +## Percentage change in practice + +Since percentage change is only a _point estimate_, its presence should +supplement---and never replace---an appropriate model-based approach to +estimating the size of the change and the uncertainty of that estimate. +Still, percentage change is a useful measure to include in reports and slide +decks when it will be familiar to the audience, which is typically the case +in a business setting. I hope this helps somebody else select the right +calculation, and explain why the results are totally different from +a “wrong” one. + +[^norm]: In the colloquial sense, not the statistical one. + +[^folks]: At least the ones who’d read a post like this. + +[^transform]: For example, the mean of the logarithm of a variable is not + the same as the logarithm of the mean of a variable. + +[^simpson]: I’m not sure if this could be considered a special case of + Simpson’s paradox.