commit 99df0aa9a4db7c7b63ded62844e8a9e7da5b5ba0
parent ecb305ec031a3645c3adac414807598552c06818
Author: Eamon Caddigan <eamon.caddigan@gmail.com>
Date: Sat, 3 Aug 2024 21:56:20 -0700
Add a post about percentage change
Diffstat:
1 file changed, 124 insertions(+), 0 deletions(-)
diff --git a/content/posts/percent-change/index.md b/content/posts/percent-change/index.md
@@ -0,0 +1,124 @@
+---
+title: "Average percentage change is not the percentage change of the average"
+date: 2024-08-03T15:31:38-07:00
+draft: true
+katex: true
+categories:
+- Science
+- Data Science
+tags:
+- Statistics
+---
+
+When I present data to non-technical stakeholders, I sometimes express
+differences (typically changes over time) in terms of _percentage change_.
+This indicator has the advantage of being unitless and normalized[^norm],
+and is familiar to broad audiences (although some aspects are
+counter-intuitive---see below).
+
+The ratio of the difference between two observations (_x₀_ and _x₁_) and the
+starting observation provides the _relative change_:
+
+$$ \frac{x_1 - x_0}{x_0} $$
+
+Multiplying this proportion by 100 yields the percentage change.
+
+Usually I have more than two observations separated by time---I’ll have _x₀_
+and _x₁_ values for many observational units. Consider the following set of
+(made up) test scores:
+
+| Student | Test 1 | Test 2 | Percentage Change |
+|------------:|-------:|-------:|-------------------:|
+| 1 | 75.0 | 81.0 | 8.00% |
+| 2 | 80.0 | 82.0 | 2.50% |
+| 3 | 80.0 | 86.0 | 7.50% |
+| 4 | 96.0 | 90.0 | -6.25% |
+| 5 | 100.0 | 90.0 | -10.00% |
+| **Average** | 86.2 | 85.8 | 0.35% |
+
+The average score on Test 2 was lower than that for Test 1, so one might
+conclude that the students did worse. However, the average percentage change
+of students’ scores is positive; in other words, relative to their starting
+scores, students typically did better on the second test. This toy example
+illustrates an important point: the average of the percentage change (0.35%
+here) is not the same thing as the percentage change of the averages
+(approximately -0.46% in this case).
+
+I find the mathematical definition of these two measures illustrative. I’ll
+stick with relative changes to keep the equations tidier.
+
+The average of the relative change is:
+
+$$ \mathbb{E}\[ \frac{x_1 - x_0}{x_0} \] = \mathbb{E}\[\frac{x_1}{x_0}\] - 1$$
+
+On the other hand, the relative change of the averages is:
+
+$$ \frac{\mathbb{E}[x_1] - \mathbb{E}[x_0]}{\mathbb{E}[x_0]} =
+\mathbb{E}[\frac{x_1}{\mathbb{E}[x_0]}] - 1$$
+
+One way to think about these equations is that the _average relative
+change_ scales each _x₁_ by its associated _x₀_ before calculating the mean
+of the scaled value. The _relative change of the averages_ scales each _x₁_
+by the _mean_ of _x₀_, and then takes the mean of these.
+
+## Which one to use
+
+Since they measure different things, each measure can be appropriate in
+different circumstances.
+
+If _x₀_ and _x₁_ comprise a sample from a larger population, then the sample
+mean of _x₀_ and _x₁_ are reliable estimates of the population mean of these
+values. In this case, the percentage change of the averages is probably most
+appropriate; it will estimate the percentage change in the population.
+
+On the other hand, if the observational units represent an entire
+population, or when using relative change to compare the behavior of
+different measures over the same time period, then calculating
+statistics---including the mean---of the relative changes is useful.
+
+## Why I’m sharing this
+
+Perhaps this distinction is obvious to most folks[^folks], especially since
+there are other transformations that behave similarly[^transform]. As
+somebody accustomed to wanting the percentage change of the averages,
+I recently saw---in real world data---a fairly large discrepancy between
+that and the average percentage change, after deciding that I needed the
+latter. Only after checking my code for obvious errors did I confirm that
+the math made sense. The experience reminds me of encountering [Simpson’s
+Paradox](https://en.wikipedia.org/wiki/Simpson%27s_paradox)[^simpson] in
+real data.
+
+## Other measures of relative change
+
+I avoid using percentage change as defined here (when I can) because it can
+lead to confusion. Among percentage change’s problems is its lack of
+“symmetry”; for example, an increase from 4 to 5 represents a 25% change,
+while a decrease from 5 to 4 represents a -20% change. This is addressed by
+other measures of relative change, such as the “arithmetic mean change”
+(where the difference is scaled by the mean of _x₀_ and _x₁_ rather than
+_x₀_ alone) or “logarithmic change” (where relative change is represented by
+the natural logarithm of the ratio of _x₁_ to _x₀_). Note that even with
+these measures, the average of relative changes are still distinct from the
+relative change of the averages; however, a symmetrical measure is probably
+a better choice when averaging changes.
+
+## Percentage change in practice
+
+Since percentage change is only a _point estimate_, its presence should
+supplement---and never replace---an appropriate model-based approach to
+estimating the size of the change and the uncertainty of that estimate.
+Still, percentage change is a useful measure to include in reports and slide
+decks when it will be familiar to the audience, which is typically the case
+in a business setting. I hope this helps somebody else select the right
+calculation, and explain why the results are totally different from
+a “wrong” one.
+
+[^norm]: In the colloquial sense, not the statistical one.
+
+[^folks]: At least the ones who’d read a post like this.
+
+[^transform]: For example, the mean of the logarithm of a variable is not
+ the same as the logarithm of the mean of a variable.
+
+[^simpson]: I’m not sure if this could be considered a special case of
+ Simpson’s paradox.
