This article was initially available on our podcast. Click here to listen.
We’re going to discuss a little bit of a combination of how to read charts and even how to design them so that we can interpret results, diagnose problems, or identify solutions well.
We’re frequently looking for relationships or something in a graphical presentation of data that allows us to understand something quickly and easily. For example, when we’re looking at denials data, one of the questions, one of the central questions that you want to be asking is, “Are denials going up, or are they going down? Or staying the same, of course?” There effectively is not precisely the same. There’s only going up and going down.
Quantify denials
If we’re looking for whether or not denials are going up, we are just looking for, on a quantitative basis, where we have a count of denials. Suppose we see denials going up on a count on a nominal basis. In that case, that still could mean that they’re not going up in the sense that if we have an increase in charges, we would expect a corresponding increase in denials to be received, even if we’re effectively performing at the same level. So an increase in the count in denials isn’t really what we’re looking for. We’re looking for an increase effectively in the percentage rate of denial.
The way we might see that in data is, you can, of course, calculate that percentage and graph that, or you might look at two lines, one of which is the count of denials and one of which is the charges. If they’re both going up or both are going down, they essentially are co-varying, and we do not see any significant change. However, if those lines diverge (one is going up while the other is staying steady or something like that), we see some net change. So it’s the relationship between those two that tells us something.
What about graphs?
The problem is that graphs can be misleading or even deceptive. I’m looking at one of those right now for a client of ours.
There’s another component to this too, which is, in these denials charts, we’re looking at the count of denials. We’re looking at the percentage of denials as well. That way, if we see an increase in the count of denials but we see the percentage staying steady, the increase in denials is just a result of the increased volume of charges.
In this case, what we’re looking at for this client is that both lines are going up. So the count of denials is going up, and the percentage of denials is going up also. If I just looked at that chart, I would reasonably and normally conclude, “Okay, the denials count is going up as a result of the increase in percentages.”
The increases
We have a significant increase in both because this is angled up at something approaching 45 degrees. It’s a steep increase over four months, a very steep increase. The denials count goes from about 4,000 in a month to almost 7,000 denials per month. In the course of four months, that’s a very, very significant increase. And it looks like both of those percentages are tracking the same.
Interpretation
The problem with that interpretation is scale. It looks like these are going in lockstep up. Yet, this is a very misleading chart. By the way, it’s our chart. I’m pointing this out about our graph of the data. Even though these both go up and to the right is almost the same slope (and they both look like they’re going up dramatically), and you focus on the count increase from 4,100 to 6,900 over the course of four months, the percentage is going from 30.6% to 33.0%.
That is an increase from roughly just under 31% to 33%. But because the scale on the right-hand side is going from effectively 30% to 33% and the scale on the left is going from 4,000 to 7,000, those are not going up at the same rate. Not even close. Effectively, the percentage is about the same.
The reason why I say that is because 31% to 33% over the course of four months (again, it’s a whole separate conversation around “Is that high or low?”; we’re not doing that for now) and all staying within that range is not much variance.
We have some confidence interval around these things because the way these are calculated has to do with the volume of charges. Therefore, if you had some slight variation in charges, you’d expect some small variation associated with timing on these percentages as well. I don’t consider 31% to 33% the significant variance. That’s not a trend that I want to pay attention to.
Percentages
Effectively, the percentage rate of denial has been roughly flat, while the actual count has gone up dramatically. That tells me that what’s driving this is a significant increase in volume, not a significant increase in problems causing denials. That’s one of the things that we have to watch out for, which is that even if we have the correct type of graph and the right information in there, the scale can throw us off. This comes into play when we have especially two scales: one on the left and one on the right. Sometimes, log scales are better. Sometimes, the linear scales are better. This is a situation where it’s deceptive just looking at it.
I made an assumption based on the look at the information that I didn’t realize was incorrect until I looked at it more carefully and realized, “Ah, no, there isn’t a significant change in the denial rate. It’s stayed roughly constant over the course of the last four months.” That doesn’t mean there isn’t a problem, meaning it shouldn’t, at an underlying rate, be over 30%. But it hasn’t gotten dramatically worse in the last few months while the graph made it seem that way.
In conclusion
The moral of the story is to design charts that make sure that somebody who casually observes it will understand exactly what’s going on without looking very closely and scrutinizing it to ensure they don’t misinterpret the data. That’s the key. There aren’t easy rules for that. A big one is to look at the scaling and relative scaling: Where’s the zero, the cross points, and things like that? Are you going from zero to something? Are you going from one number to another number? Do you have log scales? I mean, how do you make these so that the casual observer will intuit in a matter of seconds the right information? We failed in this report.
