Any Statisticians Out There?

londonblue · Oct 12, 2015

I have a question for any statisticians.

I am looking at a presentation given to us at work on a particular subject. (I can't go into details because it's confidential.) Anyway the presentation shows 4 user types for a product, but all four are above average users. I asked the people who conducted the research how all four user types can be above average. Their answer is because there is also a fifth group that won't make use of the product at all, and the average includes this group.

My question is would the average therefore be more meaningful if it was calculated excluding the non-users so that you could see who out of the actual users would be above and below average?

I'm not a statistician, and certainly not a marketeer, so I don't know what is standard practice and what isn't.

Anyone able to help?

MK Shrimper · Oct 12, 2015

Sounds like a bit of a manipulation of data to give a preffered output. Standard practice I'd imagine, but not entirely ethical.

RHB · Oct 12, 2015

londonblue said:
I have a question for any statisticians.

I am looking at a presentation given to us at work on a particular subject. (I can't go into details because it's confidential.) Anyway the presentation shows 4 user types for a product, but all four are above average users. I asked the people who conducted the research how all four user types can be above average. Their answer is because there is also a fifth group that won't make use of the product at all, and the average includes this group.

My question is would the average therefore be more meaningful if it was calculated excluding the non-users so that you could see who out of the actual users would be above and below average?

I'm not a statistician, and certainly not a marketeer, so I don't know what is standard practice and what isn't.

Anyone able to help?

Not me I'm afraid, I think you really need mrsblue on this one.

Pubey · Oct 12, 2015

Yeah, if you want to see the types of people who are the highest users of a product, then calculate your average from just people who are using it, otherwise the concept of an average user goes out of the window.

So if your product is an iPhone, and you classify 'use' as 'data used per month', then you have a nice way of measuring use. You can then define broad classes of 'user types'. quite simply this could just be by age. Then you can easily calculate the differences in average usage across the user groups.

However doing this properly is quite difficult, especially if you have more complex definitions of user type, and want to introduce other characteristics to define them by. This can get quite complex, and so you're right to question their methodology. Sounds to me like a marketer messing around in Excel to draw some simplistic conclusions.

londonblue · Oct 12, 2015

Pubey said:
Yeah, if you want to see the types of people who are the highest users of a product, then calculate your average from just people who are using it, otherwise the concept of an average user goes out of the window.

So if your product is an iPhone, and you classify 'use' as 'data used per month', then you have a nice way of measuring use. You can then define broad classes of 'user types'. quite simply this could just be by age. Then you can easily calculate the differences in average usage across the user groups.

However doing this properly is quite difficult, especially if you have more complex definitions of user type, and want to introduce other characteristics to define them by. This can get quite complex, and so you're right to question their methodology. Sounds to me like a marketer messing around in Excel to draw some simplistic conclusions.

Thanks for that. In fairness to them, this is a very complex result set. They're trying to collate responses from 6500 people all across Europe. I personally think the issue is that they (an outside company) were given a steer to ignore the non-users which skewed the results they had already calculated, and didn't think (or have time) to recalculate.

Pubey · Oct 12, 2015

londonblue said:
Thanks for that. In fairness to them, this is a very complex result set. They're trying to collate responses from 6500 people all across Europe. I personally think the issue is that they (an outside company) were given a steer to ignore the non-users which skewed the results they had already calculated, and didn't think (or have time) to recalculate.

To be fair if they've done the analysis properly, it'd take seconds to recalculate by applying a simple filter to your dataset and re-running the code on a stats programme. Sounds more like they've had a complex dataset in Excel and it's all quite messy.

The only thing their analysis sounds useful for is showing the relative ranking of usage between their four groups. That still could be informative, even though the absolute level (usage, or average usage) isn't informative.

More interesting (I'd guess) is looking for systematic differences in people who do use/purchase, and those who don't. My guess is that they'll bill you lots for the answer to that question.

londonblue · Oct 12, 2015

Pubey said:
To be fair if they've done the analysis properly, it'd take seconds to recalculate by applying a simple filter to your dataset and re-running the code on a stats programme. Sounds more like they've had a complex dataset in Excel and it's all quite messy.

The only thing their analysis sounds useful for is showing the relative ranking of usage between their four groups. That still could be informative, even though the absolute level (usage, or average usage) isn't informative.

More interesting (I'd guess) is looking for systematic differences is people who do use/purchase, and don't. My guess is that they'll bill you lots for the answer to that question.

Agreed, but I'm talking about the update to about 100 slides in a powerpoint presentation! From my perspective I now know that their averages are skewed by non-users, so I can ignore them and do exactly what you say: use the numbers to infer relative usage.