Showing posts with label if you don't understand statistics don't pretend like you do. Show all posts
Showing posts with label if you don't understand statistics don't pretend like you do. Show all posts

19.11.14

Steve Albini Was Right!

The funny thing about the music industry, as he says, is that it is fundamentally unconcerned with musicians and audiences. In the old days, it did this by making money regardless of the level of success a band experienced, as documented in its glory by "The Problem With Music". Now, it does so through analytics.

The article, on Shazam, treats data as a verifiable way to determine what music is going to be popular. It is premised on two assumptions that are quite debatable:

1. Musicians will continue to produce music in a range of genres, thus providing a constant stream of product to be sold. The quality of this music is unimportant, only whether people will buy it.

2. The important thing about audience reaction is that it exists. Why an audience reacts to a song is inconsequential.

In other words, Shazam's decisive technological victory is to take songs that have already been written, recorded and distributed, and use the fact that a growing audience has already liked the song, to determine that yet more future people will like it. Presumably, the song popular with a growing audience will continue to grow quite apart from Shazam or any other analytics. "Royals," the article's example, seems to have done just fine without concerted record industry action. Synthesizing this information isn't any kind of innovation at all: one would have to demonstrate that analytics make songs more popular than they would have been otherwise.* If you were looking for evidence that the music industry thinks of music as widgets and fans as lines on a balance sheet, you could hardly do better.


*Analytics make sense in some contexts. For example sports, where there are objective (if subjectively-defined) measures of success, and correlations can be found for the regular production of these measures. Very little is similarly objective in the arts. Certainly not amongst the whole population. If the purpose of analytics in music is to determine which sub-demographics might appreciate a certain type of music, then you're just re-creating the distribution models for independent music, or for niche markets like jazz and classical.

13.6.12

I more or less agree with this on the same-sex parenting study. It's worth pointing out the asymmetry between critiques of the study and responses to those critiques. The critiques tend to focus on social-scientific issues of study construction: the study can't be a good one because it stacks the deck by a. including any reported level of known same-sex parental interaction as functionally equivalent to being raised for life by same-sex parents and therefore b. moves over into the category of 'same-sex parents' a number of dysfunctional heterosexual relationships, thereby biasing the results. The responses to the critiques tend to focus on the political and social agendas of the people who are critiquing, e.g.

This is one of those instances where the CCOA's skepticism about social science doesn't serve him well: time spent with statistics and giving some thought to how to construct studies will, I think, make it evident that the critiques identify problems with the study that would exist regardless of what conclusion it came to. It would be interesting, though, to get hands on the raw data and attempt to re-run the study with a more neutral classification scheme. My guess would be that the highest level of dysfunction would correspond with people in a middle group--raised by putatively heterosexual parents except that one (or both) had unexpressed same-sex attraction, which contributed to tension within the marriage, or the end of the marriage. But the social and political implications of that would be much more complicated for all sides.

4.7.11

This week in "if you don't understand statistics, don't pretend like you do":

I make no claim to be great at statistics, though I did take two graduate-level semesters and can at least fake my way around an econometric model (hint for grad students: the problem is almost always in the theoretical model, or the lack thereof). However, I do enjoy a good exploration of statistics and how people fail to understand them. So I liked this rebuttal of the recent studies linking diet soda consumption to weight gain. The theoretical model left something to be desired, apparently:

What’s that? A correlation, you say? Why, the only possible explanation is that the variable randomly assigned to the x axis must have caused the differences in the variable plotted on the y-axis! It’s SCIENCE!...

Because there’s no chance there’s some confounding factor, or that the causal arrow points in the other direction. After all, people who are getting fatter wouldn’t have any reason to be more likely to drink diet soda, would they?

And this is to say nothing of the other obvious alternative explanation, that people who consume 'diet' beverages think this frees them up to consume extra calories elsewhere, and end up consuming far out of proportion to their 'savings.'

mgoblog also had a piece on statistics and how college football fans fail to understand them. The best part:

This is the disconnect. While what seems like a fairly large subset of the fanbase saw wholesale collapse in the Wisconsin game, computers saw two units failing immensely and an offense that put up 442 yards on a defense that gave up 321 on average, scored 31-ish points (computers will credit the offense with acquiring the field position for the field goal and deduct the miss from the special teams; if they deduct from the garbage TD they will use a lower denominator when trying to figure out expected points) on a defense that gave up 21. Statistically, Michigan's offense was at least a standard deviation above the mean against the Badgers.

While the Wisconsin game is the biggest outlier between the offense's actual and perceived performance, it's instructive. ...This is why statistics are useful, because meat-emotions often overwhelm our capacity for reason.
...followed, naturally by 100-ish comments that argue statistics are meaningless and we should just believe what our eyes tell us and hurr durr requisite link to Fire Joe Morgan RIP.

The past six months has been a useful exercise in learning that Michigan fans are not particularly more intelligent than any other sports fans, our own self-mythology notwithstanding, and there are a surprising number of people who believe that if you use and attempt to understand statistics, they will steal your soul.