Usual disclaimer, I’m pushing this out earlier than I’d like (draft wise) because I’m too busy and shouldn’t be focussing on this…apologies for inaccuracies and poor writing :-). Last year I went to a talk “[I am therefore I buy – Measuring consumer preference in a digital age]1” That talk suggested (something like) products signal particular personalities, so people with certain personality traits buy/like those products to say something about themselves…I was somewhat sceptical about this for at least two reasons 1) the data is filthy, and 2) the notion is circular

In the last day or two there’s been a lot of coverage of a new story, with a headline along the lines of “Facebook Preferences Predict Personality Traits” (sciencemag), “Facebook users unwittingly revealing intimate secrets, study finds” (guardian)

I (cynically) assume this is mostly based on press-releases – gotta love a bit of churnalism – based on this [PNAS article]2 First off, as was the case when I went to the talk, I have issues with psychometrics which influence my view, and I think some useful work is being done in this case – it’s just that I wouldn’t want to generalise as far as has been here.  This is unsurprising – if we’re worried about employers and governments collecting psychometric or demographic data on us using implicit data, well I worry about using psychometric tests to inform anything terribly useful (e.g. what does it mean to be 5 points more ‘extravert’ than someone else?). Also, in this case I think the article raises some really important points regarding the ethical issues of data collection using implicit trace on the web – there are potential issues in raising awareness about privacy, ensuring institutions (of whatever kind) cannot abuse such information, and exploring the issues around ownership and use of data.  They’re also quite aware of some of these issues. However, I would raise a few points on the study, including: * What the title should say is – the ‘likes’ of people who are curious about how their personality is displayed through facebook (and therefore added an app called myPersonality) might tell us something about those people.  Perhaps facebook users are odd, but myPersonality app users are likely an odder subset with particular characteristics including, I would guess, a higher use of the site.  (see below) * Conflating personality and other traits is not helpful in any way, the paper refers to ‘attributes’ but quotations and news reporting (sigh, not the author’s fault) are often just referring to personality.  This is irritating because it makes critique of specific parts of the study harder in some ways, and because it implies personality and other features can be lumped together which could lead to people leaning towards personality/politics/sexuality being biological/psychological and immutable/unlearnable * The ‘likes’ of people who declare may be different from those who do not. What’s being claimed in some places is that – from ‘likes’, we can assign particular attributes accurately. Actually though the research uses profiles in which attributes are declared and then checks their classifier against the declared attributes.  That’s a different issue – people who have already declared they’re gay might be a different group (and like different things) to those who have not done so.  This criticism doesn’t apply to some of the variables (e.g. parents staying together, personality, etc.) but other criticisms do. To be fair, in the supplementary info they do say “An important limitation of our sample is that some of the predicted variables are from Facebook profile information. Individuals who declare their political and religious views, relationship status, and sexual orientation on their profile may be different from nondeclaring members of those groups; they may associate with distinct Likes, which may lead to an overestimate of prediction accuracies for these groups.” * Classification success is based on culture and self-selected samples.  88% success in classification of gay/not gay strikes me as not great (in their sample, 4.3% males;2.4% females, I’d be interested in the false positive/negatives) also a) this is self-selecting sample, b) this tells us more about culture than it does about anything else (e.g. ideas of masculinity – it’s interesting that the classifier was only 93% accurate for gender, an example which also serves to reinforce another point: there are not only two sexualities, nor two genders).  Another e.g. on the culture point “some of the likes were clearly related to their predicted attribute, as in the case of the No H8 Campaign” – yeah, cuz wanting equality is like so gay. * bisexuals classified as ‘heterosexual’ (never mind the fact the facebook way of defining sexuality might be narrowing in any case): Bisexuals ignored… “Sexual orientation was assigned using the Facebook profile “Interested in” field; users interested only in others of the same sex were labeled as homosexual (4.3% males; 2.4% females), whereas those interested in users of the opposite gender were labeled as heterosexual.” This is bad writing, the supplementary info makes it clearer “Facebook profiles; users who listed being interested in only the opposite gender were labeled as being heterosexual, whereas users who listed only the same gender were labeled as being homosexual” * Classifier success of 60% on what distribution? From a quick search I can’t see stats pre-21, but it’s about 30% of first marriages in the US (who knows about of facebook members generally).  In this study, n = 766; 56% stayed together and 60%  of users were correctly classified.  As above my statistics (certainly off the top of my head) aren’t good enough, but I wonder how good this is…randomly assigning how successful could we be compared to this?  Anyone any thoughts?…, I’d like  to see more detail and just how obvious some of the ‘likes’ leading to classification were…still, the potential for systematic exclusion (whatever accuracy level) is scary. * (Note this again ignores some nuance in how families are constituted, e.g. if I have had a stepparent since I was a child, they might be considered my ‘parent’, self-reporting on this issue will be complicated). * You might think you’re measuring x, but it’s actually a proxy for y (or abc)…this isn’t necessarily a problem, but I think it’s probably quite important to understand the stories.  If we claim we can “detect” some attribute, we might in fact be detecting some associated attribute such as socioeconomic status which a) we might have higher success on b) we might want to understand more (e.g. because it’s a societal problem), and c) this is a reflection of culture, it isn’t immutable it is dynamic, temporally and geographically located, etc. E.g. smoking status and socioeconomic status are associated. Doesn’t mean I should charge poor people higher insurance though… * I can’t believe how many times I’ve said this in the last month…but: other countries have more than 2 parties; have more nuanced politics; have coalitions and minority governments; have different sorts of debates.  Again this is an issue of culture, and of classification. * visual inspection for ethnicity…misuse of this for discrimination is obviously problematic, but it is also a) as the visual route shows, fairly obvious (tho read on), and b) covered by existing legislation within the US/elsewhere.  However, the claim is problematic because 1) there are more than 2 ethnicities, 2) visual inspection just makes me cringe thinking e.g. of places deciding whether someone was black or not, 3) * Implications for different institutions matter – the overarching concern the paper raises is with how such data could, theoretically, be used (or misused) by various institutions – including malevolent governments.  I’m concerned that a vague reference to misuse isn’t terribly helpful though – what access would one need, what legislation is currently in place, _what do we already know about personalised search and similar customisation from trace, etc._ * if a malevolent government wanted to categorise facebook users with a low threshold for a ‘positive’ on some variable (i.e. they’re not too worried about false positives), they might be able to do so using facebook data – and that’s concerning.  But I’m not sure what we do about that other than democratisation, ensuring secure systems and data privacy in so far  as possible, etc.  This is as true at home as it is abroad – and steps by Western govts to seek access to personal data should consider this.  The issue here isn’t the capability to categorise – that’s moot – it’s the deployment of such technology for unethical purposes, that’s why accuracy doesn’t matter – dictators rarely care much about false positives. * Facebook (Google, etc.) – this is open to abuse, collaborative filtering stuff is an issue, and the fact that governments may be able to access this data is problematic, again see above * Casual users – are likely to interact with people…they’ll find stuff out that way, not by either running algorithms or even by assuming things from ‘likes’ (although they might do that) – this is another problematic but separate area. As before, I don’t have a problem with this sort of trace data research – hell it’s what I do! But why on Earth would you try and push it in to the sort of traditional psychometric research, with all of the problems that has? I can’t see the value to marketers or individuals.  So, trace is great (and I’d love to read the paper “A Young Liberal Christian Woman Is Searching The Web”) but I’m just not convinced we can make some of the claims being made from this data…and I am convinced that some of the claims are non-sequiturs, they’re important points, but they’re not coming out of the data but out of other concerns (e.g. accuracy of categorisation isn’t the concern, the ability to derive and assign to categories by malevolent institutions is) Kosinski, M., Stillwell, D., & Graepel, T. (2013). Private traits and attributes are predictable from digital records of human behavior Proceedings of the National Academy of Sciences DOI: 10.1073/pnas.1218772110