Masking sex of people in technical interviews find no bias against women

There’s a site, interviewing.io, that facilitates companies doing technical interviews, as well as apparently working to help people improve their technical interviewing skills. They tried an experiment where they masked people’s voices during the interviews, so interviewers couldn’t know if they were interviewing men or women.  The results are hilarious.  Their response is even funnier.

First, this is what happens during normal non-masked interviews:

Specifically, men were getting advanced to the next round 1.4 times more often than women. Interviewee technical score wasn’t faring that well either — men on the platform had an average technical score of 3 out of 4, as compared to a 2.5 out of 4 for women.

And this is what happens after the interviews:

As it happens, women leave interviewing.io roughly 7 times as often as men after they do badly in an interview.

And these are the results:

Contrary to what we expected (and probably contrary to what you expected as well!), masking gender had no effect on interview performance with respect to any of the scoring criteria (would advance to next round, technical ability, problem solving ability). If anything, we started to notice some trends in the opposite direction of what we expected: for technical ability, it appeared that men who were modulated to sound like women did a bit better than unmodulated men and that women who were modulated to sound like men did a bit worse than unmodulated women. Though these trends weren’t statistically significant, I am mentioning them because they were unexpected and definitely something to watch for as we collect more data.

So, far from being sexist, technical interviewers range from unbiased to slightly preferring women over men.  Which, being in the technical fields myself, is exactly what I expected.

But what’s really, amazingly funny is their “happy place”:

Once you factor out interview data from both men and women who quit after one or two bad interviews, the disparity goes away entirely

But we’re told that women drop out at 7x the rate of men after a poor interview.  So to translate your sentence into something realistic: “After we got rid of a significant number of poorer performing women, and almost none of the poorer performing men, what was left was a pool of people who performed roughly the same.”

Let me express it for you in mathematical terms:

The standard deviation between women is less than the standard deviation between men. This means that in any “pool” where entry requires above average performance on characteristics whereon average men do at least as well as women do (and intelligence is such a case), you will find more men at the positive extremes than women.  So in any environment where honest evaluation takes place, men will do better than women, and will succeed more often than women.

Now, you might want to claim that it’s better to have more “women in tech”, even if that means having dumber, less qualified programmers who write worse code. And if you want to make that argument, I’ll be happy to listen and respond.

But can we please drop the fantasy that there aren’t “enough” women in STEM because of sexism?

Advertisements