Of course I was aware of her Gender Shades paper. It's a small bias evaluation benchmark dataset.
> We developed the Pilot Parliaments Benchmark (PPB) to achieve better intersectional representation on the basis of genderand skin type. PPB consists of 1270 individuals from three African countries (Rwanda, Senegal, South Africa) and three European countries (Ice-land, Finland, Sweden) selected for gender parity in the national parliaments.
A dataset of 1270 images is hardly a breakthrough, the kind I expect to see in a small university project. But it doesn't lead to better models because it's not nearly large enough to train on. What it can do is rate existing models. Basically - useful to critique, not to improve.
A small nitpick: why just two races in a de-biasing dataset? Where are the Asians?
> We developed the Pilot Parliaments Benchmark (PPB) to achieve better intersectional representation on the basis of genderand skin type. PPB consists of 1270 individuals from three African countries (Rwanda, Senegal, South Africa) and three European countries (Ice-land, Finland, Sweden) selected for gender parity in the national parliaments.
A dataset of 1270 images is hardly a breakthrough, the kind I expect to see in a small university project. But it doesn't lead to better models because it's not nearly large enough to train on. What it can do is rate existing models. Basically - useful to critique, not to improve.
A small nitpick: why just two races in a de-biasing dataset? Where are the Asians?