No - there's very much active disagreement on which people are black (colorism in general in the black community is alive and well) and who thinks you are a jew might change a good bit if you ask the local white supremacist or a rabbi.
Just because "I know it when I see it" applies to your personal lens its an inarticulate way of viewing the world.
Right, but when we discuss racism we're not discussing who is black or not black with the proviso that "of course it's fine to treat them badly if they're actually black", whereas it is commonly accepted by people on all sides of the gender debate that people should be treated differently on the basis of their gender.
Just because "I know it when I see it" applies to your personal lens its an inarticulate way of viewing the world.
https://en.wikipedia.org/wiki/One-drop_rule