It creates an extremely subjective comparison between candidates.
Part of it is like a sibling comment said, if an interviewer's interests lie more in line with devops, then a devops project that solves a problem he's acutely aware of will be given much higher praise in his mind, and allows him to engage with the candidate more. If the interviewer rarely does any devops and isn't totally sure of the problem space, he may be biased towards thinking it's overengineering a problem that doesn't exist, or may not fully grasp the problem in the time allotted. The candidate could articulate the side project exactly the same but have completely different interview outcomes in both outcomes.
And an argument could be made that "it's a test to see if the latter candidate can articulate a problem and peak the interviewers interest" but in this case the former didn't even have to do that, and thus you are judging candidates by different subjective metrics. And it's especially an issue if the side project isn't directly related to the role (e.g. video game vs cli tool vs database vs website, etc...).
It also means that some candidates will be judged on system design (if a side project is wide scoped) while some candidates will be judged based on algorithmic design (if the side project is very narrow scoped with the details in the implementation).
It just adds so many variables to the interview project that you are no longer comparing "which candidate is the best fit for the job I am actively hiring for" and instead "which gave me the best vibes", specifically because you did not actually measure all candidates by the same measure.
Part of it is like a sibling comment said, if an interviewer's interests lie more in line with devops, then a devops project that solves a problem he's acutely aware of will be given much higher praise in his mind, and allows him to engage with the candidate more. If the interviewer rarely does any devops and isn't totally sure of the problem space, he may be biased towards thinking it's overengineering a problem that doesn't exist, or may not fully grasp the problem in the time allotted. The candidate could articulate the side project exactly the same but have completely different interview outcomes in both outcomes.
And an argument could be made that "it's a test to see if the latter candidate can articulate a problem and peak the interviewers interest" but in this case the former didn't even have to do that, and thus you are judging candidates by different subjective metrics. And it's especially an issue if the side project isn't directly related to the role (e.g. video game vs cli tool vs database vs website, etc...).
It also means that some candidates will be judged on system design (if a side project is wide scoped) while some candidates will be judged based on algorithmic design (if the side project is very narrow scoped with the details in the implementation).
It just adds so many variables to the interview project that you are no longer comparing "which candidate is the best fit for the job I am actively hiring for" and instead "which gave me the best vibes", specifically because you did not actually measure all candidates by the same measure.