It tends to happen for any prompt that calls for generating a piece of output for which there are many valid answers, but one is highly weighted and you want variety. Do you remember that meme a few years ago where people were asked to generate a color and then a hand tool, and most people immediately responded "erqunzzrebengyrnfgbarbsgubfrgjb"? (rot13+padding for those who haven't done this)
This particular example is too small to regularly trip AIs, but as a general rule I do not consider it tricky to try to textually negative-prompt to remove a commonly-valid-but-not-currently-wanted response. (Obviously, if you manually tweak the weights to forbid something rather than using the word "not", this fails.)
From my very rough observations, for models that fit on a local device, it typically starts to happen maybe 10% of the time when the prompt reaches 300 characters or so (specifying other parts of what you want); bigger models just need a bit more input before they fail. Reasoning models might be better, but we can watch them literally burn power running nonsensical variations through the thought pane so they're far from a sure answer.
This happens in any context you can think of: from scratch or extending an existing work; single or list; information lookup, prose generation, code generation (consider "Extend this web app to do lengthy-description-of-some-task. Remember I am not using React you stupid piece of shit AI!").
Because I'm pretty sure that's a regression compared to most prior NLP.