Be honest, did you use LLM to write this reply? Am just genuinely curious as I am trying to gauge and tune my senses.

OT: I do agree with your assessment.

The only AI flavor was "Semantic ablation isn't a side effect of the training process, it's the intended outcome of the objective."

But, there are hyphens used where em-dashes are expected, so that is anticorrelated with LLM writing.

GP does use a lot of "it's not this, it's that" pairs. :)

I wonder if you can use lower quality models (or some other non-llm related process) to inject more "noise" into the text in between stages. Of course it wouldn't help retain uniqueness from the original source text, just add more in between.

I’m not convinced removing RLHF would really make the probabilities generator give us distributions that can diverge from the mean while remaining useful.

In other words, this might not a problem that can be overcome in LLMs alone.