They didn't think they knew enough about CJKV to make that call, and so instead asked pre-eminent scholars from major Chinese, Japanese and Korean universities, and they replied that 21k was going to be enough.
The reason they thought that they could fit Chinese into so few letters was that at the time, the CPC supported academics who wanted to reform Chinese towards fewer letters. Not long after, views about traditional scholarship changed, and now they want to promote maintaining more of their traditional characters.
The reason they thought Han unification would work was that the time they asked was in a short period of rapprochement in Sino-Japanese relations. At the time, it was good politics for Chinese scholars to co-operate with Japanese ones. Very soon after this changed.
The current policy seems to be to add characters to Unicode even if nobody uses them any more. Is that a complete flip from what it used to be? Because I feel like that's the only way character reform would have mattered.
Yes. The current policy is made possibly by the adoption of UCS-2 and UTF-8, which expanded the amount of representable characters from 64k to >1M, of which only 10% are in use.
Or was there an intent to not encode some of these rarer characters? I haven't been able to find any info.