In normal texts, that's correct. However, written Chinese does contains semantic information which the spoken language and Pinyin lack and, unlike English, has fewer distinct syllables, and seldom borrows words from other languages. So someone who's literate in Chinese would usually be able to infer the meaning of unfamiliar words when written down, as they would already know the meaning of all their component characters, but might struggle if they were written
phonetically. This is like having a good knowledge of Classical Greek when encountering words like nephropathy or myocarditis for the first time.
It still isn't a very good argument, though. Most English speakers get by without any knowledge of classical languages, and accept having to look up words in a dictionary.
Someone who's literate in Chinese would only be able to infer the meaning of an unfamiliar character if they already knew all of the surrounding characters. Then, you can guess the meaning of the character based on context, and possibly hints from the character itself about pronunciation and/or meaning (though this is very hit-or-miss, because many characters don't contain obvious hints). In order to reliably know all of the context surrounding a character, you need to know about 3000 characters total (that's the point at which you can recognize 99% of characters on a page). This is still a very tall order, which takes years of study to achieve.
The Chinese characters do indeed contain semantic information that Pinyin (the standard Romanization) does not, but in practice, you don't need that extra semantic information. If you write down a single word in Pinyin, it may have a few homophones, whereas the same word, written in Chinese characters, would be unambiguous. However, in written Pinyin texts, you would almost always be able to figure out which word is meant from context. In the few cases in which that would not be possible, the author could slightly rephrase the text to make it unambiguous.
Most languages on Earth (that have a writing system) are written using alphabets. Chinese is not so special that it could not be written using an alphabet as well. The reason why China hasn't switched to an alphabetic script is because of cultural attachment to the script, not because the Pinyin doesn't work just as well in a practical sense.
> Someone who's literate in Chinese would only be able to infer the meaning of an unfamiliar character if they already knew all of the surrounding characters.
In what I wrote, I was assuming there would be no unfamiliar characters, but there would be one or more unfamiliar words composed of two or more characters.
I was trying to put forward the best argument I could think of for retaining the characters, but like you, have decided it isn't worth the additional effort of learning thousands of characters up front to become literate when you can use a phonetic script and look up any unfamiliar words in a dictionary instead.
It still isn't a very good argument, though. Most English speakers get by without any knowledge of classical languages, and accept having to look up words in a dictionary.