> can you do encrypted communications with a ham license?
No. It's prohibited internationally and enforced by all ITU member states, the only exception which encryption is acceptable is the control uplink to aircraft and spacecraft. Although in some countries, using encryption domestically, purely for access control (not for confidentiality) is acceptable under some circumstances, if you publish your key.
> does spread spectrum qualify?
No. There are additional rules governing spread spectrum communications, but it doesn't automatically imply encryption. Many spread spectrum systems use weak PRNG and not meant for encryption.
The motivation is to keep (accidental) unlicensed users out of amateur radio service. For example, 2.4 GHz is both an unlicensed band usable under Part 15, and simultaneously an amateur radio band usable under Part 97, these are very different rules. Since amateur radio operators can use any public protocols (including 802.11) on the air, there is a risk of accidentally exchanging communication with devices owned by unlicensed operators (such as your neighbor's laptop), which violates radio regulations. Under this circumstance, some people believed that in the United States, encryption with a key published in a publicly accessible place is allowed (and there are still other people who says it's not allowed, but the debate has never been legally challenged until a high-profile abuse appears in the future, but so far there is none, so currently it is a de facto acceptable practice).
No doubt to prevent abuse and allow identification of violators.
Encryption would open the bands to general commercial usage. My uncle works in Canada's spectrum enforcement. You would be surprised at how many small business attempt to violate band licenses to save a few bucks. Think taxi or logging services with commms on protected bands.
You have it backwards. They can't violate the rules now because it would be obvious. If encryption were allowed it would be impossible to know if a transmission is violating certain rules or not.
Like what? Plotting to kill somebody over the radio? Logically there must be rules about hogging spectrum or harassing people but I'm having a hard time imagining how encryption could stop someone from knowing if you're breaking another law.
It served three important purposes. (1) To persuade the governments to allow the existence of amateur radio, (2) To force the publication of technical specifications, and (3) To restrain its use (abuse?) by commercial vendors.
First, it's important to understand that radio communication is a modern creation, not a post-modern creation like the Internet. From the beginning, telecommunication and radio communication was/is an area under heavy regulations by powerful governments. Since a hundred years ago, almost all aspects of communications (modulation, equipment, content, frequencies, price, recipients), even whether one is allowed to listen a broadcast station, were heavily regulated, often under the name of national security (and commercial interests). Today's regulations still follow the same framework. Amateur radio service was created to allow noncommercial technical experimentation - encryption is prohibited as parts of the compromise made by experimenters, in exchange, Big Brothers allowed experimenters to perform noncommercial communications under greatly reduced regulations, this even includes some authoritarian regimes such as the Soviet Union. Without such a compromise, amateur radio service wouldn't even exist at all. Unless the underlying political landscape and framework of radio and telecommunication is changed greatly, such as the vision advocated by the supporters of spectrum commons theory [0], this situation will remain.
Second, what has been prohibited is not encryption itself, but all forms of encoding that cannot be understood by others ("secret code"), this includes encryption, and also includes proprietary protocols. This regulation forces all communicators and commercial equipment vendors to document the technical information of the communication protocols being used to facilitate experimentation, thus it serves as a barrier against the use of proprietary protocols that is detrimental to the community. There is only one major flaw - there are no rules about patents, and there are already protocols designed by commercial vendors that cannot be legally decoded without obtaining a patent license. Nevertheless, the non-encryption rule works well to keep all protocols open, at least, its specification.
Third, amateur radio is, by definition, non-commercial, and it's not negotiable. There is the fact that the existence of amateur radio frequency allocation is constantly under the threat of reallocation to commercial services (the latest one is 5G). And there is also an opinion commonly held by many in the community, that there's always a trend that some commercial users may choose to abuse amateur frequencies to avoid the costs of using a commercial radio service, and that if proprietary protocols or encrypted communication is allowed, there's a danger of a de facto commercial exploitation of the amateur radio service when the nature of communication cannot be determined.
So, in conclusion, encryption was originally prohibited for "national security" to allow the existence of amateur radio, and many members in the community today believes the rule is still necessary to keep the proprietary protocols out of amateur radio, and/or to keep commercial abusers out of amateur radio. Although there are many situations which encryption can be beneficial for the community, but the consensus seems to be that, even if the state allows it (which is unlikely), its threats to the community is greater than its benefits. Any argument to allow encryption in amateur radio will be faced with suspect by many that it's only a disguised attempt to introduce proprietary protocols or commercial uses.
Amateur radio has usually been sold to regulators as a way to encourage international goodwill. Setting up a private point to point link is inherently in opposition to public goodwill. Like the difference between zoning real estate for public park to increase public happiness, vs zoning private warehouse space; they can't coexist in the same location.
The FCC provides elaborate services ... to SOME services. If you interfere with aviation navigation signals, ambulance dispatching, or megacorporation broadcasting they have in some cases visited with swat teams. If you have no borders and no categorization there can be no differential services. If some idiot decides to use the middle of the 80 meter ham radio band for broadcasting and spectrum enforcement is a free for all then nothing will work for anyone anymore. This was the justification for a TV broadcast license costing roughly 1e5 times as much as a ham radio license, which also makes problems. The real estate analogy is no zoning at all of any kind usually doesn't sell very well to endusers once its explained what will happen. No one likes excessive zoning regulation, but none at all is usually worse.
We're exiting a local maxima of personal freedom and for most of the lifespan of the technology the idea of letting people say what they want on the radio is on the downswing along. The easiest way historically to deal with "this speech is not a hate crime if spoken in Morocco but is a hate crime to be heard in Germany" or similar was always to make encryption illegal thus abstract out the technology of radio. This also helps with espionage. If its illegal to send encrypted codes (internationally and locally), and you don't send any, its kinda hard to get prosecuted for sending CP or industrial secrets or hate speech or whatever.
Your link isn't arguing that encryption being prohibited is silly, it's arguing that proprietary protocols like D-STAR are effectively "intended to obscure the meaning of the communication" (for profit rather than secrecy) and shouldn't be allowed.
My understanding was under one interpretation encryption was always permitted by the FCC on ham bands over 50mhz but by a 2:1 majority common wisdom within the community (and the ARRL itself) hadn't come around to this line or reasoning[1]. Specifically encryption is not allowed if it's "for obscuring meaning except for the following circumstances" of which authentication and securing network operations are two. I would want some legal precedent or an amendment to the code before weighing on this too heavily to see which of the two interpretations takes priority.
That article isn't the final word but it's interesting following the work of the broadband-hamnet folks - particularly in Texas[2]. They mostly say one can't use encrypted traffic. They are however relaying IP traffic but imply one shouldn't use https.
During National Emergencies (like now?) a lot of the rules around call signs, power, and presumably encryption fall by the wayside. In practice it looks like the FCC grants per disaster waivers[3].