Testing sirens every few years is crying wolf for you?
Switzerland does annual testing. The media mention the date a few days in advance (it's also something semi-static like "the first Wednesday of month X"), everyone knows when the test is coming, and I'm pretty sure people would pay attention if the sirens went off on any other day.
Other places test much more frequently, which feels like harmfull overkill, but an annual test seems very sensible.
Germany reintroduced annual testing after doing no testing for a long time and found out what you would expect (the system didn't work). It went so badly that they cancelled next year's test to be able to fix their system. A flood decided to test the system for real, the main difference in outcome being that this time the failure came with a high body count.
After that, they overhauled their doomed-to-fail custom app bullshit and started using cell broadcast (like e.g. the US) and a few annual tests later, they seem to be in a somewhat better shape now.
You can simulate everything to some extent but you can't have an end-to-end test without actually testing.
You also want to make sure that people know what the sirens sound like (no, they don't sound the same everywhere), and a national test day is a great opportunity to remind people how they should act if the sirens go off for real. Having them go off once a year at a pre-announced time also isn't particularly disruptive.
The proper way to test this would be to also make sure that e.g. the radio stations are informed of a test scenario and broadcast it (sirens means "turn on the radio and check for what is happening", which means the sirens alone are useless without that part of the alerting working). This would also help if someone lives under a rock and knows how to act but forgot that testing is happening.
Since this part is not being tested, I think there's a high risk that the announcement in the radio won't work in an actual emergency, and an announcement on the Internet is extremely likely to be unreachable due to overload.
I think it would be ideal to actually run large scale emergency response tests (pick a small town, pick a disaster, and actually run the whole scenario, either asking people to volunteer, compensating them, or making it mandatory under a draft-like law). Yes, this would be extremely disruptive (hence a small town), but from having run emergency responses (to IT incidents, not life-or-death incidents), my experience is that any procedure that isn't regularly practiced won't work when needed, and running exercises and fixing what you find will improve this a lot.
I think some places in Germany have weekly tests, but their sirens are (or used to be before mobile phones) a way to call up the local volunteer fire brigade in smaller communities.
The audid linked two posts above is titled "Feuerwehralarm" = fire brigade alarm.
Switzerland does annual testing. The media mention the date a few days in advance (it's also something semi-static like "the first Wednesday of month X"), everyone knows when the test is coming, and I'm pretty sure people would pay attention if the sirens went off on any other day.
Other places test much more frequently, which feels like harmfull overkill, but an annual test seems very sensible.
Germany reintroduced annual testing after doing no testing for a long time and found out what you would expect (the system didn't work). It went so badly that they cancelled next year's test to be able to fix their system. A flood decided to test the system for real, the main difference in outcome being that this time the failure came with a high body count.
After that, they overhauled their doomed-to-fail custom app bullshit and started using cell broadcast (like e.g. the US) and a few annual tests later, they seem to be in a somewhat better shape now.