"Is it worth" is entirely subjective, I don't think someone who's happy with perf board particularly cares about RF performance or routing complex BGAs against the increased complexity of debugging a multi layer design. In general I would think that if you aren't sure you need a multilayer board, you probably don't.

The more layers, the easier to design IMO. You have to do more QC when placing a via, and there's more layers in general to flip between, but routing puzzles become easier.

I find having access to traces to cut and probe to be a lot more important for hobby projects (that usually don't go through a simulation first). Routing is a very simple exercise below RF. YMMV though, exact application and goal probably matters more for this discussion.