I use an ssh virtual host pattern *-tmux to copy over an ssh-specific tmux.conf to a remote host before connect and then connect to it with -f tmux-ssh.conf. I can also connect to a host directly without tmux stuff by omitting the -tmux suffix.