Which display are you using and does it wake up properly when you wake your mac mini? I've had a problem with displays that wake up normally with an intel based mac but not with an M1 mac.
I'm also using a 144hz monitor (Samsung 28" G70A) with a DP-USB-C cable. Wakes up normally and VRR works both with my new MBP (M1 Pro) and with the previous MBA (M1).