I had that problem as well. Especially when connected to two external monitors.
I did not love the machine and the M1 Max was such a big upgrade because of that and I could upgrade to the m3 max later and give my m1 to somebody else. Both apple silicon machines are going strong for a long time I guess.