There's a lot of science behind this phenomenon, this article does a good job of discussing the process https://www.psychologytoday.com/blog/your-brain-work/201209/...

But tl;dr your brain is adept at working on problems in the background after you've defocused on them. Your work is still getting done, even if you cut out of work and unwind, no guilt necessary ;-)