That's a good thing actually since it means you never have cold starts (unlike Google Cloud Run). Their micro instances cost like $3-4 which isn't that much.
It should be an option in my opinion, as cold start time could vary a lot by which image and runtime you are running, if is a Java app or a statically linked Rust program.
I have a bunch of very small and simple services, which I run on a single VPN. Moving to fly.io would mean to pay various time more. I tried that, and the service is nice indeed, but not for me I guess.