One thing I haven’t gotten right is caching. Some of my builds have targets they depend on that are very expensive but rarely change, and I haven’t been able to make sense of when those targets are (re)computed and when they aren’t. This results in many builds taking a long time, when the targets could be reused from a cache.
Earthly is also missing something that I find to be a glaring omission from make—give me a command to simply list the targets in an Earthfile. ‘earthly ls’? Maybe have a column or two that indicates whether the target saves an image or artifact. When building a project I didn’t author, it’d be so helpful for the targets to be discoverable.
+1 on `earthly ls`. I've also thought it would be valuable to have "public" and "private" targets. When looking at a new codebase it can be hard to know which targets I should use when developing, and which are purely for internal use. Good docs help with this too, but they are not always there.
The `earthly ls` idea is great. We do have shell autocompletions but that is not quite the same thing. I will add a ticket for that.
Cache misses can be a bit inscrutable. It could be the buildkit GC is running, because disk space is getting scarce, or that some arg or file change caused the cache to be considered invalid.
Caching is an area we will continue to improve. We have a proposal for extended cache mounts here[1].
I didn't know about the autocomplete, that is really helpful; and I'm excited to see `ls` PR'd! Huge thanks to the contributors.
I think some of my cache misses were a rookie mistake: I was building a few expensive targets on a `FROM ...:latest` base. The base image changed often enough to make the cache misses annoying, but infrequent enough to make me think it was GC. My builds are much happier now! :)
re ls: `https://github.com/TekWizely/run` displays targets in the usage command (ie `run -h`). It might be worth looking at their output for inspiration. (Runfiles are doing something similar to Earthfiles).
Just a quick drive-by to say thanks for the call-out - I hope you're finding run to be a useful tool and would love to hear more about how you're using it!
One thing I haven’t gotten right is caching. Some of my builds have targets they depend on that are very expensive but rarely change, and I haven’t been able to make sense of when those targets are (re)computed and when they aren’t. This results in many builds taking a long time, when the targets could be reused from a cache.
Earthly is also missing something that I find to be a glaring omission from make—give me a command to simply list the targets in an Earthfile. ‘earthly ls’? Maybe have a column or two that indicates whether the target saves an image or artifact. When building a project I didn’t author, it’d be so helpful for the targets to be discoverable.