This is really cool. My optimistic take on GenAI, at least with regard to software engineering, is that it seems like we're gonna have a lot of the boring / tedious parts of our jobs get a lot easier!
Claude 3.5 Sonnet still can’t cut me a diff summary based on the patch that I’m generally willing to hand in as my own work and it’s by far the best API-mediated, investor-subsidized one.
Forget the diff, I don’t want my name on the natural language summary.
Even under the most generous nomenclature, no contemporary LLM understands anything.
They approximate argmax(P_sub_theta(token|prefix)).
This approximation is sometimes useful. I’ve found it to never be useful in writing code or prose about code of any difficulty. That’s my personal anecdote, but one will note that OpenAI and Anthropic still employ a great many software engineers.
I know that, likely everyone here knows that. But understanding is a good approximation for what we mean. Pointing out implementation is needlessly pedantic.
