The hard part
Nick Scialli puts very nicely something that non-technical folks don’t often say (or understand?) when extolling the virtue of general purpose LLM use cases on the topic of “replacing” roles like developers when it comes to code generation:
Impressive! But here’s the problem: generating code was never the hard part.
The value I bring to my job is not generating code. I mean, I suppose that is a part of it in the end, but my value is largely in the work that happens before I generate the code: things like requirements clarification, negotiation, technical design, and tradeoff analysis.
He goes on to outline how the model should have A) asked clarifying questions to better tailor the output to the requirements and B) essentially used abstraction laddering to ensure that the function was even needed.
He’s right, but also: the model (ChatGPT in this case) can and probably already does do those things if told to or asked to. What will be more important is knowing how to work well with these capabilities to allow someone like Scialli to spend more quality time doing the valuable things he outlines. That part has probably been talked about ad nauseam.
To strengthen the case for the role being much more than code generation, he points out that the task of clarifying requirements of a request and determining if the initial request is correct needs to be done “on a much, much larger scale,” as he puts it. That point is starting to get at the corollary from the “how to work well” point I made above: when to work with models and agents. Because we’re still a ways off from those models being good at what Steve Jobs described:
Designing a product is keeping five thousand things in your brain and fitting them all together in new and different ways to get what you want. And every day you discover something new that is a new problem or a new opportunity to fit these things together a little differently.
That’s the broader value that Scialli is getting at. Productivity tools are not replacements, but—while impressive, revolutionary, and largely mind-blowing—still very primitive tools.
