Can AI replace Software Engineers?

There’s considerable buzz around OpenAI since the ChatGPT beta, mostly I suspect because using it hands-on is an experience like no other (outside of DALL-E), but after all the fun and pushing its boundaries, how much impact does it have on software development?

In my view, it’s an incremental change, however revolutionary it feels when you first touch it. It will not replace our role as software engineers, but in the long term (>10 years) software engineering will look very different. It will make us perhaps 5% more efficient (illustrative ballpark) in the short to medium term, for a fee.

Why do I say all this?

Incremental and only 5%? Pas de révolution? — After using a mix of GitHub Copilot and ChatGPT, I found it impressive but limitations quickly emerge:

• The large language model (LLM) is trained on a huge dataset but can only keep a few thousand words of context for the task in hand. In order to work with any sizeable codebase, that needs to scale up.
• It’s often wrong. Coding is weirdly both creative and precise. A tiny error can have impact far beyond the module where it arises. The engineer still needs to read and understand the code, make decisions about its design, devise test cases and find bugs. AI can help with the latter too, but it’s hit and miss for now.
• Writing code is only one skill; optimising and reducing the amount of code is another.
• It takes effort and practice to prompt the AI, which really doesn’t read minds! Just as with a junior assistant, a well meaning brief can give unexpected results when you forgot to articulate some part of the problem.
• Even if it makes the coding task 20% more efficient, an engineer might only spend 25% of their time writing code, tests and documentation. Wait, what are they doing?! Well, they might even spend 75% of their time at the keyboard but a large chunk of that is reading, thinking, exploring, trying things out. As I say, it’s a creative endeavour.

In the long term software engineering will look very different, just as our current methods differ from the pre-web era. While engineers 10–20 years hence will barely recognise the primitive tools we have now. I expect it to remain a skilled discipline.

Eventually we will be instructing computers with much higher level concepts. Just as we have compilers and interpreters now, AI will add another layer of abstraction. We’ll be able to define an architecture, domain boundaries, APIs and ask it to write code to perform business logic. But how? Human languages are full of ambiguity so we’ll need to define a common language or use existing concepts (UML, ubiquitous language etc.) and here we are, still programming albeit further removed from the mechanics and with greater impact.

Despite the learning and discipline needed to effectively wield these powerful tools, engineers will be more productive. Just as the emergence of APIs has given enormous leverage to build more capable applications, AI will provide a similar boost. Do you remember which year APIs changed the industry? I don’t; it was a gradual change over a decade or more.

It’s going to take time to improve the models and deliver practical, usable tools. It’s going to cost (R&D, running costs). OpenAI will want a share of any bottom line savings, adding another brake on the immediate impact. The competitive market for generative AI is set to explode and will take a while to settle and processing costs to come down.

A good example of the long timescales is self-driving cars. It’s around a decade ago that Google made real headway, sparking a buzz and competition before spinning off Waymo. As I will never be allowed to drive I’ve been following this with keen interest. Five years ago it was going to be ready by the end of the decade. Now, we have a very carefully orchestrated service running in the easy suburbs of Phoenix, Arizona and some trials elsewhere. I do not expect us to be routinely using self driving cars here in the UK for another decade at least.

For all that I’m really excited to be using ChatGPT and Copilot. Who doesn’t want a tool that makes their life easier? Also, the potential in decades to come is truly impressive.

In the short term, generative AI is already a boon to polyglot engineers switching between languages—a step up from language sensitive editors and regular autocomplete, as it can draft significant blocks of code. It certainly saves a lot of typing, can answer questions and give a second perspective.

Rather than putting engineers out of a job, I think AI will make them more badass. The day when engineering discipline is no longer a thing seems a long way off.

Footnote

OpenAI have announced ChatGPT Plus, a paid version of their hypebot*. I’ll probably pass on the $20/month fee for now—while the beta was impressive and I hope the fee resolves the capacity problems, it’s too limited in its disconnected form.

Imagine if Apple had made a beta of the iPhone with its touchscreen, apps and so on — but with no connectivity. Sure, I’d be impressed with its potential, but I wouldn’t pay for a version with no Internet.

There are numerous technical, privacy and safety challenges involved in adapting ChatGPT to the connected world, but it’s a prerequisite for mass adoption. No doubt this is in the works.

(* yep, I know I’m contributing to the hype)