Source URL: https://simonwillison.net/2024/Sep/10/software-misadventures/#atom-everything
Source: Simon Willison’s Weblog
Title: Notes from my appearance on the Software Misadventures Podcast
Feedly Summary: I was a guest on Ronak Nathani and Guang Yang’s Software Misadventures Podcast, which interviews seasoned software engineers about their careers so far and their misadventures along the way. Here’s the episode: LLMs are like your weird, over-confident intern | Simon Willison (Datasette).
You can get the audio version on Overcast, on Apple Podcasts or on Spotify – or you can watch the video version on YouTube.
I ran the video through MacWhisper to get a transcript, then spent some time editing out my own favourite quotes, trying to focus on things I haven’t written about previously on this blog.
Having a blog
Aligning LLMs with your own expertise
The usability of LLM chat interfaces
Benefits for people with English as a second language
Are we all going to lose your jobs?
Prompt engineering and evals
Letting skills atrophy
Imitation intelligence
The weird intern
Having a blog
23:15
There’s something wholesome about having a little corner of the internet just for you.
It feels a little bit subversive as well in this day and age, with all of these giant walled platforms and you’re like, “Yeah, no, I’ve got domain name and I’m running a web app.”
It used to be that 10, 15 years ago, everyone’s intro to web development was building your own blog system. I don’t think people do that anymore.
That’s really sad because it’s such a good project – you get to learn databases and HTML and URL design and SEO and all of these different skills.
Aligning LLMs with your own expertise
37:10
As an experienced software engineer, I can get great code from LLMs because I’ve got that expertise in what kind of questions to ask. I can spot when it makes mistakes very quickly. I know how to test the things it’s giving me.
Occasionally I’ll ask it legal questions – I’ll paste in terms of service and ask, "Is there anything in here that looks a bit dodgy?"
I know for a fact that this is a terrible idea because I have no legal knowledge! I’m sort of like play acting with it and nodding along, but I would never make a life altering decision based on legal advice from LLM that I got, because I’m not a lawyer.
If I was a lawyer, I’d use them all the time because I’d be able to fall back on my actual expertise to make sure that I’m using them responsibly.
The usability of LLM chat interfaces
40:30
It’s like taking a brand new computer user and dumping them in a Linux machine with a terminal prompt and say, "There you go, figure it out."
It’s an absolute joke that we’ve got this incredibly sophisticated software and we’ve given it a command line interface and launched it to a hundred million people.
Benefits for people with English as a second language
41:53
For people who don’t speak English or have English as a second language, this stuff is incredible.
We live in a society where having really good spoken and written English puts you at a huge advantage.
The street light outside your house is broken and you need to write a letter to the council to get it fixed? That used to be a significant barrier.
It’s not anymore. ChatGPT will write a formal letter to the council complaining about a broken street light that is absolutely flawless.
And you can prompt it in any language. I’m so excited about that.
Interestingly, it sort of breaks aspects of society as well – because we’ve been using written English skills as a filter for so many different things.
If you want to get into university, you have to write formal letters and all of that kind of stuff, which used to keep people out.
Now it doesn’t anymore, which I think is thrilling…. but at the same time, if you’ve got institutions that are designed around the idea that you can evaluate everyone and filter them based on written essays, and now you can’t, we’ve got to redesign those institutions.
That’s going to take a while. What does that even look like? It’s so disruptive to society in all of these different ways.
Are we all going to lose your jobs?
46:39
As a professional programmer, there’s an aspect where you ask, OK, does this mean that our jobs are all gonna dry up?
I don’t think the jobs dry up. I think more companies start commissioning custom software because the cost of developing custom software goes down, which I think increases the demand for engineers who know what they’re doing.
But I’m not an economist. Maybe this is the death knell for six figure programmer salaries and we’re gonna end up working for peanuts?
[… later 1:32:12 …]
Every now and then you hear a story of a company who got software built for them, and it turns out it was the boss’s cousin, who’s like a 15-year-old who’s good with computers, and they built software, and it’s garbage.
Maybe we’ve just given everyone in the world the overconfident 15-year-old cousin who’s gonna claim to be able to build something, and build them something that maybe kind of works.
And maybe society’s okay with that?
This is why I don’t feel threatened as a senior engineer, because I know that if you sit down somebody who doesn’t know how to program with an LLM, and you sit me with an LLM, and ask us to build the same thing, I will build better software than they will.
Hopefully market forces come into play, and the demand is there for software that actually works, and is fast and reliable.
And so people who can build software that’s fast and reliable, often with LLM assistance, used responsibly, benefit from that.
Prompt engineering and evals
54:08
For me, prompt engineering is about figuring out things like – for a SQL query – we need to send the full schema and we need to send these three example responses.
That’s engineering. It’s complicated.
The hardest part of prompt engineering is evaluating. Figuring out, of these two prompts, which one is better?
I still don’t have a great way of doing that myself.
The people who are doing the most sophisticated development on top of LLMs are all about evals. They’ve got really sophisticated ways of evaluating their prompts.
Letting skills atrophy
1:26:12
We talked about the risk of learned helplessness, and letting our skills atrophy by outsourting so much of our work to LLMs.
The other day I reported a bug against GitHub Actions complaining that the windows-latest version of Python couldn’t load SQLite extensions.
Then after I’d filed the bug, I realized that I’d got Claude to write my test code and it had hallucinated the wrong SQLite code for loading an extension!
I had to close that bug and say, no, sorry, this was my fault.
That was a bit embarrassing. I should know better than most people that you have to check everything these things do, and it had caught me out. Python and SQLite are my bread and butter. I really should have caught that one!
But my counter to this is that I feel like my overall capabilities are expanding so quickly. I can get so much more stuff done that I’m willing to pay with a little bit of my soul.
I’m willing to accept a little bit of atrophying in some of my abilities in exchange for, honestly, a two to five X productivity boost on the time that I spend typing code into a computer.
That’s like 10% of my job, so it’s not like I’m two to five times more productive overall. But it’s still a material improvement.
It’s making me more ambitious. I’m writing software I would never have even dared to write before. So I think that’s worth the risk.
Imitation intelligence
1:53:35
I feel like artificial intelligence has all of these science fiction ideas around it. People will get into heated debates about whether this is artificial intelligence at all.
I’ve been thinking about it in terms of imitation intelligence, because everything these models do is effectively imitating something that they saw in their training data.
And that actually really helps you form a mental model of what they can do and why they’re useful. It means that you can think, "Okay, if the training data has shown it how to do this thing, it can probably help me with this thing."
If you want to cure cancer, the training data doesn’t know how to cure cancer. It’s not gonna come up with a novel cure for cancer just out of nothing.
The weird intern
I’ve used the weird intern analogy a few times before. Here’s the version Ronak and Guang extracted as the trailer for our episode:
1:18:00
I call it my weird intern. I’ll say to my wife, Natalie, sometimes, "Hey, so I got my weird intern to do this." And that works, right?
It’s a good mental model for these things as well, because it’s like having an intern who has read all of the documentation and memorized the documentation for every programming language, and is a wild conspiracy theorist, and sometimes comes up with absurd ideas, and they’re massively overconfident.
It’s the intern that always believes that they’re right. But it’s an intern who you can, I hate to say it, you can kind of bully them.
You can be like, "Do it again, do that again." "No, that’s wrong." And you don’t have to feel guilty about it, which is great!
Or one of my favorite prompts is you just say, "Do better." And it works. It’s the craziest thing. It’ll write some code, you say, "Do better." And it goes, "Oh, I’m sorry, I should…"
And then it will churn out better code, which is so stupid that that’s how this technology works. But it’s kind of fun.
Tags: blogging, podcasts, ai, prompt-engineering, generative-ai, llms
AI Summary and Description: Yes
Summary: The text captures insights from a podcast episode discussing the impact of Large Language Models (LLMs) on software engineering, usability, and societal implications. It delves into the transformative advantages of LLMs for professionals, especially those with domain expertise, while addressing concerns about job displacement and the risk of skill atrophy.
Detailed Description:
The podcast discussion features Simon Willison, an experienced software engineer, who shares his thoughts on various aspects of LLMs and their influence on software development:
* **Aligning LLMs with Expertise**:
– LLMs can provide substantial support for experienced engineers who know how to effectively query the models and critically evaluate their output.
– Willison mentions the importance of legal knowledge, underscoring the risks of relying on LLMs for critical decisions.
* **Usability of LLM Interfaces**:
– The initial impression of LLM interfaces can be overwhelming for users without a technical background, suggesting a need for more user-friendly designs.
* **Benefits for Non-native English Speakers**:
– LLMs democratize access to formal communication, enabling non-native speakers to articulate their needs and concerns more effectively.
* **Job Market Discussion**:
– Willison expresses a belief that the demand for skilled engineers will not diminish; instead, it may increase as companies seek custom software development.
– He raises concerns about the potential lowering of salaries in the industry.
* **Prompt Engineering and Evaluation**:
– The text highlights prompt engineering as a critical skill, particularly the challenges in evaluating different prompts for effectiveness.
* **Risks of Skill Attrition**:
– The discussion touches on the danger of dependency on LLMs leading to a decline in foundational skills among engineers.
– Willison reflects on a personal experience where he realized he overlooked a critical error due to reliance on LLM-generated code.
* **Imitation Intelligence**:
– He introduces the concept of “imitation intelligence,” defining LLMs as tools that learn from existing data rather than producing original concepts or solutions.
* **Analogy of the “Weird Intern”**:
– Willison creatively likens LLMs to a “weird intern”—overconfident but requiring guidance, thus emphasizing the need for human oversight in the coding process.
* **Societal Transformation**:
– He argues that the introduction of LLMs challenges existing evaluation systems in education and professional fields that rely heavily on writing skills, calling for a rethinking of how individuals are assessed.
Overall, the discussion sheds light on the capabilities and limitations of LLMs from a security and compliance perspective, particularly the necessity for engineers to maintain their skills while leveraging these transformative technologies. For security professionals, this underscores the importance of understanding LLMs’ implications for compliance, governance, and the evolving landscape of software development.