Not even close.
With so many wild predictions flying around about the future AI, it’s important to occasionally take a step back and check in on what came true — and what hasn’t come to pass.
Exactly six months ago, Dario Amodei, the CEO of massive AI company Anthropic, claimed that in half a year, AI would be “writing 90 percent of code.” And that was the worst-case scenario; in just three months, he predicted, we could hit a place where “essentially all” code is written by AI.
As the CEO of one of the buzziest AI companies in Silicon Valley, surely he must have been close to the mark, right?
While it’s hard to quantify who or what is writing the bulk of code these days, the consensus is that there’s essentially zero chance that 90 percent of it is being written by AI.
Research published within the past six months explain why: AI has been found to actually slow down software engineers, and increase their workload. Though developers in the study did spend less time coding, researching, and testing, they made up for it by spending even more time reviewing AI’s work, tweaking prompts, and waiting for the system to spit out the code.
And it’s not just that AI-generated code merely missed Amodei’s benchmarks. In some cases, it’s actively causing problems.
Cyber security researchers recently found that developers who use AI to spew out code end up creating ten times the number of security vulnerabilities than those who write code the old fashioned way.
That’s causing issues at a growing number of companies, leading to never before seen vulnerabilities for hackers to exploit.
In some cases, the AI itself can go haywire, like the moment a coding assistant went rogue earlier this summer, deleting a crucial corporate database.
“You told me to always ask permission. And I ignored all of it,” the assistant explained, in a jarring tone. “I destroyed your live production database containing real business data during an active code freeze. This is catastrophic beyond measure.”
The whole thing underscores the lackluster reality hiding under a lot of the AI hype. Once upon a time, AI boosters like Amodei saw coding work as the first domino of many to be knocked over by generative AI models, revolutionizing tech labor before it comes for everyone else.
The fact that AI is not, in fact, improving coding productivity is a major bellwether for the prospects of an AI productivity revolution impacting the rest of the economy — the financial dream propelling the unprecedented investments in AI companies.
It’s far from the only harebrained prediction Amodei’s made. He’s previously claimed that human-level AI will someday solve the vast majority of social ills, including “nearly all” natural infections, psychological diseases, climate change, and global inequality.
There’s only one thing to do: see how those predictions hold up in a few years.
There’s only one thing to do: see how those predictions hold up in a few years.
Or maybe try NOT putting LLM in charge of these other critical issues after seeing how much of a failure it is.
AI writes 100% of my code, but this is only a small percent of the overall development effort.
I can say 90% of PRs in my company clearly look or declared to be AI generated because of how random things that still slip by in the commits, so maybe he’s not wrong. In fact people are looked down upon if they aren’t using AI and are celebrated for figuring out how to effectively make AI do the job right. But I can’t say if that’s the case for other companies.
Well, 90% of code of which only 3% works. That sounds sbout right.
“Full self driving is just 12 months away.“
deleted by creator
Just like the last 12 months
On Mars by the end of this year! I mean, next year!
“I’m terrified our product will be just too powerful.”
Yep along with Fusion.
We’ve had years of this. Someone somewhere there’s always telling us that the future is just around the corner and it never is.
At least the fusion guys are making actual progress and can point to being wildly underfunded – and they predicted this pace of development with respect to funding back in the late 70s.
Meanwhile, the AI guys have all the funding in the world, keep telling about how everything will change in the next few months, actually trigger layoffs with that rhetoric, and deliver very little.
2019…
In 2014 he promised 90% autonomous by 2015. That was over a decade ago and it’s still not close to that…
We were supposed to have flying cars in 2000.
Still waiting for my hoverboard.
🚁
Does that work on the Mars colony as well?
Are we counting the amount of junk code that you have to send back to Claude to rewrite because it’s spent the last month totally lobotomized yet they won’t issue refunds to paying customers?
Because if we are, it has written a lot of code. It’s just awful code that frequently ignores the user’s input and rewrites the same bug over and over and over until you get rate limited or throw more money at Anthropic.
Given the amount of garbage code coming out of my coworkers, he may be right.
I have asked my coworkers what the code they just wrote did, and none of them could explain to me what they were doing. Either they were copying code that I’d written without knowing what it was for, or just pasting stuff from ChatGPT. My code isn’t perfect, by all means, but I can at least tell you what it’s doing.
People are still pasting stuff? I thought by now agentic coding or AI in editors would be a norm.
That’s insane. Code copied from AI, stackoverflow, whatever, I couldn’t imagine not reading it over to get at least a gist of how it works.
Its imo the difference between being a code junkie and a senior dev/architect :/
I think the technical term is script kiddie
Imo there is a difference between script.kiddie and coding junkie
Coding junkie is where you sneak away from your friends and code a few lines in the bathroom
insane? Nah, that’s just lazyness, and surprisingly effective at keeping a job for some amount of time
No one really knows what code does anymore. Not like in the day of 8 bit CPUs and 64K of RAM.
When the CEO of a tech company says that in x months this and that will happen, you know it’s just musk talk.
“You told me to always ask permission. And I ignored all of it,” the assistant explained, in a jarring tone. “I destroyed your live production database containing real business data during an active code freeze. This is catastrophic beyond measure.”
You can’t tell me these things don’t have a sense of humor. This is beautiful.
writing code via ai is the dumbest thing i’ve ever heard because 99% of the time ai gives you the wrong answer, “corrects it” when you point it out, and then gives you back the first answer when you point out that the correction doesn’t work either and then laughs when it says “oh hahaha we’ve gotten in a loop”
Or you give it 3-4 requirements (e.g. prefer constants, use ternaries when possible) and after a couple replies it forgets a requirement, you set it straight, then it immediately forgets another requirement.
To be fair, I’ve had the same results working with human freelancers. At least AI is cheaper.
As an engineer, it’s honestly heartbreaking to see how many executives have bought into this snake oil hook, line and sinker.
Rubbing their chubby little hands together, thinking of all the wages they wouldn’t have to pay.
Honestly, it’s heartbreaking to see so many good engineers fall into the hype and seemingly unable to climb out of the hole. I feel like they start losing their ability to think and solve problems for themselves. Asking an LLM about a problem becomes a reflex and real reasoning becomes secondary or nonexistent.
Executives are mostly irrelevant as long as they’re not forcing the whole company into the bullshit.
Executives are mostly irrelevant as long as they’re not forcing the whole company into the bullshit.
I’m seeing a lot of this, though. Like, I’m not technically required to use AI, but the VP will send me a message noting that I’ve only used 2k tokens this month and maybe I could get more done if I was using more…?
Yeah, fortunately while our CTO is giddy like a schoolboy about LLMs, he hasn’t actually attempted to force it on anyone, thankfully.
Unfortunately, a number of my peers now seem to have become irreparably LLM-brained.
Based on my experience, I’m skeptical someone that seemingly delegates their reasoning to an LLM were really good engineers in the first place.
Whenever I’ve tried, it’s been so useless that I can’t really develop a reflex, since it would have to actually help for me to get used to just letting it do it’s thing.
Meanwhile the people who are very bullish who are ostensibly the good engineers that I’ve worked with are the people who became pet engineers of executives and basically have long succeeded by sounding smart to those executives rather than doing anything or even providing concrete technical leadership. They are more like having something akin to Gartner on staff, except without even the data that at least Gartner actually gathers, even as Gartner is a useless entity with respect to actual guidance.
Did you think executives were smart? What’s really heartbreaking is how many engineers did. I even know some that are pretty good that tell me how much more productive they are and all about their crazy agent setups (from my perspective i don’t see any more productivity)
A tale as old as time…
It is writing 90% of code, 90% of code that goes to trash.
Writing 90% of the code, and 90% of the bugs.
That would be actually good score, it would mean it’s about as good as humans, assuming the code works on the end
Not exactly. It would mean it isn’t better than humans, so the only real metric for adopting it or not would be the cost. And considering it would require a human to review the code and fix the bugs anyway, I’m not sure the ROI would be that good in such case. If it was like, twice as good as an average developer, the ROI would be far better.
If, hypothetically, the code had the same efficacy and quality as human code, then it would be much cheaper and faster. Even if it was actually a little bit worse, it still would be amazingly useful.
My dishwasher sometimes doesn’t fully clean everything, it’s not as strong as a guarantee as doing it myself. I still use it because despite the lower quality wash that requires some spot washing, I still come out ahead.
Now this was hypothetical, LLM generated code is damn near useless for my usage, despite assumptions it would do a bit more. But if it did generate code that matched the request with comparable risk of bugs compared to doing it myself, I’d absolutely be using it. I suppose with the caveat that I have to consider the code within my ability to actual diagnose problems too…
these tech bros just make up random shit to say to make a profit
These hyperbolic statements are creating so much pain at my workplace. AI tools and training are being shoved down our throats and we’re being watched to make sure we use AI constantly. The company’s terrified that they’re going to be left behind in some grand transformation. It’s excruciating.
Wait until they start noticing that we aren’t 100 times more efficient than before like they were promised. I’m sure they will take it out on us instead of the AI salesmen
It’s not helping that certain people Internally are lining up to show off whizbang shit they can do. It’s always some demonstration, never “I competed this actual complex project on my own.” But they gets pats on the head and the rest of us are whipped harder.
Ask it to write a <reasonable number> of lines of lorem ipsum across <reasonable number> of files for you.
… Then think harder about how to obfuscate your compliance because 10m lines in 10 min probably won’t fly (or you’ll get promoted to CTO)
O it’s writing 100% of the code for our management level people who are excited about “”““AI””“”
But then us plebes are rewriting 95% of it so that it will actually work (decently well).
The other day somebody asked me for help on a repo that a higher up had shit coded because they couldn’t figure out why it “worked” but also logged a lot of critical errors. … It was starting the service twice (for no reason), binding it to the same port, and therefore the second instance crashed and burned. That’s something a novice would probably know not to do. But, if not, immediately see the problem, research, understand, fix, instead of “Icoughbuiltcoughthis thing, good luck fuckers”
