Not because they are brilliant, but because they are pretty good at throwing pretty much all known techniques at a problem. And they also don't tire of profiling and running experiments.
If there's one thing LLMs are really, really good at, it's having a target and then hitting / improving upon that target.
If you have a comprehensive test suite or a realistic benchmark, saying "make tests pass" or "make benchmark go up" works wonders.
LLMs are really good at knowing patterns, we still need programmers to know which pattern to apply when. We'll soon reach a point where you'll be able to say "X is slow, do autoresearch on X" and X will just magically get faster.
The reason we can't yet isn't because LLMs are stupid, it's because autoresearch is a relatively new (last month or so) concept and hasn't yet entered into LLM pretraining corpora. LLMs can already do this, you just need to be a little bit more explicit in explaining exactly what you need them to do.
> The reason we can't yet isn't because LLMs are stupid, it's because autoresearch is a relatively new (last month or so) concept [...]
I'm not so sure. People have been doing stuff like (hyper) parameter search for ages. And profiling and trying out lots of things systematically has been the go-to approach for performance optimisation since forever; making an LLM instead of a human do that is the obvious thing to try?
The concept of 'autoresearch' might bring with it some interesting and useful new wrinkles, but on a fundamental level it's not rocket science.
Recently I tried Codex/GPT5 with updating a bluetooth library for batteries and it was able to start capturing bluetooth packets and comparing them with the libraries other models. It was indefatigable. I didn't even know if was so easy to capture BLE packets.
I find writing by hand is the best. LLMs spit out such linked-in writing that I don’t even want to read it. ;)
But that would be a good blog post and I got some travel coming up. But honestly it was just “oh here’s a BLE python library, see if we can get it running”. I prefer Codex because it seems to do well for guiding the LLMs for complete engineering changes.
Wireshark would do that. But you need to understand low level tools because in case on some BGP attack you all LLM developers will be fired in the spot.
Flakey internet connection: most of current 'soy devs' would be useless. Even more with boosted up chatbots.
And it's partially true. Offline documentation should be mandatory everywhere. Networks can be degraded tomorrow in the current 2nd Cold War we are living.
And, yes, the states and goverments have private backbones for the military/academia/healthcare and so on, but the rest it's screwed.
When the blackout the only protocols which worked fine where IRC, Gopher and Gemini. I could resort to using IRC->Bitlbee to chat against different people of the world, read news and proxy web sites over Gemini (the proto, not the shitty AI). But, for the rest, the average folk? half an our to fetch a non-working page.
That with a newspaper, go figure with the rest. And today tons of projects use sites with tons of JS and unnecesary trackers and data. In case of a small BGP attack, most projects done with LLM's will be damned. Because they won't even have experience on coding without LLM's. Without docs it's game over.
Also tons of languages pull dependencies. Linux distros with tons of DVD's can survive offline with Python, but good luck deploying NPM, Python and the rest projects to different OSes. If you are lucky you can resort to the bundled Go dependencies in Debian and cross compile, and the same with MinGW cross compiling against Windows with some Win32, SDL, DX support but that's it.
With QT Creator and MinGW, well, yes, you could build something reliable enough -being cross platform- and with Lazarus/Free Pascal, but forget about current projects downloading 20000 dependencies.
Heh, my preferred language is Nim which has good docs for the stdlib. It also does static binaries and runs on esp32 like a dream. I’m not worried about some internet downtime, but I also enjoy what I can guide LLMs to build for me.
The BLE battery syncing was a nice-to-have for an IoT prototype. Not something I wanted to spend hours digging through wireshark to figure out but fine for some LLM hacking.
> Offline documentation should be mandatory everywhere. Networks can be degraded tomorrow in the current 2nd Cold War we are living.
Eh? It's all about trade-offs. If our infrastructure is degraded enough that the internet goes down, I have more important things to do than work through a few more Jira tickets.
Especially since a lot of the work me and a lot of other folks are doing is delivered to customers via the internet anyway.
Not because they are brilliant, but because they are pretty good at throwing pretty much all known techniques at a problem. And they also don't tire of profiling and running experiments.