I stumbled on a
post about Gas Town, a chaotic system in which an A.I. agent oversees other A.I. agents and the person (proudly) never looks at the code. It costs him $5000 a month to run.
A lot of that post is about how some version of that is the future, and I don't debate that. I've learned that the future is not what's most efficient or effective, short-run or long-run, but rather what interests those with capital.
However, I just want to note how quickly the "Overton window" on understanding what your programs are doing and how predictable they are has shifted.
“Should developers still look at code?” will become one of the most divisive and heated debates over the coming years. You might be offended by the question, and find it absurd anyone is asking. But it’s a sincere question and the answer will change faster than you think.
I mean, yes, the answer will change because of the interests of billionaires.
I’m already seeing people divide along moralistic, personal identity lines as they try to answer it. Some declare themselves purist, AI sceptic, Real Developers who check every diff and hand-adjust specific lines, sneering at anyone reckless enough to let agents run free. While others lean into agentic maximalism, directing fleets from on high and pitying the mass of luddites still faffing about with manual edits like it’s 2019. Both camps mistake a contextual judgement for a personality trait and firm moral position.
This person is most likely not right wing, but putting aside the A.I. stuff, there's (probably accidentally) a few techniques here used by the right to move things Trumpward:
1. "Both sides" talk. I think you all know how that works.
2. Using "personal identity" to discredit a side.
3. Shrugging things off with "context." e.g. DHS might say, "We can't decide if an ICE agent should shoot a protestor in every situation. It depends on context." An engineering VP might say, "We can't say if some person should know how our software works in every situation. It depends on the context."
4. After the scramble is over, setting the "center" between the "two sides" pretty close to the "right". (Sorry about all the "quotes.") Sure, maybe NASA and medical device managers should look at their code, but Amazon should never look at code.
Specific to the A.I. issue, though, there's another argument that I'm sure is thrown a lot: It is just another level of abstraction.
Can I tell you what happens in the CPU cores when something I write in JavaScript executes? No.
But I can read JS code and know what's happening on that level, and I don't have to trust a non-deterministic agent to tell me what's happening. I can get a ground truth, even if it's not at the lowest level. That's important, even if you're making a commerce application or an art program.
I unfortunately can see plenty of people saying, "No, it is not important. Drop some orders, let your shader crash, have the context menu not appear sometimes — you'll make it up by having A.I. rewrite it and try again."
That may be the crux of my disagreement with this age of software. I like software because it's sort of reified thinking. Now software engineers are expected to be incredibly thoughtless slot machine operators.