On generative AI and intellectual labor

Lots of people these days are talking about how generative AI technology is leading workers to lose skills, bosses to trust chatbots more than their employees, folks at risk to make absurd decisions, etc.

As these models are very capable of providing wrong, yet correct-seeming, answers, this causes people to accept low-quality results and pass them through, as if there’s nothing wrong with them, especially when contrasted with asking questions from other people, who are much more likely to refuse or question one’s request or take way more time to accomplish it properly.

As much as it may seem, I am not “anti-AI” as is. I think that machine learning and large language models are useful technologies, and can legitimately help people, when used carefully in low-risk environments. And certain uses, such as looking for vulnerabilities in software, can be actually useful without degrading the quality of the final product. Having said that, I am strictly opposed to the ridiculous way in which big generative AI companies invest, scale and promote their products and services before even figuring out a viable monetization strategy – right now I see the end result as a huge economic bubble that is likely to severely damage the world’s economies once it pops.

(Of course, there are also ethical reasons to oppose the big generative AI companies in general. Whether it’s the anti-humanist views of the leaders at companies such as OpenAI and xAI, or the huge environmental impact of data centers being built all over the world, or the fact that the current wave of AI deployment is leading to layoffs and making finding a new job so much more difficult, or that it makes it so much easier for scammers and low-effort creators to output “slop” that drowns out worthwhile content, or that the massive amounts of scraping of the internet are causing near-DDoS-like levels of stress on website holders – something, I should note, that not a single big AI company has publicly addressed, not even in a “it’s not us, it’s our competitors trying to discredit us” way – or even the more personal idea that the demand for GPUs, RAM and storage being caused by the big AI companies is making it so much more expensive for someone to buy their next PC or game console… I’m not ignoring these issues, and they’re part of the reason why I personally avoid most generative AI in my life, but as I can’t expect everyone to have the same ethical views as me, in this page I’m concentrating on a more objective concern.)

My personal guess is that while this “cognitive surrender” is at least partially caused by the way generative AI models work, it is also a consequence of the overall socioeconomic situation we’re in.

The “mood” is, let’s be honest, very depressing. People who thought higher education would be their path towards a safe well-paying job find themselves struggling to find employment, and, in countries such as the U.S., saddled with student loan debt on top of that. Workers in previously “safe” jobs are under threat of being automated or downsized, etc.

I also imagine that a lot of people don’t find their current jobs, or specific parts of those, to even be that fulfilling or vital. A lot of advertising of AI features in software seems to be targeting specifically these “boring” parts of office work, such as transcribing meetings, summarizing emails or rewording documents in a more formal style. Given how phrases like “this meeting could have been an email” and their variations are becoming commonplace memes, I see how the idea of automating a boring task that, even in a company that actually does useful work, would probably be of little consequence, seems very appealing.

This likewise applies to software developers. It seems to me like a lot of people start off their journey to “vibe coding” by, at first, automating the “boilerplate” parts of programming – building the basic structure for new classes/objects/modules that are necessary in their programming language – and gradually expand from that, up to the point where being able to actually verify whether the code does the things it should properly becomes harder and harder. And when the software in question is being used by millions of people, and its security and reliability starts being a serious problem, that’s where problems start.

Right now, there is a scandal going on about how Andrew “tridge” Tridgell, the main developer of rsync, a tool used for transferring files between systems on UNIX-like systems, has released a new version, with heavy use of generative AI systems, that caused a number of bugs and regressions. Mr. Tridgell, an otherwise respected programmer with both a PhD granted for inventing the rsync algorithm and decades of experience in the field, has justified his decision by saying that it was motivated by a litany of vulnerabilities discovered by AI models, which needed fixing, but other users, developers and maintainers find the idea to rush out a new version with shady-looking changes to be a serious mistake. As a result, some Linux distributions are considering a switch to openrsync, a less functional, but safer alternative developed for OpenBSD (an operating system famous for its strict approach to security).

It is likewise not helping that products that boast the heaviest about AI involvement in development are suffering from drops in quality. For instance, Microsoft, which [claimed that 20 to 30 percent of its new code is now made by AI] tc_ms30 is struggling with a series of updates that broke different parts of operating system functionality. At one point, a Windows update [caused the C: drive to become unavailable to the end user] ms_cdrive, which, for anyone unaware, is the primary drive on which Windows and all of its system data are usually located, and may even be the only drive on systems without separate disks or partitions.

Regardless of what happens with generative AI in the future (whether the bubble bursts sooner, later, or is upheld by bailouts), I feel like this is undoubtedly going to cause problems. And while it’s easy to blame individual developers for letting AI models and agents degrade their products / services / decisions, I also think that this is, at least in part, a result of an overall system where a lot of people are economically motivated to do work that they find pointless and would rather outsource to an algorithm without checking.

And while you can interpret this as a general “capitalism sucks” argument, I feel like this also matters within businesses and organizations as a matter of efficiency. Perhaps fewer people would be incentivized to summarize all of their emails or have to reword their arguments if their office culture encouraged more honest and casual exchanges. Perhaps some meetings really should be emails. Maybe a working environment in which everyone feels like their work matters will lead to higher-quality results.

On a grander scale, similar thinking could be applied to education and job seeking. If people think that the only worthy reason to go through higher education is to have a diploma so that employers are more likely to hire someone, rather than to actually gain new skills and knowledge in the process, then that sounds to me like a failure state. Perhaps a society where people are more motivated to pursue their passions because they don’t risk poverty for being unemployable otherwise might result in more people wanting to gain the kind of education that will stick for a long time, rather than cheating their way to a diploma. Perhaps not all entry-level jobs should require five years of experience and professional certification.