On the weird plague of LLM (aka AI)

Large Language Models (I much prefer this term to AI) – it’s a hard topic to avoid these days, and it’s also a vast and complicated one.

Perhaps if this had come along 15 years ago, I might have had some enthusiasm – but I’ve found a lot of cynicism has crept into my thoughts regarding technology over the past 10-15 years (possibly starting with the Edward Snowden revelations a little over 10 years ago).

But I noticed something since re-starting blogging that reminded me of some thoughts I’ve had of late regarding the impact LLM may have.

A new kind of spam

Of course we’ve dealt with spam in our lives for a long time, but in general, at least for me, it’s mostly been immediately recognisable.

When I restarted my photo blog over at Eidetic Opacity, imagine my delight when I had a number of followers appear after my first few posts. I thought to myself how I missed this, simple pleasure of knowing someone has seen the photos I posted and liked them enough to want to see more.

On further investigation though, I saw a pattern – all of the ‘people’ that had followed me had eerily similar WordPress blogs, with eerily similar posts that followed a similar pattern. In isolation, and without having seen it before, you might not have realised that all of these blogs were in fact being written with the aid of LLMs.

So sadly, I guess these follows were just an effort to get me to look at their sites and hopefully earn some advertising money.

A few interesting articles on this topic

This occurrence reminded me of a couple of recent articles in The Verge one titled “inside the AI Factory” another titled “A Storefront for Robots”. They’re both fairly long reads, but I think well worth spending time on. In the first we see that, despite proponents of the technology talking about it removing dull repetitive tasks, it has in fact spawned a huge industry of even duller and more repetitive tasks; the second, discussing how SEO requirements, now coupled with LLM writing tools, has resulted in a huge amount of garbage appearing on the internet.

So we end up in this strange situation.

We have increasingly convincing content being mass generated by huge spam farms all hoping to cash in on a drip feed of advertising dollars, while at the same time trying to fit in with the often strange dictates of search engine requirements.

Companies find it harder to be discovered amongst all the spam noise, thus spend more money on advertising to drive visitors to their sites, but also start using LLMs to generate more content that matches the weird requirements of SEO.

Google et al are laughing all the way to the bank.

The spam creators then use bots to visit spam farms and “click” on adverts to increase their revenue.

We end up with an internet where substantial portions of it are fake generated content, and potentially much of the ‘browsing’ is also fake – and it’s only going to get worse. This old article from 2018 in the NY Magazine suggests that studies back then show less than 60% of internet traffic is real (and it gets even worse on YouTube).

At what cost?

There are unfortunately numerous costs to consider.

Terrible “real” user experience

Have you ever searched for a recipe online wanting to quickly check what ingredients are needed, only then to be greeted with a huge amount of filler content at the start of the article, in-depth histories of the foods involved, descriptions of the methodologies, and of course, a heavy dose of advertising. Only at the very end of the article do you get a simple “Ingredients” and the short steps involved.

Every time I come across this it irks me – but this is what Google has mandated as the better format, and if a recipe site wants to surface in organic search results, it has to follow this format or languish in obscurity.

It goes further than recipes of course, many search results land you into bland generic keyword stuffed articles.

New Insecure Dull Repetitive Work

As discussed in the AI Factory Verge article, there is now this weird hidden workforce who spend their days chasing unstable work which is dull, repetitive, and entirely void of any obvious reason (due to tasks being parcelled up in such a way that workers don’t really know what they’re working on).

A hugely unregulated area where worker rights no doubt take a back seat.

Massive Energy Waste

Another thing that just doesn’t seem to get discussed nearly often enough regarding this whole situation is the massive waste of energy that is going into all of this.

It’s probably extremely difficult to accurately quantify, and in my own research it’s hard to find good information regarding this topic. I have found some articles discussing the energy costs of training a LLM. I also found some discussion on Stack Overflow estimated between 100Wh and 300Wh per ChatGPT request (the equivalent of charging your phone 20-60 times). If this is the case, it’s enormously wasteful – just take a few moments to write that email reply manually!

I dread to think the cost of generating images with tools like Dall-E and Midjourney.

And all the major tech companies are racing to put these tools right in front of people – write your emails, answer messages, draft document layouts etc etc.

All of these queries are using energy at a time we desperately need to be finding ways to reduce our energy consumption.

And what should we do?

I don’t have any clear answers for you here. I guess I’d like to raise awareness and make people think a bit about what is going on when they use a LLM to generate some stock text. Make some noise about what the environmental cost of all of this is. Is there any clarity on the working conditions of the people making these LLMs possible?

Only by making noise on these issues can it become either a regulatory topic, or a market force that requires tech companies to effect change in order to stay competitive.

I welcome any feedback though – like I say, it’s a complex and difficult topic, and a lot of the companies involve don’t release detailed information on either the ‘supply chain’ or the energy consumption.