WWWord Salad

AI LLMs destroying ... most everything.

Schizophrenics often speak “word salad”. This is a spectrum, ranging from completely incoherent babble to the shallow end where one finds things that get passed off as poetry … or perhaps prophecy.

While Large Language Models are getting … both larger … and smaller, in terms of needs, and more accurate because they’re “distilling” each other, sans human curated input, I think they’re going to suffer the same fate as Chaos theory. There was a period where any visually interesting mathematical construct got research money thrown at it. Then as suddenly as it began, the “so what” factor engaged.

There are a LOT of good things that will come from AI, but it’s just a flock of stochastic parrots right now.

Attention Conservation Notice:

This was actually written by a mildly annoyed human, finally freed from months of PHP suffering. Any slop herein will be entirely intentional, and probably for comic relief.

Hallucinations:

The evening ChatGPT first became broadly available I happened to be trying to identify an inter-war cruiser based on a grainy photo. Many images of ships from 1919 to 1939 are grainy, and from the crucible of World War II the only ships that were used as their designers intended were minesweepers and ocean escorts.

Since I’m a fan of Drachinifel, I know more about these things than the average bear.

So when I described a cruiser with three forward turrets and two aft, having the second lower than first and third, ChatGPT quickly came to the point of claiming it was an Iowa class battleship and that the second turret was lower “to facilitate hitting targets at close range”. The Iowas are battleships, four times the size of even large cruisers of the era, they have three turrets, not five, and all such ships have secondary batteries meant for close in work. No captain in the history of post-Dreadnought ships ever flooded combat side compartments to further depress guns, but there are instances of precisely the opposite - the Texas counterflooded to increase her inland reach during the Normandy landings.

ChatGPT had been fed a lot of information about the Iowas, they’re probably the best known battleship of that era, and it apparently jumped from three forward turrets to three in total. The gun depression thing … I guess maybe because there were lots of blog posts about this scene in Greyhound during the time ChatGPT was training?

So right away I knew ChatGPT was not to be asked unbounded questions requiring expertise.

Programming:

As I described in My Big AI Successes, LLMs have proven useful for programming. I’ve been using Unix for forty years, but I am dramatically more fluent with the find command in the last couple months, and I’ve discovered half a dozen others I’d never previously seen that have proven invaluable. I’ve probably rewritten the seq command in Pascal, Perl, and Python at least a hundred times each …

But one of the biggest benefits is that it’ll quickly produce an example of how to use an unfamiliar API or library. Declarative programming is burned into my brain, but Python data structures have been painfully slow to stick. We’re talking thirteen years and I have to review every time I have a break of even a few weeks.

But trying to get a more complex solution? Like asking a complex question about an old warship, an LLM will just spit out something that sounds right … but will never work. I went around and around and around with PHP issues, finally peeling off truly annoying bits and implementing them as external python scripts.

Prosaic Prose:

One of the Unix things that’s truly an acid test of an LLM is … regular expressions. As an example, I wrote most of this by hand, then gave it to ChatGPT to finish.

ls | grep ^.*[^a-zA-Z0-9._%+\-@].*$

Utterly mystifying? If you’ve got a folder full of directories where each one’s name is supposed to be a correct email address, that will show you the ones that are incorrect.

Like concise, well written English, that string of mumbo jumbo packs a lot of details into a little space. LLMs are utterly mystified by it, trying to get one to write a regex from scratch, using only words to describe it, rather than giving it a partly finished example, is a tail chase in the making.

Now think about your current world. If you’re reading this, you’re likely at least conversational in English, you know a portion of the idiomatic use of the language, and you have some sense of what’s happening in the news. Is there an LLM out there that is trained on TODAY? Not usually, no. The only way to get one top of this level of language use would be a well done LLM, a bunch of carefully curated idiomatic English, and a broad feed of current events.

This probably exists, but not for groundlings. You’d need money, a trained staff, and a reason to be putting those resources into it. We used to be able to do this with social media platforms - like with Twitter, back in 2010, when Progressive Congress News started with a $15 budget for the domain name, and rose to reaching 23% of Congressional staff eighteen months later.

Can’t do that today, that former public common is owned by a delusional oligarch, and the world in general is in what we’re calling “the post truth era”.

Conclusion:

I’ve been thinking about this problem a bit, here and there, as I make my way in the world. Assuming I can keep it together for the summer, I think I have any idea of how to go about building something akin to PCN again.

PCN was a financially pointless effort, at least for me … but if there were a way to plant such a thing within Shall We Play A Game? …

Just kidding. There is. And I’m confident enough it’ll work that I even reassembled that Proxmox system I broke down when I moved. Now I just need to watch the ArangoDB material on their GraphRAG implementation …