Mining Misinformation Middens

Every landfill has a bit of gold in it ... but how do you get it?

Every landfill in this country has a bit of gold in it thanks to discarded electronics. If you could snap your fingers and extract it, that would be fine. But it’s in a heterogenous mess and it wouldn’t begin to to be cost effective.

There are misinformation middens out there, trash piles assembled by people trying to make sense of something, but they lack the training to get at the objective reality, or they have the time to spend on such pursuits because they’ve got an organic or structural brain problem that keeps them isolated. Given that I’ve suggested that you Stop Ingesting Crap, what can be done?

Faulty analysts produce results with distinctly negative value, unless you happen specifically to be evaluating what they’re doing, in which case a fulsome exam is in order. But there are a couple instances where people, whose conclusions I find to be erratic at best, actually are worth a look …

Because they’re erratic curators in an area of interest that is unserved by quality sources.

Attention Conservation Notice:

I’ve counseled avoiding those who are not well and that’s still the order of the day, but there are cases where J. Random Crazypants will be the ONLY source you have available for something that DOES matter. This post will cover some of the methods I use, so it’s about collection discipline and cognitive defense in an environment that is de facto deceptive. If you’re neither reaping nor threshing, you can probably just move on. (But if you do I’d wonder why you’re here at all tbh).

Middenism

This would help if you have a midden in mind in the first place. I am struggling a bit here, there are a couple I do have to review at times, but I don’t want to start shit by naming them. I just looked at the Insane Clown Posse folder in my Inoreader and picked the first likely subject - Arthurania. I am not sure how this one initially made it into ICP, perhaps chattering about Johnathan Buma, or maybe because he’s got an issue with Joe Menn. You see Rudy Giuliani in red here because I’m using an Inoreader highlighter to pick out groups of names.

Here’s what an Inoreader highlighter looks like. Have I mentioned that Inoreader is really great for overall situational awareness? Probably at least a couple times but it never hurts to repeat this.

So within Inoreader, since I’ve got 395 feeds I’m following, I can just drop a name and where ever they’ve turned up I’ll get some hits. Not a lot of recent stuff on FBI whistleblower Johnathan Buma but I’ve got it without having ever read much of Charles Johnson, or any of Arthurania prior to picking him as a midden example.

Now I could fiddle with that search box and conjure a way to limit results to just Arthurania, but since I operate Disinfodrome, a document indexing service, and I’ve got a bit of automation in Inoreader that sends starred articles straight to Dropbox … all 223 articles from that Substack are easily captured in PDF form. Just select them all, click the star, and wait patiently.

This took roughly two minutes. I’m not sure about the 27 missing, it may be that they predate my adding Arthurania’s feed to ICP, and if I was the first Inoreader user to ever do that, then those first ones are over the event horizon.

Mastication:

So I can search, and you can, too, as soon as you get yourself an Inoreader account. Did I mention how much I like Inoreader? OK then, moving on …

Search is good. Highlighters are good. But I’ve gone and captured these PDFs, which I could slip into an Open Semantic Search instance on Disinfodrome. The system already has a demo area that contains all the Congressional documents for the Trump Russia investigations.

There are lots of other things lurking back there, among them 15,700 Twitter threads from about a hundred sensible observers that were automatically captured with Threadreader app. Right next to that there are a bit more than 93,000 individual tweets that someone fished out of the Wayback Machine. It’s been years since anyone cared about any of that stuff, but it isn’t much disk space and it’s a nice historical record.

There’s even a Substack folder, which had 3,580 items in it before I added Arthurania just now. You never know when you might need to search a pile of Substacks, and there are a couple that got preserved because it was feared, and rightly so, that they were about to go missing.

Searchability is good, but what’s coming in July should be either a Retrieval Augmented Generation AI solution, or maybe there will be a GraphRAG solution that will be mature enough to use. I think I first mentioned this in Artificial Intelligence vs. Search Engines and the great big pig that’s been struck in our collective python - bulk translation of content from leaks - looks like it might have a workable solution. Once that first hurdle is cleared some of these other lingering issues should fall pretty quickly.

Moulding:

OK, so getting some stuff in a pile is a nice start and maybe that’s the bulk of your personal battle - just finding all three times some odd source mentioned some specific thing of interest, and once you’ve got that you’re off to exploit the find in some fashion.

I have things like that, but I have other things that require sterner measures. I’ve published Maltego Intro, Actual Maltego Link Analysis, and there are probably some other things like that around here. There are a goodly number of problems I face which are mine specifically because they’re far too large to get one’s head around in a single examination. The biggest example I have is the MAGA Meltdown graph. This morning it’s 1,428 citation URLs for 3,270 people and 1,245 organizations that have appeared in conjunction with Trump et. al.’s troubles over the last four years.

The point here is that if you’ve got a complex problem you probably need a non-linear representation of the information - aka a graph, and sooner or later you’ll need a timeline of events.

But presuming you’re not in the business of analyzing what’s going on in an influence operation, with the attendant reduced regard for the veracity of the content, the thing about handling a misinformation midden is this:

It’s bad enough that you even have to review such things, but you can NEVER let them bleed into your quality sources.

That’s just deadly serious analytical tradecraft advice - when large scale professional teams have to handle something like this, they will insert a filter layer between the sketchy content and the sense making. Things are busy enough that I can ask people to look at certain things, but I’d really need someone with skills akin to mine I could direct about half time in order to really do this right.

But since this is just a hobby gone terribly wrong, I avoid noisy, flaky sources as much as I can, and grit my teeth when I can’t dodge.

Conclusion:

When this Substack was rebooted at the end of August last year, I was marveling over the fact that a series of incremental improvements suddenly lead to more or less normal cognitive function, after sixteen years of post-Lyme troubles.

The avowed purpose of this thing is ensuring that new people don’t have to learn everything the hard way. Part of doing that is honoring the constraints of my “battlefield commission” - I can’t very well be a seasoned observer if I’m periodically rushing to the front line in a public fashion, thereby creating an attack surface.

This article is meant as an exposition on what to do with a source that you don’t find entirely trustworthy. I’m not really sure what Arthurania’s deal is and I don’t have any motivation to really dig in and find out. Maybe I’ve even misfiled him by putting him in ICP in the first place.

The stuff that truly matters ends up in Disinfodrome and it’s there so others can explore content that might otherwise be out of reach. It’s entirely likely that, having used his feed as an example here, I’ll never have reason to inspect it again. So it goes.