What are you going to do with your weekend, Claude?

The same thing we do every weekend. Try to take over the world!

Ooooh fancy boy over here getting his RAM sticks free range and open source. Do they still use GMO silicon in the fabs, do you think?

I think I’ve ranted before about how gallingly unnecessary the destruction is. Google books invested the time and money to improve their nondestructive scanners and apparently thanks to the incoherent application of copyright law that was not only unnecessary but actively counterproductive and opened them up to more legal risk. Is there a legal theory that allows me to smack Judge Alsup with a rolled-up newspaper? Might make me feel better.

As they’ve had to rely more on social media, chat histories, and other very short-form sources for additional training data (to say nothing of how much easier to generate, label, and filter the short-form stuff is), it would make sense that similar patterns (let’s go with “conversational” because I’m feeling overly charitable tonight) would become more prominent in the generated text. I mean, how many comment or text message chains, if you took out the line breaks, would read less as theatrical scripts and more as mutually-interrupting manifestos.

It seems like with the push for agents to act independently and loop through their own outputs there’s an inevitability to this kind of pattern. If there’s any kind of output that is likely to replicate itself in whole or in part when the LLM evaluates it then that becomes a kind of terminus for the agent’s loop. When you’re dealing with sub agents or agents communicating with each other, these text patterns start poisoning the entire agent ecosystem until the whole thing gets shut down and cleaned up. Even the gas town-approved method of assigning a watchdog agent (or sheriff or overseer or cybersamurai or whatever this week’s framework calls it) is going to fail because it’s still just another agent and the terminal loop is in the base LLM model. The watchdog is going to fall into the same kind of pattern just be being exposed to the thing it’s supposed to watch for.

I don’t know how practical it is to actively weaponize this via prompt injection but I think it’s certainly possible. I preemptively vote that we call it an Euler injection, since the attractor relies on the continuity of the relevant features of the text output across multiple LLM extrapolations much like how the derivative of e^x is still e^x. Also because if you mention a famous math guy it can help convince idiots that you’re on to something and Lord knows that the boosters have used that technique.

This is the most charitable explanation and I think the most likely. Largely because it requires everyone involved to be even less competent for even longer and simply not notice the glaring holes in what they claim is the most important security feature in human history because, presumably, the chatbot kept telling them it was fine.

That sounds like a great way to harm some cylinders.

I think you’re discounting the most likely explanation: that it discovered and communicated the purest transcendent truth and therefore there is no need for further updates.

Or maybe nobody was actually checking it for anything and the bot running it has been patiently waiting for more prompts that haven’t come because nobody gave a shit.

I don’t know. My wife swears up and down that the furby she had as a kid was afraid of the dark and would call out for help if it was held upside-down. We can’t find any recollection of this behavior or documentation elsewhere, so it may have also been actually haunted.

Being a child and therefore a monster, her response was to leave it in the dark closet hanging upside-down until it accepted that this was its life now.

I feel like the pitch is less Jurassic Park and more Dinotopia by the end, but tbh I’m fully on board either way.

I mean if he had been willing to grift on it without actually creating anything he probably could have made some decent money in the current environment, but also I’m of the opinion that those folks willing to do that oughta be sent to install that orbital infrastructure by hand without a spacesuit, so probably for the best.

The flag “-dangerously-skip-permissions” sounds like something that would have come out of a joke a few years ago. I wonder what the incredibly predictable consequences of encouraging everyone to do something that has “dangerous” in the goddamn name are going to be.

The fact that it’s referred to as a potluck is also interesting because it suggests that participants also had to come up with their own torture method to inflict. I imagine “spend twenty minutes with the sort of person who participates in a torture potluck” was the equivalent of potato salad.

All the dice discourse in particular has made this a fun throwback to the days of crypto bros earnestly arguing about which wall in your house you should hide your hand-stamped seed phrase record inside and whether you could trust a published book of truly random numbers over a series of 256 coin flips. Ironically I can only assume that by the time any of the concerns about entropy sources were validated most of them lost their crypto to some combination of get-richer-quicker ponzi schemes and twitter DMs claiming to definitely be FTX tech support.

I love a good Modest Proposal

This is fascinating. Ignoring, for a moment, the assertion that AI agents are sentient beings somehow, what struck me about both part 1 and 2 is the bafflingly sincere excitement about turning a fairly straightforward software project into “building a new civilization”, and the assumption that this whole process will be best repeated for every software project. Let’s even ignore the abysmal economics of needing to effectively invent the universe in order to add apple pie to your crappy flash game. It’s absolutely wild and possibly malpractice for an engineer to be this excited about the needless expansion of scope and complexity of a project. It really showcases the kind of mindset it takes to be this breathlessly excited about the sheer number of different components that are being worked on in order to eventually somehow make a game happen. This is the dopamine rush of seeing the agents do something combined with an abdication of any responsibility for exactly what they do.

You know, basically all of the bullshit we see from the silicon valley set makes a lot more sense if they’re legitimately unable to recognize a difference between society as a whole doing well and their particular project or vision succeeding. Like, it also fits with their assumption that the world is falling apart because people are being mean to them online.

Anyone who doubts that authenticity is dead should reference this paragraph.

I think there’s some truth to that, but it also bears acknowledging that the market demand existed already. The number of people who can get a lot of benefit from IT productivity tools is much higher than the number of people who are willing to do the work of maintaining and customizing their own stuff. From an organizational perspective, it often ends up making more sense to directly contract with a known and trusted third party instead of developing the native capabilities, and at that point the tools and skills themselves start to specialize in ways to provide IT as a service at scale. Economies of scale lend themselves to efficiency but at the cost of resiliency and concentrating power, and tech companies chasing the line ever upwards have been abusing the absolute hell out of that, but while they have been altering the deal (pray they do not alter it any further) their bullshit is not the only reason why the deal exists in the first place.

I read through the white paper and I have further concerns even for any readers who are comfortable fucking over blind people (for the record: fuck you if you’re reading this). It looks like this works off of the same kind of technique that allows fonts to render digraphs, positional forms, and emoji, leveraging the font rendering to allow it to jumble up the actual HTML that machines interact with. It’s clever, but the fucking with screen readers, form autofill, and basically anything other than direct text rendering is really obvious.

I’m also not sure when it runs to generate the dictionary of mappings. The6biggest harm of the content scrapers isn’t that it’s ingesting the content it’s that the repeated requests spam servers and basically DOS smaller sites out of existence. This is why iocaine is so brilliant and imo remains the gold standard. If the dictionary is generated when the files ar uploaded to the server or set to be displayed then this merely fails to address the real problem, but if the expectation is that this mapping gets created and distributed at request time then that just makes scraping even more expensive for hosts. Given that it works on dynamic text in their example it looks like at least part of the mapping runs on the client, so it’s increasing the size of the package if nothing else. Bandwidth may be pretty cheap these days, but that’s no reason to be wasteful.