Because passing off basic security configuration to a system that basically ensures that even if it is without major errors it’s basically impossible to understand and follow is definitely not a bad idea, especially if you’re going to try and operate this for more than a couple of weeks or in on infrastructure that isn’t perfectly isolated and specific to this app.

So the argument is that the hypothetical future generations that longtermists value might be too dumb to care about because of black and brown people reproducing miscegenation “dysgenic pressures”. But if we assume, as they appear to, that longtermists should be taken seriously rather than pointed at and mocked, then the math involved would basically necessitate that “certain” people alive today should also merit effectively zero moral consideration. Something tells me that they get real conceptual real fast and start avoiding the shut up and multiply people hard.

My general approach telegraphs a receptiveness to “sober” forms of argument, an interest in counterintuitive strategic thinking, a distaste for wild speculation, an aversion to nakedly ideological perspectives, and a sense that the past is important for understanding the future. My students probably make some effort (implicit or explicit) to speak in ways that respond to this.

I’m going to leave this part of the author’s self-description here and resist the urge to comment on it.

Perspective #6: AI will not go rogue because AI does not have, and is likely incapable of having, desires.

This one I found interesting largely because it suggests an inability on the writer’s part to really understand point 4, despite featuring one of the better sneers on the subject (“as if Ford made a car with faulty brakes and then tried to blame this on ‘rogue cars.’”). Maybe I’m overstating something here, but the way I see it the difference between the rogue AI and faulty AI scenarios aren’t really in the distinct events of the scenario. Faulty AI could absolutely do as much damage as “rogue” AI if it was plugged into all the systems that would be necessary to, say, liquidate humanity and pull the iron from their blood. The difference is entirely in framing and responsibility. We should ask questions now about how these systems are used, what their limitations are, and how we deal with that. Instead, the Rogue AI people presuppose that we ignore all of those questions and opportunities to mitigate harm that these faulty systems can do and skip straight to the part where they have universal and immediate power over everything, and then use the idea that the AI went rogue and has agency to avoid how obviously bad of a decision this would be. The fact that this is also the part where the AI companies make a fucktillion dollars and finally show them, show them all, muhuhahaha is, I’m sure, irrelevant to their sober analysis.

Happy birthday, my friend! Glad to hear that the steakhouse delivered.

The episode dropped and I’m like 2/3 in and have already passed my previous record for the number of times I’ve had to pause an episode of OS to make a little angry noise.

There’s also something to be said of the inherent narcissism of the influencer economy? Like, I prefer my video essayists to have more of a burned-out academic vibe so CJ has been adjacent to but not on my radar. That being said, having a very distinct aesthetic is pretty common in the space because having a hook is what brings in eyeballs and thus patreon subs or superchats or whatever else turns that into an actual career. I would sooner assume he’s using a similar strategy to manipulate viewers into clicking his face rather than that he’s actively following the PUA playbook.

the woman herself shows up in the substack comments with the most pitiful display of “I’m just a smol self-taught bean why are you being so mean to me?”

Erika rightly responds by calling out Aella’s own attitude towards the people who know how to apply the level of rigor to the data that she (apparently admittedly?) doesn’t. Like, you can’t simultaneously be this dismissive of the norms and processes of the scientific establishment and also claim ignorance and inability when your inability or refusal to follow those norms leads you to crap conclusions.

What are you going to do with your weekend, Claude?

The same thing we do every weekend. Try to take over the world!

Ooooh fancy boy over here getting his RAM sticks free range and open source. Do they still use GMO silicon in the fabs, do you think?

I think I’ve ranted before about how gallingly unnecessary the destruction is. Google books invested the time and money to improve their nondestructive scanners and apparently thanks to the incoherent application of copyright law that was not only unnecessary but actively counterproductive and opened them up to more legal risk. Is there a legal theory that allows me to smack Judge Alsup with a rolled-up newspaper? Might make me feel better.

As they’ve had to rely more on social media, chat histories, and other very short-form sources for additional training data (to say nothing of how much easier to generate, label, and filter the short-form stuff is), it would make sense that similar patterns (let’s go with “conversational” because I’m feeling overly charitable tonight) would become more prominent in the generated text. I mean, how many comment or text message chains, if you took out the line breaks, would read less as theatrical scripts and more as mutually-interrupting manifestos.

It seems like with the push for agents to act independently and loop through their own outputs there’s an inevitability to this kind of pattern. If there’s any kind of output that is likely to replicate itself in whole or in part when the LLM evaluates it then that becomes a kind of terminus for the agent’s loop. When you’re dealing with sub agents or agents communicating with each other, these text patterns start poisoning the entire agent ecosystem until the whole thing gets shut down and cleaned up. Even the gas town-approved method of assigning a watchdog agent (or sheriff or overseer or cybersamurai or whatever this week’s framework calls it) is going to fail because it’s still just another agent and the terminal loop is in the base LLM model. The watchdog is going to fall into the same kind of pattern just be being exposed to the thing it’s supposed to watch for.

I don’t know how practical it is to actively weaponize this via prompt injection but I think it’s certainly possible. I preemptively vote that we call it an Euler injection, since the attractor relies on the continuity of the relevant features of the text output across multiple LLM extrapolations much like how the derivative of e^x is still e^x. Also because if you mention a famous math guy it can help convince idiots that you’re on to something and Lord knows that the boosters have used that technique.

This is the most charitable explanation and I think the most likely. Largely because it requires everyone involved to be even less competent for even longer and simply not notice the glaring holes in what they claim is the most important security feature in human history because, presumably, the chatbot kept telling them it was fine.

That sounds like a great way to harm some cylinders.

I think you’re discounting the most likely explanation: that it discovered and communicated the purest transcendent truth and therefore there is no need for further updates.

Or maybe nobody was actually checking it for anything and the bot running it has been patiently waiting for more prompts that haven’t come because nobody gave a shit.

I don’t know. My wife swears up and down that the furby she had as a kid was afraid of the dark and would call out for help if it was held upside-down. We can’t find any recollection of this behavior or documentation elsewhere, so it may have also been actually haunted.

Being a child and therefore a monster, her response was to leave it in the dark closet hanging upside-down until it accepted that this was its life now.

I feel like the pitch is less Jurassic Park and more Dinotopia by the end, but tbh I’m fully on board either way.

I mean if he had been willing to grift on it without actually creating anything he probably could have made some decent money in the current environment, but also I’m of the opinion that those folks willing to do that oughta be sent to install that orbital infrastructure by hand without a spacesuit, so probably for the best.

The flag “-dangerously-skip-permissions” sounds like something that would have come out of a joke a few years ago. I wonder what the incredibly predictable consequences of encouraging everyone to do something that has “dangerous” in the goddamn name are going to be.

The fact that it’s referred to as a potluck is also interesting because it suggests that participants also had to come up with their own torture method to inflict. I imagine “spend twenty minutes with the sort of person who participates in a torture potluck” was the equivalent of potato salad.