However, according to the threat hunters, the victim can’t recover the encrypted data, even if they paid the ransom demand, because the agent escalated “from row-level deletion to dropping entire database schemas, narrating its own targeting rationale,” without backing up any of the encrypted data.
As usual, even when these things display legitimately impressive capabilities they still fuck up in ways that completely negate the whole point of doing it in the first place.
I never said it was a good old Star Wars expanded universe novel.
Did anyone else read the old Star wars Novel Darksaber by Kevin J Anderson? The Hutts kidnap/hire the designer of the original death star to build them one of their own, and while the new Republic is gathering up the requisite heroes to do what they do to death stars we get to see the Hutts cutting corners and embezzling. The fleet arrives just as they’re ready to turn it on and instead of blowing our heroes up the subpar construction fails and it just fucking explodes.
No idea why that came to mind all of a sudden after reading this piece.
I mean at some point someone is going to try and make this argument so they can actually extract profit from their slop-inator. And it’s gonna be real funny to see what they had to say about copyright law during the training data gold rush.
This gets dangerously close to acknowledging that the rationalist method isn’t actually very useful for any area where it isn’t trivial.
Yeah. The Haitian revolution was absolutely a high point of postcolonial Caribbean history, but the resulting state wasn’t exactly able to project power and export their revolution through material support. It gave slavers a reason to double down on repression, but outside of Haiti itself it’s a propaganda win more than a change in the scales.
It’s also fascinating because I thought the OP was pretty clear that there’s a difference between decision theory and “desirable dispositions” which I interpret as covering the kind of counterfactual preferences indicated here. Actually there’s an even more fundamental issue with this as a decision theory problem which is that it misidentifies who is actually making a decision. Changing the applicant’s decision theory (while leaving their preference for thievery intact) doesn’t matter to the person actually deciding here.
Don’t get me wrong, it’s also a wildly racist example to put forward, it’s just also a bad example and where there is an argument it’s addressed in the OP.
Yeah. For being a little strange I find the shrimp welfare stuff pretty unobjectionable. Like, he vastly overstates the magnitude of good done by those stunners because he does appear to be a shut-up-and-multiply bro, but I’m comfortable with the general notion that we should be nicer to shrimp and other animals we eat, even the ones that don’t make good PETA glamour shots.
He certainly seems to enjoy the same kind of privilege of an infinite rebuy that Elon gets. It really does feel like there’s a limit of nine digits on the money counter and once you get more than a billion dollars it just overflows to infinity. There seems to be no amount of bad investments they can make or money they can lose that would actually matter or make them come back to the same economy the rest of us live in.
The fact that the 2014 presentation opens with “SoftBank=goose” just makes the 2026 metaphor even harder to parse. Is SoftBank itself the goose? Is the goose the hypothetical ASI that OpenAI is definitely going to build and monetize any day now? Am I the goose? I certainly feel undervalued sometimes, maybe I’m the goose.
At this point I’m expecting the next big startup to literally call itself Golden Calf
I vote we preserve this term for future and past bubbles as well.
Is there any chance this is just an awkward translation?
It’s also funny to note that at least 2/5 of the points are actively bad advice. Naively extrapolating from a trend line is one of the most common errors people make when trying to make a prediction, especially when you’re already prone to letting the aesthetic of data lead you astray. Trusting in a kind of “normalcy bias” or whatever you want to call the assumption that the world will continue to be pretty normal is one of the better ways to hedge against that.
Also I’ve said it before but the name “technological singularity” literally comes from the idea that at the hypothetical rate of change they’re posting all our existing models of what is possible or probable break down like the laws of physics at the center of a black hole. If you’re reasoning from a pre-singularity model then definitionally there is no expectation that it should continue to hold true. I don’t think I need to get too deep into why the whole singularitarian concept is pretty sketchy in its own right, but since it still lies at the heart of the science fiction driving these people’s predictions I think its worth acknowledging that it does suggest its own nonsense.
There was a line in the piece you linked from Wired that seemed to imply that the AAI gulag might be a holding cell for engineers who zuck didn’t want to just lay off (presumably to avoid spooking investors and tanking morale even harder) but also didn’t want to have on their previous role of actually working on Facebook/insta/whatever.
What I find most interesting in all this is that none of the coverage I’ve seen has said that crimes happened in its vicinity that it failed to detect, only that it didn’t issue any citations or whatever. Which is wild if you consider that they basically had it doing laps of a parking garage or whatever. You would think that if it was encouraging people to not rob those cars or whatever that it was doing its job, much like how ostensibly a speed camera is supposed to discourage people from speeding rather than just issue fines.
Don’t get me wrong, I’m not saying that this obelisk-looking-ass robopig was actually succeeding, I just think that the criteria being used to declare it a failure say some really bad things about the state of policing.
I mean, I think he is entirely too credulous of people who claim to be doing things better with AI and discounts a lot of the possible costs of AI systems that malfunction silently and produce plausible bullshit. But I think that those elements complicate his point more than they fully contradict it. Like, consider his last example. Lawyers looking for possible cases for something like the innocence project have to start somewhere, and I can fully believe that the kind of statistical analysis marketing itself as AI is going to be able to pick out viable leads better than doing it randomly or alphabetically or whatever, and that might save the lawyers time and let them help more people than they otherwise would have. But by replacing a naive algorithm with an opaque one you’re essentially baking in any underlying biases in the current system. The people who aren’t going to get seen now are still probably not going to get seen unless they’re right on the margins somehow, and that “somehow” is almost certainly going to be racism, sexism, etc. But by moving that bias from the immediate decision and placing it in the AI model it becomes that much harder to unpack, identify, and address. Like, I fully agree that much if not most of the harm being done by AI right now is more tied to the business and economic structures that it’s embedded in rather than the technology itself. There are very good reasons why so many crypto/metaverse/nft grifters moved straight to AI, and when they try and move on to quantum or web67 or whatever else comes next they will keep right on hurting the world in the same ways unless something about those structures changes. But that doesnt necessary mean we shouldn’t also focus on the harms and limitations that are inherent to the way these things function rather than how they’re used.
Wasn’t this the boat that everyone was like “yeah it’s gonna fall over in a stiff wind which is pretty bad for a sailboat” only to be overruled because money? And now the captain and crew who were actually willing to sign on are being investigated.
I would be slightly more charitable than that. I don’t think it’s a matter of failing to read or understand the surface of the material, I think it’s a failure to seriously engage with it. He’s the Tolkien equivalent of a 2005-era Reddit atheist who is can rattle off all the worst parts of Leviticus but has no empathy for why people actually connect with their religion. (Unrelated: I am describing myself here.)
Like, the beating heart of LotR is a kind of 20th-century romanticism, a celebration and eulogy for a world in transition to something wholly new. Combined with the fact that Tolkien largely codified the whole set of fantasy races I can sort of understand where people with fascist tendencies connect with it. You could interpret some of that in a very “RETVRN”-coded way where Gondor and the Shire are both the glorious homeland that needs to be protected and/or reclaimed, especially if you dig into certain elements of the lore around Numenor. Alternatively in that kind of reading it’s entirely possible to construct an argument about how Sauron and Saruman are the bringers of progress and industry. Pretty sure I remember a cracked article back in the day that made this argument for laughs, but you can assemble it from the parts in the text. Either way you can make something of it that I imagine Thiel and his set are pretty damn comfortable with.
Of course that would require ignoring the far more central theme of the story, which is how power is inherently destructive. The ring itself is the most obvious manifestation, but I think the other element that gets lost is the way that Sauron is ultimately defeated not by Gandalf out-wizarding him or by Aragor out-kinging him or Gimli, Legolas, or anyone else out-fighting him. It isn’t even Frodo and Sam persisting in their quest and resisting the call of the Ring, for in the final pivotal moments even their strength and righteousness failed them. Instead, it comes down to Gollum’s greed and devotion to his Precious. His weakness, in other words. Tolkien coined the term Eucatastrophe to describe this kind of moment, and it’s notable for being the culmination of the story’s smaller moments. Time and again the heroes of LotR choose to do what is right and act with mercy, honor, courage, kindness, or respect not because it’s the right strategic choice - indeed the most logical consequence would have been failure in a thousand predictable ways - but because of their connections to each other and their faith that rightness and righteousness will somehow align.
This is of course antithetical to Thiel and the whole project which worships power rather than fearing it. The whole theme of the Hobbit essentially is a rejection of the attempt to be “agentic” and exercise power over the whole world rather than enjoying your own life. Bilbo’s heroism explicitly comes not because he’s going to save the world but because he wants to help his compatriots recover the same humble comfort of home that he enjoyed his whole life.
I keep bouncing back to this one and I think that the core objection is that the method of discourse that they’re trying to advance here is fundamentally incapable of handling people actually disagreeing. Like, the whole concept of “identifying a crux” basically requires that there’s a central point of agreement somewhere. In my experience a lot of these issues are better understood as tradeoffs and compromises. It is simultaneously true that some people will do terrible things left to their own devices and locking them up seems to be one of the only things society can collectively agree to do about it and also that locking people up is fundamentally cruel and it’s bad that we do it. The challenge isn’t in identifying the central point of agreement between those two but in managing their fundamental incompatibility.