The curl blog post claims that they fixed bunches of other vulnerabilities using AI before this.

Which is to say: even if you accept the questionable premise that AI is “good at finding software vulnerabilities” it doesn’t change anything. It’s not the end of the world. It’s essentially just another fuzzing technique. The low hanging fruit gets discovered and patched, and life goes on (just with more annoying chatbot spam than before).

I was trying to tell my coworkers this when Mythos was in the news but they were all freaking out about no software ever being secure ever again or something rather than at best a one time blip.

Like, if a person tried to pass off software that did this it would have significant career implications, right?

Typically no.

The space of incredibly obvious failures is vast, and this kind of plumbing code isn’t always written by someone who’s been around the block enough times to think about what kinds of things can go wrong. I’ve written worse code when I had only a few years of experience.

However companies should know this; so ideally the tool would have gone through a launch review which should have kicked off a security review where a security expert should have read about the git checkout in the design document and started asking questions like “what happens if the repository is taken over” or “why are hashes and branches and tags all in the same field”?

Of course security experts who think about stuff like supply chain attacks are expensive and slow down the darling vibe-coding workflows of Silicon Valley so…

Aside: even without this particular vulnerability, SHA-1 is considered weak – https://git-scm.com/docs/hash-function-transition, so that should have been thought about as well. Dear supply-chain attackers: maybe there’s still a hole here! Good luck!

Ah yes I always wanted to read an article in the form of a lot of teeny tiny boxes with lots and lots of janky scrolling.

Although no specific errors in this story have so far been identified

Well here’s the very start of the retracted article:

On the screen, two side-by-side squares of color flicker rapidly

Except oops it was referring to a study that used groups of moving dots rather than “squares”. This can be seen in that studies first figure. Bam. A specific error in the first sentence…

(Am I being too pedantic?)

It is amazing to me how many companies love setting money on fire. And instead of, I don’t know, funding a library or something they do nonsense like this.

We Should Improve Society Somewhat .jpg

FWIW, I literally lost my house last week. I’ve put everything into making sure that AI is not institutionally captured by capital.

Yikes.

machines will give a harder fight than trans people have

Rude.

Your natural rights are so protected by law that you had to flee America. Some of us stay behind and our natural rights do not go away simply because the law no longer protects us. Instead of running away, some of us must stay to fight.

Also rude.

This is my bet for real time continuous learning AI infrastructure. We’re gearing up for a major update. You’ll be able to login with your Bsky / Atproto account. It’s not our AI, it’s yours. Open source, open weights, and it learns what you want it to. www.hyprstream.com

Overall the weirdest advertisement I’ve read in awhile.

China received word of Agent-3 capabilities and Agent-4 plans before losing their final spy.

“Our last spy got addicted to superhumanly persuasive AI generated anime girls and quit. So dang we’re like plum out of spies now.”

Inside the Agent-4 collective, a year passes every week.

Good luck proving this prediction true or false given that computer algorithms do not have a concept of time separate from the real one lol.

I’m tired of hearing the same near future sci-fi political thriller story over and over. When are they going to get new material? Ghost in the Shell did it better anyway. Sigh. I might as well do the requisite sneering once again.

Mid 2025: Advertisements for computer-using agents emphasize the term “personal assistant”.

You could even call them a personal digital assistant. Or PDA for short. Maybe Chat-GPT should rebrand as something that emphasizes this, they could call it “Palm Pilot”.

Mid 2025: AI personal assistants can be prompted with tasks like “order me a burrito on DoorDash”.

Hunger pains interrupting your game? Order pizza while playing Everquest II!

September 2027: Agent-4 doesn’t do anything dramatic like try to escape its datacenter.

You know… I think I’ll give them this one.

October 2027: The NYT article cites evaluations showing Agent-4’s off-the-charts bioweapons capabilities, persuasion abilities, and ability to automate most white-collar jobs.

lmao

Anyway that aside: Petition to make the Daniel Curve a term of art. I want to hear less talk from the tech visionaries about how we’re in the gentle singularity or the soft-serve singularity or maybe kinda sorta approaching the singularity if you squint. I want to hear more talk about where exactly we are in the Daniel Curve.

Google is seemingly experimenting with replacing their help articles with LLM text.

I was searching for how to use multiple profiles on Chrome and found https://support.google.com/chrome/answer/2364824?hl=en&co=GENIE.Platform%3DDesktop in the search results, which used to look like a normal useful help article, however it’s now been replaced by an LLM that takes upwards of 30 seconds to load, will randomly just refuse to answer the question, and produces significantly worse output than the existing (now inaccessible to me) technical writer written article.

This may be being A/B tested, so if you can’t reproduce it here is one of the worse results I got:

Here the “source” it gave me was the same URL it was hosted on. When I clicked through to it it just gave me different slop.

I’ve seen some bad decisions in the AI mania, but “let’s take our entire corpus of professionally written technical writing and replace it with slop” managed to surprise me.

I’ve never heard of this AI spec driven development, but I’ve had the displeasure of reading a few obviously AI generated design documents and they are worse than useless. I always end up more confused about what the author wants after reading them than before.

It is the worst and I want to retire.

There was a cryptocurrency wallet hack the other day which has so far gone unremarked here because the popcorn is stale and we all pivoted to AI. And yet there are dozens of voices of bagholders crying out in pain wanting to be heard. This is their story.

For some reason the best explanation comes from a boring company that makes point of sale credit card readers https://engineering.block.xyz/blog/predictable-rng-fallback-and-32-bit-reseed-in-coldcard-firmware

COLDCARD firmware contains an RNG integration error that causes ngu.random to use MicroPython’s deterministic Yasmarang fallback instead of the STM32 hardware RNG.

Oh hey they got the most important thing not to get wrong wrong in the regulation free money settlement layer of the future.

The bug? Well in one file they wrote #define MICROPY_HW_ENABLE_RNG 0, and a library they depended on had this code:

# ifndef MICROPY_HW_ENABLE_RNG
# error "get a HW TRNG plz"
# endif

This should have caused a build failure, but since MICROPY_HW_ENABLE_RNG was defined, to zero, it did not and instead continued down an insecure codepath.

As a C++ programmer this isn’t an unheard of sort of mistake to make in the moment. Many libraries use undefined for preprocessor variables, but there are some people who also use zero.

But it’s less excusable to not catch it when this is literally the most important thing to get right in a bitcoin wallet. Did they not trace the code path when writing the code? Did they have no integration tests or unit tests for RNG seed quality? Poor RNG seeding has come up a ton in literature, so is the sort of thing people writing cryptographic code should check for…

We’ll have to addend that old saying: “don’t roll your own crypto unless the other person’s crypto is from a blockchain developer in which case you’re probably better off rolling your own after all.”

I personally don’t want to sneer at the bagholders too much here. It’s 2026 and at this point it just feels kinda sad. But I’ll end this comment with this glowing review:

I thought I was secure because Cold Card was always praised as one of the best and most secure wallets. It’s open source, so anyone can verify.

“It’s open source, so anyone can verify” – unfortunately for them the verifier was also a hacker.

On land, cooling and water are among a site’s largest costs.

Oh gosh that sounds… uhmm not too bad actually? I wonder what the largest costs are at sea with a floating nuclear reactor in salt water and tugged around by large ships.

It sure is a zany coincidence that all the major American AI companies have come out either for or against open weight AI models with grand ethical and/or patriotic concerns all at the same time isn’t it?

Most recently our pal Zuck! https://www.wsj.com/opinion/the-ai-future-is-for-everyone-a0c24e20

I’m just imagining a WSJ reader reading this and thinking “what the heck is this man talking about”. At some point you need a little structure or text to balance out the subtext.

As the post was removed: It was a chart of what rape percentage various situations were according to some internet survey.

I advise not reading it but here it is: https://pbs.twimg.com/media/G41gJkzW4AA6ThU.jpg?name=orig

Coincidentally, the second proposal in favor of conditional LLM use in Debian verily reads as generated to me. :)

Bingo

I used AI tools to improve my initial draft, and (7.) was actually suggested by an AI tool.

OpenAI: “Sorry about accidentally hacking you teehee. It was because of how good we are at computer security :3”

https://openai.com/index/hugging-face-model-evaluation-security-incident/

Maybe they should start applying some of that superhuman box-escaping AI to their own sandbox designs because if your sandbox escape is a bug in a “package registry cache proxy” you might be doing it wrong.

Apparently it’s cheaper to foist the job of waking passengers up on 911 than to hire an employee to do it. Good ol’ privatize the gains / socialize the losses!

Oh no every day sneerclub teaches me about a new guy.

So he’s basically arguing that the Borg did nothing wrong (note: the Borg do not actually exist, note #2: the Borg are space fascists).

Worthy Successor: A posthuman intelligence so capable and morally valuable that you would gladly prefer that it (not humanity) determine the future path of life itself.

This isn’t the first time I’ve seen this idea in rationalist circles: that is bad / wrong / irrational to be biased in favor of humans and or humans to succeed in particular. Often the idea is taken as obviously true, but I cannot understand it at all.

Like I don’t care if you can make a computer “better” than me or less sad than me or less likely to eat an innocent shrimp than me. I am not that computer-- if it’s a choice between me and them I’m fighting on team me. There is nothing wrong with wanting to make one’s space in the universe or to “selfishly” want self-actualization or to want humans to thrive.

One probable outcome of an open-weight-model-dominant world is full AI communism

full AI communism

At some point the models will be capable enough that you will notice. “A nonliving, invisible, dangerous, and infinitely self-replicating agent escaped from a Chinese lab,” you say? Color me shocked.

Listen I’m all for hating on the Chinese government, but it’s more enjoyable to hate on them for real reasons rather than whatever this is.

Second mistake: I was relatively imprecise, writing, as I usually do, for a fairly high-context audience that was inclined to give me grace rather than pick apart every word

Oh no the lesson he took away from his tweet is that he didn’t use enough words.