~~In the dead of night, knocking on the door~~

Screaming at the void towards Chuunibyou (wiki) Eliezer: YOU ARE NOT A NOVEL CHARACTER, THINKING OF WHAT BENEFITS THE NOVELIST vs THE CHARACTER HAS NO BEARING ON REAL LIFE.

Sorry for yelling.

Minor notes:

But <Employee> thinks I should say it, so I will say it. […] <Employee> asked me to speak them anyways, so I will.

It’s quite petty of Yud to be so passive-aggressive towards his employee insisted he at least try to discuss coping. Name dropping him not once but twice (although that is also likely to just be poor editing)

“How are you coping with the end of the world?” […Blah…Blah…Spiel about going mad tropes…]

Yud, when journalists ask you “How are you coping?”, they don’t expect you to be “going mad facing apocalypse”, that is YOUR poor imagination as a writer/empathetic person. They expect you to be answering how you are managing your emotions and your stress, or bar that give a message of hope or of some desperation, they are trying to engage with you as real human being, not as a novel character.

Alternatively it’s also a question to gauge how full of shit you may be. (By gauging how emotionally invested you are)

The trope of somebody going insane as the world ends, does not appeal to me as an author, including in my role as the author of my own life. It seems obvious, cliche, predictable, and contrary to the ideals of writing intelligent characters. Nothing about it seems fresh or interesting. It doesn’t tempt me to write, and it doesn’t tempt me to be.

Emotional turmoil and how characters cope, or fail to cope makes excellent literature! That all you can think of is “going mad”, reflects only your poor imagination as both a writer and a reader.

I predict, because to them I am the subject of the story and it has not occurred to them that there’s a whole planet out there too to be the story-subject.

This is only true if they actually accept the premise of what you are trying to sell them.

[…] I was rolling my eyes about how they’d now found a new way of being the story’s subject.

That is deeply Ironic, coming from someone who makes choice based on him being the main character of a novel.

Besides being a thing I can just decide, my decision to stay sane is also something that I implement by not writing an expectation of future insanity into my internal script / pseudo-predictive sort-of-world-model that instead connects to motor output.

If you are truly doing this, I would say that means you are expecting insanity wayyyyy to much. (also psychobabble)

[…Too painful to actually quote psychobabble about getting out of bed in the morning…]

In which Yud goes in depth, and self-aggrandizing nonsensical detail about a very mundane trick about getting out of bed in the morning.

A fairly good and nuanced guide. No magic silver-bullet shibboleths for us.

I particularly like this section:

Consequently, the LLM tends to omit specific, unusual, nuanced facts (which are statistically rare) and replace them with more generic, positive descriptions (which are statistically common). Thus the highly specific “inventor of the first train-coupling device” might become “a revolutionary titan of industry.” It is like shouting louder and louder that a portrait shows a uniquely important person, while the portrait itself is fading from a sharp photograph into a blurry, generic sketch. The subject becomes simultaneously less specific and more exaggerated.

I think it’s an excellent summary, and connects with the “Barnum-effect” of LLMs, making them appear smarter than they are. And that it’s not the presence of certain words, but the absence of certain others (and well content) that is a good indicator of LLM extruded garbage.

I guess my P(Doom|Bathroom) should have been higher.

Pessimistically I think this scourge will be with us for as long as there are people willing to put code “that-mostly-works” in production. It won’t be making decisions, but we’ll get a new faucet of poor code sludge to enjoy and repair.

In French, ChatGPT sounds like « Chatte, j’ai pété » meaning “Pussy, I farted”.

The power, of words:
Is all but naught, if not heard.
And a bot, cannot.

Of course! It’s to know less and less, until truly, the only thing they know is that they know nothing.

It’s clearly meant to mean /HalleluJah

To be fair though it’s not just their brains turning to mush, google has genuinely been getting worse too.

Ahh the missing period, an even worse tone indicator compared to /hj (youtube).

Assuming they have any amount of good faith, I would make the illustration that using AI is like dunning-kruger effect on steroids. It’s precisely especially dangerous when you think know enough, but don’t know enough to know that you don’t.

TIHI

I reiterate the hope that AI slop, will eventually push us towards better sourcing of resources/articles as a society going forwards, but yikes in the meantime.

On this topic I’ve been seeing more 503 lately, are the servers running into issue, or am i getting caught in anti-scraper cross-fire?

Some changes to adventofcode this year, will only have 12-days of puzzles, and no longer have global leaderboard according to the faq:

Why did the number of days per event change?

It takes a ton of my free time every year to run Advent of Code, and building the puzzles accounts for the majority of that time. After keeping a consistent schedule for ten years(!), I needed a change. The puzzles still start on December 1st so that the day numbers make sense (Day 1 = Dec 1), and puzzles come out every day (ending mid-December).

Scaling it a bit down rather than completely burning out is nice i think.

What happened to the global leaderboard?

The global leaderboard was one of the largest sources of stress for me, for the infrastructure, and for many users. People took things too seriously, going way outside the spirit of the contest; some people even resorted to things like DDoS attacks. Many people incorrectly concluded that they were somehow worse programmers because their own times didn’t compare. What started as a fun feature in 2015 became an ever-growing problem, and so, after ten years of Advent of Code, I removed the global leaderboard. (However, I’ve made it so you can share a read-only view of your private leaderboard. Please don’t use this feature or data to create a “new” global leaderboard.)

While trying to get a fast time on a private leaderboard, may I use AI / watch streamers / check the solution threads / ask a friend for help / etc?

If you are a member of any private leaderboards, you should ask the people that run them what their expectations are of their members. If you don’t agree with those expectations, you should find a new private leaderboard or start your own! Private leaderboards might have rules like maximum runtime, allowed programming language, what time you can first open the puzzle, what tools you can use, or whether you have to wear a silly hat while working.

Probably the most positive change here, it’s a bit of shame we can’t have nice things, a no real way to police stuff like people using AI for leaderboard times. Still keeping the private one, for smaller groups of people, that can set expectations is unfortunately the only pragmatic thing to do.

Should I use AI to solve Advent of Code puzzles?

No. If you send a friend to the gym on your behalf, would you expect to get stronger? Advent of Code puzzles are designed to be interesting for humans to solve - no consideration is made for whether AI can or cannot solve a puzzle. If you want practice prompting an AI, there are almost certainly better exercises elsewhere designed with that in mind.

It’s nice to know the creator (Eric Wastl) has a good head on his shoulders.

It’s also incredibly unimaginative to try and frankenstein the very same “remake” concept as one year ago.

Some juicy extracts:

Soon enough then the appointed day came to pass, that Mr. Assi began playing some of the town’s players, defeating them all without exception. Mr. Assi did sometimes let some of the youngest children take a piece or two, of his, and get very excited about that, but he did not go so far as to let them win. It wasn’t even so much that Mr. Assi had his pride, although he did, but that he also had his honesty; Mr. Assi would have felt bad about deceiving anyone in that way, even a child, almost as if children were people.

Yud: “Woe is me, a child who was lied to!”

Tessa sighed performatively. “It really is a classic midwit trap, Mr. Humman, to be smart enough to spout out words about possible complications, until you’ve counterargued any truth you don’t want to hear. But not smart enough to know how to think through those complications, and see how the unpleasant truth is true anyways, after all the realistic details are taken into account.” […] “Why, of course it’s the same,” said Mr. Humman. “You’d know that for yourself, if you were a top-tier chess-player. The thing you’re not realizing, young lady, is that no matter how many fancy words you use, they won’t be as complicated as real reality, which is infinitely complicated. And therefore, all these things you are saying, which are less than infinitely complicated, must be wrong.”

Your flaw dear Yud isn’t that your thoughts cannot out-compete the complexity of reality, it’s that it’s a new complexity untethered from the original. Retorts to you wild sci-fi speculations are just minor complications brought by midwits, you very often get the science critically wrong, but expect to still be taken seriously! (One might say you share a lot of Humman misquoting and misapplying “econ 101”. )

“Look, Mr. Humman. You may not be the best chess-player in the world, but you are above average. [… Blah blah IQ blah blah …] You ought to be smart enough to understand this idea.”

Funilly enough the very best chess players like Nakamura or Carlsen will readily call themselves dumbasses outside of chess.

“Well, by coincidence, that is sort of the topic of the book I’m reading now,” said Tessa. “It’s about Artificial Intelligence – artificial super-intelligence, rather. The authors say that if anyone on Earth builds anything like that, everyone everywhere will die. All at the same time, they obviously mean. And that book is a few years old, now! I’m a little worried about all the things the news is saying, about AI and AI companies, and I think everyone else should be a little worried too.”

Of course this a meandering plug to his book!

“The authors don’t mean it as a joke, and I don’t think everyone dying is actually funny,” said the woman, allowing just enough emotion into her voice to make it clear that the early death of her and her family and everyone she knew was not a socially acceptable thing to find funny. “Why is it obviously wrong?”

They aren’t laughing at everyone dying, they’re laughing at you. I would be more charitable with you if the religion you cultivate was not so dangerous, most of your anguish is self-inflicted.

“So there’s no sense in which you’re smarter than a squirrel?” she said. “Because by default, any vaguely plausible sequence of words that sounds it can prove that machine superintelligence can’t possibly be smarter than a human, will prove too much, and will also argue that a human can’t be smarter than a squirrel.”

Importantly you often portray ASI as being able to manipulate humans into doing any number of random shit, and you have an unhealthy association of intelligence with manipulation. I’m quite certain I couldn’t get at squirrel to do anything I wanted.

"You’re not worried about how an ASI […] beyond what humans have in the way of vision and hearing and spatial visualization of 3D rotating shapes.

Is that… an incel shape-rotator reference?

Wrong link. this points to the NeurIPS post for this week.

My dad was a bit freaked out by a video version (We’re not ready for super-intelligence)of the “AI 2027” paper, particularly finding two end scenarios a bit spooky: colossus-style cooperating AIs taking over the world, and the oligarch concentration of power one, which i think definitely echoed sci-fi he watched/read as a teen.

In case anyone else finds it useful these are the “Comments as I watch it”, that I compiled for him

Before watching Video Notes:

• AI Only channel with only 3 videos

• Produced By “80000hours”, which is an EA branch (trying to peddle to you the best way to organize 40years * 50 weeks * 40 hours [I love that they assume only 2 weeks of holidays]); which is definitely cult adjacent: https://80000hours.org/about/#what-do-we-do. Mostly appears to be attempting to steer young people to what they believe are “High impact” jobs.

Video Notes:

• The backing paper is a bit of a joke, one “AI 2027”, for reference one of the main authors is very much a “cult member”, Scott Alexander Siskind, author of “Slate Star Codex” and “Astral Codex Ten”.

• Other authors include [AI Futures Project] :

  • Daniel Kokotajlo (podcast co-host of siskind, ex open-ai employee, LessWrong/EA regular)
  • Thomas Larsen (ex MIRI [Machine Intelligence Research Institute = really really culty], LessWrong/EA regular)
  • Eli Lifland (LessWrong/EA regular)
  • Romeo Dean (Astra Fellowship recipient = money for AI Safety research, definitely EA sphere)

• A lot of fluff trying to hype up the credentials of the authors.

• AGI does not have a bounded definition.

• They are playing up the China angle to try and drum up jingoistic support.

• Exaggerating Chat GPT-3 success, by merely citing “users”, without mentioning actual revenue, or actual quality.

• Quote:

  How do these things interact, well we don’t know but thinking through in detail how it might go is the way to start grappling with that.

  -> I think this epitomises the biggest flaw of their movement, they believe that from “first-principles” it’s possible to think hard enough (without needing to confront it to reality) and you can divine the future.

  -> You can look up “Prediction Markets”, which is another of their ontological sins.

• I will note that the prediction of “Agents” was not a hard one, since this is what all this circle wants to achieve, and as the video itself points out it’s fantastically incompetent/unreliable.

• Note: This video was made before the release of GPT-5. We don’t know precisely how much more compute altogether GPT-5 truly required, but it’s very incremental changes compared to GPT-4. I think this philosophy of “More training” is why OpenAI is currently trying (half-succeeding half failing) to raise Trillions of dollars to build out data-centers, my prediction is that the AI bubble bursts before these data centers come to fruition.

• Note: The video assumes keeping models secret, but in reality OpenAI would have a very vested interest in displaying capability, even if not making a model available to the public. Also even on consumer models, OpenAI currently loses a bunch of money for every query.

• Note: The video assumes “Singularitarianism”, of ever acceleration in quality of code, and that’s why they keep secret models. I think this hits a compute/energy wall in real life, even if you assume that LLMs are actually useful for making “quality” code. These ideas are not new, and these people would raise alarms about it with or without current LLM tech.

• Specific threats of “Bio-weapon”, which a priori can not really be achieved without experimentation, and while “automated” labs half exis, they still require a lot of human involvement/resources. Technically grad students could also make deadly bioweapons, but no one is being alarmist about them.

• Note: “Agent 2” Continuous Online learning is gobbledygook, that isn’t how ML, even today works. At some point there are very diminishing returns, and it’s a complete waste of time/energy to continue training a specific model, a qualitative difference would be achieved with a different model. I suspect this sneakily displays “Singularitarianism” dogma.

• Quote:

  Hack into other servers Install a copy of itself Evade detection

  -> This is just science-fiction, in the real world these models require specialized hardware to be run at any effective speed, this would be extremely unlikely to evade detection. Also this treats the model as a single entity with single goals, when in reality any time it’s “run” is effectively a new instance.

• Note: This subculture loves the concept of “science in secrecy”, which features a lot in the writings of Elizer Yudkowsky. Which is cultish both in keeping their own deeds “in a veil of secrecy”, and helpful here when making a prophecy/conspiracy theory, by making the claim hard to disprove specifically (it’s happening in secret!)

• Note: Even today Chain-of-thought is not that reliable at explaining why a bot gives a particular answer. It’s more analog to guiding “search”, rather than true thought as in humans anyway. Them using “Alien-Language” would not be that different.

• Agent 3, magically fast-and-cheap, assuming there are now minimum energy requirements. Then you can magically run 200,000 copies of. magically equivalent to 50,000 humans sped up by 30x. (The magic is “explained” in the paper by big assumptions, and just equating essentially how fast you can talk with the quality of talking, which given the length of their typical blog posts is actually quite funny)

• Note: “Alignment” was the core mission of MIRI/Eliezer Yudkowsky

• Note: Equating Power and Intelligence a lot (not in this video, but in general being suspiciously racist/eugenicist about it), ignoring the material constraints of actual power [echo: Again the epitomical sin of “If you just think hard enough”]

• Note: Also assuming that trillions of dollars of growth can actually happen, simultaneously with millions losing their jobs.

• I am betting that the “There is another” part of the video is probably deliberately echoing Colossus.

• The video casually assumes that the only limits to practical fusion and nanotech just intelligence (instead of potential dead-ends, actually the nanotech part is a particular fancy of theirs, you can lookup “diamondoid bacteria” on LessWrong if you want a laugh)

• The two outcomes at the end of the video are literally robo-heaven and robo-hell, and if you just follow our teachings (in this case slow-downs on AI) you can get to robo-heaven. You will notice they don’t imagine/advocate for a future with no massive AI integration into society, they want their robo-heaven.

• Quote:

  None of the experts are disagreeing about a wild future.

  -> I would say specifically some of them are suggesting that AGI soon is implausible quite strongly. I think many would agree that right now the future looks dire with or without super-AI, or even regular AI.

Takeaway section:

Yeah this really is a cult recruitment video essentially.

Reading up a bit more on “superdeterminism” I guess it explain a bit more why she made that video attempting to debunk free will Compatibilism as a cooky idea cooked up by new cooky philosophers (Not realising it’s about as ancient as western philosophy itself).

For the “esthetics” of presenting superdeterminism as a “pure-common-sense” the no free will just sells it better.

EDIT: From memory maybe it was about “Hard Compatibilism” (free will requires determinism) which might not be explicitly so old, though I would say a natural consequence of most Compatibilist positions.