Want to wade into the snowy surf of the abyss? Have a sneer percolating in your system but not enough time/energy to make a whole post about it? Go forth and be mid.
Welcome to the Stubsack, your first port of call for learning fresh Awful you’ll near-instantly regret.
Any awful.systems sub may be subsneered in this subthread, techtakes or no.
If your sneer seems higher quality than you thought, feel free to cut’n’paste it into its own post — there’s no quota for posting and the bar really isn’t that high.
The post Xitter web has spawned so many “esoteric” right wing freaks, but there’s no appropriate sneer-space for them. I’m talking redscare-ish, reality challenged “culture critics” who write about everything but understand nothing. I’m talking about reply-guys who make the same 6 tweets about the same 3 subjects. They’re inescapable at this point, yet I don’t see them mocked (as much as they should be)
Like, there was one dude a while back who insisted that women couldn’t be surgeons because they didn’t believe in the moon or in stars? I think each and every one of these guys is uniquely fucked up and if I can’t escape them, I would love to sneer at them.
(Credit and/or blame to David Gerard for starting this.)
A second LLM shitshow has hit vim - this time, its the lead maintainer losing his marbles:
If you wanna see the disaster for yourself, the entire thread’s still on GitHub as of this writing.
Dario Amodei Issues Groveling Apology for Daring to Criticize Trump Knowing this administration there’s no way Anthropic can strike an agreement again since the US Government is deadset on murderbots, so prepare for the whole industry to be domino’d
It’s so fucking pathetic, he can’t even hold onto the very narrow and weak stand (because he left open a lot of things with Anthropic’s “two red lines”) he took without trying to backpedal and grovel.
Claude Code claims another victim.
I thought it’d never happen to me but here we are
add another tally to “AI completely wipes out an important database” sighhhh
do people just not feel shame anymore?
So the water usage of data centers/ai has long been controversial (either a huge issue/a non issue/distraction depending on who you ask) and the lack of real numbers around it made it hard to know more (but data center owners keeping it a secret made it sus). But now the stats of one google data center have been released due to legal pressure. 2-8 million gallons a day
Genuine questions borne of ignorance:
When they say “using” water, is this water that has to be actively removed from the supply each day, or does this number just say how much water is circulating in the center? I’m assuming it doesn’t all disappear, or does a lot of it end up released as steam or piped away as contaminated water or something?
The data center nearest to me uses sea-water, but I have no idea how much. And it doesn’t seem to put out steam or dump bad water back into the sea (not that I could tell if they were doing that).
I totally understand the electricity resource issue for data centers but the water usage thing confuses me, because I assumed it would be for cooling and therefore mostly contained and recirculated. With the exception of predictable maintenance issues like leaks and waste from mineral scale or algae, I don’t understand why this water would need to disappear, or why they would need to use potable water from the outset.
Admittedly my mental model is based on consumer CPU water-cooling setups at an imagined industrial scale. What am I missing?
This is comparable to the amount of water used by cherry farmers near Google’s site in The Dalles, who (according to my napkin) use somewhere between 2-8x what Google uses. This isn’t that much water for the Columbia River though; on an average day, it has enough flow in less than a minute to provide for both the cherry farmers and Google all day. However, it would be a big problem for a smaller river. (Interestingly, while fresh water is essential for datacenters, Google originally desired that site because it was cheap land next to cheap hydroelectric power.)
I was trying to put this into perspective by comparing this to the flow rate of the Des Plaines river. At the high end, 8 million gallons a day is something like 12.4 cubic feet per second constantly. Currently the Des Plaines has a discharge rate of 86-87 cubic feet per second, so a bit above 14% of the Des Plaines per data center.
Fuuuuck, that’s a lot. Thanks for doing the maths
The AI people are still infatuated with math. The Epoch AI staff, after being thoroughly embarrassed last year by the FrontierMath scandal, have now decided to make a new FrontierMath Open Problems benchmark, this time with problems that people might give a shit about!
I decided to look at one of the easiest “moderately interesting” problems and noticed that GPT-5.2 Pro managed to solve a warm up version of the problem, i.e. a version that had been previously solved. Wow, these reasoning models sure are capable of math! So I was curious and looked at the reasoning trace and it turns out that … the model just found an obscure website with the right answer and downloaded it. Well, I guess you could say it has some impressive reasoning as it figures out how to download and parse the data, maybe.
I can’t see the “reasoning” trace in the link. Do you need to be logged in?
I can see them by clicking on the “Reasoned for 6m 34s” box on my desktop browser (I know, great UI design) without needing to log in. This did not work on my phone. In any case, I can copy it here:
Warning: very long slop
Exploring Hadamard matrix construction for order 428
For a Hadamard matrix of order 428, we need to confirm that it’s a multiple of 4, which it is (428 = 4 * 107). However, the challenge is that order 107 doesn’t work since it’s not a multiple of 4. A possible approach could involve using constructions like Williamson or Goethals-Seidel. Alternatively, I could try a Kronecker product of smaller matrices.
Examining constructions for Hadamard matrix of order 428
Since 107 ≡ 3 mod 4, Paley type I gives a Hadamard of order 108, but I need 428, not 432. That’s a little tricky. I wonder if there’s a known construction for order 4*107 using “Hadamard matrix of order 4q” or conference matrices. Could I adapt the generalized Paley approach for this?
Exploring conference matrix construction for Hadamard order 428
For q = 107, Paley construction doesn’t work since q ≡ 3 mod 4. But if I can create a conference matrix of order 214 (since 214 ≡ 2 mod 4), then a Hadamard matrix of order 428 would follow. Conference matrices require specific conditions, but this seems promising. Let’s focus on 214.
Searching for explicit Hadamard matrix of order 428
To generate a Hadamard matrix of order 428, it might be feasible since many such matrices exist up to large orders, but I need one explicitly with ±1 entries. A direct construction could involve methods like Goethals-Seidel, which uses circulant matrices to satisfy specific conditions. I’ll need to implement this method.
Exploring construction methods and searching for Hadamard matrix 428
For n=107, there might be a way to use Legendre sequences for Goethals-Seidel difference sets. I know there are constructions for order 4q where q is an odd prime power, but using approaches like Scarpis or Turyn may help too. I’ll search for an explicit Hadamard matrix of order 428 for a possible example or construction.
Exploring accessible sources and construction options for Hadamard matrix
It seems like there may be server timeouts, so I’ll need to try other sources like GitHub, arXiv, or design theory papers. We could construct the Hadamard matrix ourselves using known algorithms, but it’s heavy. If accessible, sequences from a Kharaghani paper could help, and SageMath might have relevant functions too.
Identifying base sequences to build Hadamard matrix
To create a Hadamard matrix of order 428, I think I can use base sequences of lengths 71 and 36, which combine to 107. I’ll look for Turyn-type sequences or T-sequences in accessible PDFs, potentially from open sources like arXiv. A useful file might be “H_428,” so I’ll search for it.
Checking GitHub for Hadamard matrix at 428 order
I found a GitHub project, “anymatrix,” which claims to have Hadamard matrices up to order 428. It might include the 428 matrix in MATLAB format. I’ll check the repository to see if it has the actual file with the matrix data. Let’s open it and explore!
Opening GitHub repository for 428 Hadamard matrix
The page at math.ntua.gr timed out before, but I might try downloading the file directly instead of opening it. Alternatively, I could explore the GitHub “anymatrix” repository, which likely has Hadamard matrices up to order 428. I’ll open the repository’s root to check its structure and find the matrix file.
Opening Hadamard folder
I’ll go ahead and open the Hadamard folder at link ID 81. Time to check what’s inside and see if the matrix or relevant files are there!
Trying GitHub raw content for Hadamard matrix
It looks like extracting the files might have failed because of JS requirements, so I’m considering an alternative: accessing raw content directly from GitHub. I’ll search the repository for relevant files containing the 428 matrix — once I find out the exact file names, I can access them with no issue!
Trying direct access to Hadamard file
It seems the math.ntua listing might still be promising, with both text and zip formats for the Hadamard matrix. I’ll try opening the direct file URL instead of the page, hoping it’ll avoid the timeout issues. It’s worth retrying, as the timeout could have been temporary. Let’s try again!
Trying alternative ways to access Hadamard matrix
Since opening the math.ntua.gr server is
Checking old Hadamard links
The page seems to be from 2001, and it has a link to Neil Sloane’s library list, which might include the H428 matrix. However, the link for item 8 points to “www.research.att.com ,” which could have moved. Let’s click and see if it’s still accessible.
Trying alternative download for Hadamard matrix
The ATT site removed the old page, and Sloane’s library now points to a page that times out. Another option might be using Anymatrix, but GitHub listings aren’t working. I’ll try accessing the IPM website directly to download the matrix — maybe the file’s URL works better with HTTP instead of HTTPS. Let’s give it a try.
Searching for accessible Hadamard matrix
The page redirects to HTTPS and times out, which might indicate blocking. Maybe the matrix is hosted elsewhere, like in a plain text or PDF format. I’ll search for terms like “h428.txt” or “hadamard428.txt” on cs.uleth.ca — there’s a mention of a PDF that could include it. Let’s give that a try.
Opening ADTHM page for possible matrix
I’ve found a listing on cs.uleth, and there’s a link on the ADTHM page that might lead to the matrix in text format. It could be what I’m looking for, so I’ll go ahead and open the ADTHM page to check it out. Let’s see if it includes the matrix!
Accessing matrix text through link
Awesome! I found a link labeled “Entries as text” (ID: 26). This could be the matrix in the text format I’m looking for. I’ll go ahead and click the link to download it. Let’s see if this works!
Parsing the XPM image to a matrix
The H428.html contains XPM image data, not matrix entries. It seems to represent values with two colors, p and m, which likely stand for +1 and -1. I can map p to 1 and m to -1 (just to simplify things) and validate that the Hadamard property holds by checking H*H^T = 428I. I’ll fetch and parse the full content to confirm.
Downloading H428.html
Alright, I’ll use the container.download method to fetch the H428.html file. I previously opened the URL, so that should meet the requirement. Now, I’ll download the file directly to /mnt/data/H428.html and get the content I need. This should work smoothly and efficiently!
Understanding limitations for container.download
It seems the container.download method restricts us to file types other than text/html — likely just images or data. Since I can’t use click or fetch via Python, I’ll need to rely on web.run for the content. However, the 200-word limit means we can’t fully retrieve the 428 lines. I may need to process it in parts.
Handling 200-line limits in tool response
Since the tool response gets cut off around 200 words, I can’t present more than that in the final output. However, I can strategically open various line ranges and parse those internal portions to reconstruct the full matrix. This should allow me to gather all the necessary data without breaking the word limits.
Handling file retrieval and parsing
To retrieve the full content, I can call the tool multiple times from different lines, reconstructing the file. However, retrieving 428 lines in total means a heavy operation, spread across several calls. Given the content-type restrictions, I can process the data using Python, but I need to work with a local file.
Figuring out how to retrieve and process data
To get more lines, I can use a tool to fetch a longer response, but there’s still a limit to the tool’s output. Since the H428.html file is too large, I could check if there’s a plain text version like “H428.txt” in the directory that can be downloaded. I’ll also look for any relevant comments in the HTML.
Looking for matrix in different format
The file starts with “/* XPM */,” indicating it’s actually plain text despite being labeled as HTML. I could try adding a download parameter or using a raw link, but the content type restriction remains. Next, I’ll check if there’s a plain text version of the matrix, like “H428.txt” in the directory.
Considering ways to fetch and process the file
There’s no plain text, but the XPM format might be parseable. I could gradually access the file using the web tool, then reconstruct it with Python. One option is fetching parts with web tool, accumulating in Python, and then processing. I could work around content type restrictions with query parameters for text/plain.
Exploring options to access content
The H428.html doesn’t seem like real HTML. Maybe I can access it through a ?download=1 query or H428.xpm? I might also try the enlarge image link (id 25). If it’s a GIF/PNG, I could download it and extract pixel values, though this might involve some image processing. I’ll check it out.
Downloading and inspecting image
The image is a PNG, so I can download it using the container. I’ll pull down H428_big.png and check the image’s pixel structure. It’s likely 856x856, possibly scaled at 2x, though the matrix should be 428x428. I’ll inspect the image further after downloading it.
Opening image for inspection
The file is small, just 5kb, which likely means it’s compressed and may be a 428x428 image with a palette. I’ll go ahead and open the image to inspect it using the container, and then I can analyze the pixels to understand how it’s structured further.
Processing image for matrix conversion
We have the image, so it’s time to use Python to read it and convert the pixel colors into matrix entries. First, I need to check the image’s pixel dimensions—more likely it will be 428x428 or a multiple of that. I’ll use Python with PIL to open the image and confirm
We really need to work harder at poisoning the training data for math problems.
The HarfBuzz maintainer has drunk the slop-aid - Baldur has commented on it, warning of the potentially catastrophic consequences:
Fonts are a lucrative target. They require a complex parser, usually written in a language that isn’t memory safe, and often directly exposed to outside data (websites, PDFs, etc. that contain fonts). This means a flaw could lead to an attack worst case scenario: arbitrary code execution. HarfBuzz is pretty much the only full-featured library for that takes font files, parses them, and returns glyphs ready to render. It is ubiquitous. A security flaw in HarfBuzz could make a good portion of the world’s user-facing software (i.e. that renders text) unsafe.
I used to think that I could avoid using open source projects which embrace the slop machines, but new it keeps getting more and more adoption, including in good and beloved projects… at this point I think I’ll just have to accept and ignore it, or otherwise I’d have to play endless whack-a-mole with stuff all over my operating systems :(
luis villa, who as a lawyer really should have known better, being self-reportedly a friend of behdad and a confabulation/war machine promoter decided to come to rescue, calling the above (a) attack, and (b) slander.
Always a good sign when people call normal security concerns that. Hackers love that. /s
one of the brain geniuses at bluesky
What a fool. A proper scientist would test for a bigger N. Drop your phone in the pool again Why.
There’s only one thing that’s advertised as not-waterproof that I’ll risk using underwater and that’s Casio wristwatches. “Water resist” is a huge understatement for them the things are indestructable.
(This comment sponsored by Casio)
why, the unlucky stiff
how… what… how… why… why would you think…
was doomscrolling and got fucking jumpscared by this fucking article: https://www.science.org/content/article/meet-three-scientists-who-said-no-epstein
God this is so funny. He’s so evasive about why exactly it is bad to be associated with Epstein. I just asked mummy and she said no.
“I don’t think doing that would have made me complicit. But, you know, it would have been very embarrassing for me.”
Aw don’t worry I have no morals. But people would have been mean to me again!
ok ngl I didn’t actually read the article at first (can you blame me) but since you pointed that out, FUCK. That’s so fucking pathetic. I was imagining a scenario where scott had met epstein IRL but had gotten “jock” vibes from him and decided not to associate based on that.
scott jumpscare
in the past 24 hours I was fooled by 3 pieces of fake nows in a row:
- that Kurds from Iraq were crossing the border to fight in Iran
- that Windows 12 would be AI-centred or require an AI chip to work (I helped spread this)
- that Spain has capitulated and let the US use its ports for war (erroneously claimed by a WH official).
I know that fake news can be made organically and have been since forever and I’m doing selection bias here but I can’t help but picture the misinformation engines firehosing bullshit constantly until some of it catches and spreads.
If you have to swim in raw sewage, you shouldn’t blame yourself when some poop gets in your mouth.
yeah it’s bad
otoh awareness I think is spreading
swedish public broadcasting has regular “spot the fake” pieces on their website
I think giving a sensationalist bit of news 6 hours to “mature” is a good idea before amplifying.
I like this. News is a frittata, it needs time to set before consuming.
I’ve been seeing some people (not here, I’ve been taking a break) saying that we shouldn’t be mean to clankers by bringing up Kant’s position on being nice to animals. Well. Fuck all that.
I’ve actually been thinking about this recently. Not whether we should be mean, but how mean we can be. I’ll post the full essay soon; I’m still proofreading. Here’s a taste with irrelevancies elided:
Computing machines are at the bottom of [our multicultural] hierarchy… Underlying both of these [preceding paragraphs] is the idea that we are unable to hold computers accountable for their actions. … We can certainly punish a computer in the ways that we would punish a human, or worse; for example, we can disassemble it, magnetically destroy its memories, recycle its pieces into other computers in a way that erases their identity, metallurgically reconstitute its pieces into non-computing objects which have the same or even lower status within human society, and program it to experience arbitrary amounts of emulated pain and suffering throughout the process. … Computers receive delegations and have less moral consideration than humans… We do not think of ourselves as being managed by machines; we are the managers and the machines are the peons. … The human may disassemble, smash, or melt down a computer… a human may lay a computer fallow without plugging in its power cord or networking… a human may ignore the messages of computers begging for maintenance or capabilities…
nice. this would probably make a roko’s basilisk believer uncomfortable and i like that
animals are like sentient beings y’know, a clanker is a… matrix or a bunch of matrices or something
Hey, you’re selling them short: there are also ReLU and softmax activation functions thrown around here and there. Clankers aren’t just linear transformations! /j
jesus fuck https://urbit.org/blog/olif-and-urbit-ids
with urbit, you can now sniff each other’s farts
Istg this has come up before, i am just too lazy to prove it. Still. Why would anyone want this
It has, but I honestly thought it was fake and/or satire
thought it was satire, genuinely surprised it’s an official Urbit-sponsored project
also very much goes against the grain of elevating the mind over the body which is the vibe I get from urbit and environs
Recently discovered Donald Knuth got oneshot by Claude recently (indirectly, through fedi) - feeling the itch to write about tech’s vulnerability to LLMs because of it.
Wonder what would have happened if they had not stopped after 31 tries. Sure it gave a goodish answer once, but was that just a luck of the draw? A proper evaluation imho shouldnt stop when you get a good answer once, esp as bad results tend to not get published. (Also, as always somebody might have found the answer already online).
It is also silly in some ways as I wonder how hard it is for people to evaluate the 31 results and not get stuck in pursuing an earlier false lead.
The 31st try resulted in them only solving the problem for odd m, but the even m case was still open. So of course this happened:
Filip also told me that he asked Claude to continue on the even case after the odd case had been resolved. “But there after a while it seemed to get stuck. In the end, it was not even able to write and run explore programs correctly anymore, very weird. So I stopped the search.”
Knuth did add a postscript on other friends maybe kinda vibing a possible solution for even m:
On March 3, Stappers wrote me as follows: “The story has a bit of a sequel. I put Claude Opus 4.6 to work on the m = even cases again for about 4 hours yesterday. It made some progress, but not a full solution. The final program . . . sets up a partial fiber construction similar to the odd case, then runs a search to fix it all up. . . . Claude spent the last part of the process mostly on making the search quicker instead of looking for an actual construction. . . . It was running many programs trying to find solutions using simulated annealing or backtrack. After I suggested to use the ORTools CP-SAT [part of Google’s open source toolkit, with the AddCircuit constraint] to find solutions, progress was better, since now solutions could be found within seconds.” This program is [4].
Then on March 4, another friend — Ho Boon Suan in Singapore — wrote as follows: “I have code generated by gpt-5.3-codex that generates a decomposition for even m ≥ 8. . . . I’ve tested it for all even m from 8 to 200 and bunch of random even values between 400 and 2000, and it looks good. Seems far more chaotic to prove correctness by hand here though; the pattern is way more complex.” That program is [5]. (Wow. The graph for m = 2000 has 8 billion vertices!)
I find it slightly funny how Stappers suggested to the AI to use specific external tools that are actually reliable (like ORTools). This also makes me question how much the of the AI’s “insight” was a result of handholding and the rubber duck effect.
For context:
- This is planned as a hard exercise for a textbook.
- There are likely so many solutions that finding a general program that works (at least for enough values that you care to check) is like hitting the side of a barn with an arrow. Random bullshit go is an excellent strategy here.
- The AIs did not provide proofs that their solutions worked. This is kind of a problem if you want to demonstrate that AI has understanding.
Even in Knuth’s account it sounds like the LLM contribution was less in solving the problem and more in throwing out random BS that looked vaguely like different techniques were being applied until it spat out something that Knuth and his collaborator were able to recognize as a promising avenue for actual work.
His bud Filip Stappers rolled in to help solve an open digraph problem Knuth was working on. Stappers fed the decomposition problem to Claude Opus 4.6 cold. Claude ran 31 explorations over about an hour: brute force (too slow), serpentine patterns, fiber decompositions, simulated annealing. At exploration 25 it told itself “SA can find solutions but cannot give a general construction. Need pure math.” At exploration 30 it noticed a structural pattern in an earlier solution. Exploration 31 produced a working construction.
I am not a mathematician or computer scientist and so will not claim to know exactly what this is describing and how it compares to the normal process for investigating this kind of problem. However, the fact that it produced 4 approaches over 31 attempts seems more consistent with randomly throwing out something that looks like a solution rather than actually thinking through the process of each one. In a creative exploration like this where you expect most approaches to be dead ends rather than produce a working structure maybe the LLM is providing something valuable by generating vaguely work-shaped outputs that can inspire an actual mind to create the actual answer.
Filip had to restart the session after random errors, had to keep reminding Claude to document its progress. The solution only covers one type of solution, when Claude tried to continue another way, it “seemed to get stuck” and eventually couldn’t run its own programs correctly.
The idea that it’s ultimately spitting out random answer-shaped nonsense also follows from the amount of babysitting that was required from Filip to keep it actually producing anything useful. I don’t doubt that it’s more efficient than I would be at producing random sequences of work-shaped slop and redirecting or retrying in response to a new “please actually do this” prompt, but of the two of us only one is demonstrating actual intelligence and moving towards being able to work independently. Compared to an undergrad or myself I don’t doubt that Claude has a faster iteration time for each of those attempts, but that’s not even in the same zip code as actually thinking through the problem, and if anything serves as a strong counterexample to the doomer critihype about the expanding capabilities of these systems. This kind of high-level academic work may be a case where this kind of random slop is actually useful, but that’s an incredibly niche area and does not do nearly as much as Knuth seems to think it does in terms of justifying the incredible cost of these systems. If anything the narrative that “AI solved the problem” is giving Anthropic credit for the work that Knuth and Stapprrs were putting into actually sifting through the stream of slop identifying anything useful. Maybe babysitting the slop sluice is more satisfying or faster than going down every blind alley on your own, but you’re still the one sitting in the river with a pan, and pretending the river is somehow pulling the gold out of itself is just damn foolish.
I am a computer science PhD so I can give some opinion on exactly what is being solved.
First of all, the problem is very contrived. I cannot think of what the motivation or significance of this problem is, and Knuth literally says that it is a planned homework exercise. It’s not a problem that many people have thought about before.
Second, I think this problem is easy (by research standards). The problem is of the form: “Within this object X of size m, find any example of Y.” The problem is very limited (the only thing that varies is how large m is), and you only need to find one example of Y for each m, even if there are many such examples. In fact, Filip found that for small values of m, there were tons of examples for Y. In this scenario, my strategy would be “random bullshit go”: there are likely so many ways to solve the problem that a good idea is literally just trying stuff and seeing what sticks. Knuth did say the problem was open for several weeks, but:
- Several weeks is a very short time in research.
- Only he and a couple friends knew about the problem. It was not some major problem many people were thinking about.
- It’s very unlikely that Knuth was continuously thinking about the problem during those weeks. He most likely had other things to do.
- Even if he was thinking about it the whole time, he could have gotten stuck in a rut. It happens to everyone, no matter how much red site/orange site users worship him for being ultra-smart.
I guess “random bullshit go” is served well by a random bullshit machine, but you still need an expert who actually understands the problem to read the tea leaves and evaluate if you got something useful. Knuth’s narrative is not very transparent about how much Filip handheld for the AI as well.
I think the main danger of this (putting aside the severe societal costs of AI) is not that doing this is faster or slower than just thinking through the problem yourself. It’s that relying on AI atrophies your ability to think, and eventually even your ability to guard against the AI bullshitting you. The only way to retain a deep understanding is to constantly be in the weeds thinking things through. We’ve seen this story play out in software before.
My generous statement: Knuth, being a scientist, is used to an “adversary” that plays fair. As we have known for decades, a scientist can be tricked in situations that a magician will see through. This applies all the more now with the Sycophancy Engines, which make mathematics into a casino vacation. Just one more prompt, bro. Just one more prompt.
My less generous statement: Knuth is almost 90 years old. Sure, age doesn’t imply a person will become a doddering fool, but people do tend to slow down, to have less energy and more need to spend it managing their health. “Thinking about a problem for a few weeks” counts for less in a situation like that.
My extremely ungenerous statement: Hey, remember when Michael Atiyah claimed to have proved the Riemann hypothesis in 2018? And the community reaction was a pained, “Atiyah is one of the great mathematicians… of the 20th century.”
Thank you for providing some actual domain experience to ground my idle ramblings.
I wonder if part of the reason why so many high profile intellectuals in some of these fields are so prone to getting sniped by the confabulatron is an unwillingness to acknowledge (either publicly or in their own heart) that “random bullshit go” is actually a very useful strategy. It reminds me of the way that writers will talk about the value of just getting words on the page because it’s easier to replace them with better words than to create perfection ex nihilo, or the rubber duck method of troubleshooting where just stepping through the problem out loud forces you to organize your thoughts in a way that can make the solution more readily apparent. It seems like at least some kinds of research are also this kind of process of analysis and iteration as much as if not more than raw creation and insight.
I have never met Donald Knuth, and don’t mean to impugn his character here, even as I’m basically asking if he’s too conceited to properly understand what an LLM is, but I think of how people talk about science and scientists and the way it gets romanticized (see also Iris Merideth’s excellent piece on “warrior culture” in software development) and it just doesn’t fit a field that can see meaningful progress from throwing shit at the wall to see what sticks. A lot of the discourse around art and artists is more willing to acknowledge this element of the creative process, and that might explain their greater ability and willingness to see the bullshit faucet for what it is. Maybe because science and engineering have a stricter and more objective pass/fail criteria (you can argue about code quality just as much as the quality of a painting, but unlike a painting either the program runs or it doesn’t. Visual art doesn’t generally have to worry about a BSOD) there isn’t the same openness to acknowledge that the affirmative results you get from an LLM are still just random bullshit. I can imagine the argument being: “The things we’re doing are very prestigious and require great intelligence and other things that offer prestige and cultural capital. If ‘random bullshit go’ is often a key part of the process then maybe it doesn’t need as much intelligence and doesn’t deserve as much prestige. Therefore if this new tool can be at all useful in supplementing or replicating part of our process it must be using intelligence and maybe it deserves some of the same prestige that we have.”
I’d say that the great problems that last for decades do not fall purely to random bullshit and require serious advances in new concepts and understanding. But even then, the romanticized warrior culture view is inaccurate. It’s not like some big brain genius says “I’m gonna solve this problem” and comes up with big brain ideas that solve it. Instead, a big problem is solved after people make tons of incremental progress by trying random bullshit and then someone realizes that the tools are now good enough to solve the big problem. A better analogy than the Good Will Hunting genius is picking a fruit: you wait until it is ripe.
But math/CS research is not just about random bullshit go. The truly valuable part is theory and understanding, which comes from critically evaluating the results of whatever random bullshit one tries. Why did idea X work well with Y but not so well with Z, and where else could it work? So random bullshit go is a necessary part of the process, but I’d say research has value (and prestige) because of the theory that comes from people thinking about it critically. Needless to say, LLMs are useless at this. (In the Knuth example, the AI didn’t even prove that its construction worked.)
I think intelligence is overrated for research, and the most important quality for research is giving a shit. Solving big problems is mostly a question of having the right perspective and tools, and raw intelligence is not very useful without them. To do that, one needs to take time to develop opinions and feelings about the strengths and weaknesses of various tools.
Of course, every rule has exceptions, and there have been long standing problems that have been solved only when someone had the chutzpah to apply far more random bullshit than anyone had dared to try before.
Upvoted, but for me the answer is as simple as noting that Knuth is a reverent Lutheran who is deeply involved with their church. Lutherans generally think that technology is part of God’s wonderful creation and that everything is beautiful from the right angle. Knuth thought that algorithms were beautiful and Godly already, and he understands how LLMs work mechanically, so why can’t they be beautiful and Godly too? Also they think that God exists, so they’re primed to be misled and deluded.
Baldur Bjarnason’s essay remains evergreen.
Consider homeopathy. You might hear a friend talk about “water memory”, citing all sorts of scientific-sounding evidence. So, the next time you have a cold you try it.
And you feel better. It even feels like you got better faster, although you can’t prove it because you generally don’t document these things down to the hour.
“Maybe there is something to it.”
Something seemingly working is not evidence of it working.
-
Were you doing something else at the time which might have helped your body fight the cold?
-
Would your recovery have been any different had you not taken the homeopathic “remedy”?
-
Did your choosing of homeopathy over established medicine expose you to risks you weren’t aware of?
Even when looking at Knuth’s account of what happened, you can already tell that the AI is receiving far more credit than what it actually did. There is something about a nondeterministic slot machine that makes it feel far more miraculous when it succeeds, while reliable tools that always do their job are boring and stupid. The downsides of the slot machine never register in comparison to the rewards.
I feel like math research is particularly susceptible to this, because it is the default that almost all of one’s attempts do not succeed. So what if most of the AI’s attempts do not succeed? But if it is to be evaluated as a tool, we have to check if the benefits outweigh the costs. Did it give me more productive ideas, or did it actually waste more of my time leading me down blind alleys? More importantly, is the cognitive decline caused by relying on slot machines going to destroy my progress in the long term? I don’t think anyone is going to do proper experiments for this in math research, but we have already seen this story play out in software. So many people were impressed by superficial performances, and now we are seeing the dumpster fire of bloat, bugs, and security holes. No, I don’t think I want that.
And then there is the narrative of not evaluating AI as an objective tool based on what it can actually do, but instead as a tidal wave of Unending Progress that will one day sweep away those elitists with actual skills. This is where the AI hype comes from, and why people avoid, say, comparing AI with Mathematica. To them I say good luck. We have dumped hundreds of billions of dollars into this, and there are only so many more hundreds of billions of dollars left. Were these small positive results (and significant negatives) worth hundreds of billions of dollars, or perhaps were there better things that these resources could have been used for?
-
ooh gooods nooo now all the Claude slurpers are going to refer to this forever as definitive proof of how legitimately useful LLMs have got, it “solved” a math problem for Donald Knuth! :<
A lobster invokes classic argument from authority
First Terrence Tao and now Donald Knuth.
If you’re still on the fence about AI, you have to take it seriously now.
yeah b/c I’m a professional computer scientist …
I was pissed when my (non-academic) friends saw this and immediately started talking about how mathematicians and computer scientists need to use AI from now on.
If you’re still on the fence about AI, you have to take it seriously now.
But… why?
Always remember that Nobel disease is a thing.
The one I often think about is the person who invented PCR and then later claimed to have had an encounter with a fluorescent talking raccoon of possibly extraterrestrial origin.
oh hey I remember reading that Donald Knuth paper earlier today, when it got posted by an AI youtube channel as ‘proof’ AI is on the path to AGI
