Not sure I agree with this. The math guy at anthropic's prompts are essentially:
"suppose you’ve gotta resolve the $CONJECTURE, like absolutely have to, everything depends on it. think really hard, and try to come up with a bunch of ideas to try. but remember to trust yourself and not necessarily in conventional wisdom!!"
https://claude.ai/share/25740bd5-aa97-4bd7-bf58-c4df3793fda7
https://xcancel.com/__alpoge__/status/2083855298239078748
Tao's chat was for him to gain intuition, not to solve the problem from the outset.
What's funny is that every other person gets a different conclusion about who these models reward/empower. I've seen people say that the generalist stands to gain the most and others say that it's the experts. Like all of life, maybe the "winner" is the person who just does stuff.
It depends on the levels. People with differing fitness levels and ages run at very different paces. Now, do cars make them more equal or less? On the bottom end, the tide lifts all boats. Most healthy people can learn to drive and will drive "fine", they get from A to B. Out there in the city streets the car flattens the differences, everyone roughly takes the same time to get from A to B in a car.
But at the top of top, the gap probably widens. A professional F1 driver will drive laps around some random guy. It amplifies reflexes etc, because at that speed little differences in timing make a big difference.
Now, AI coding isn't exactly analogous, but I think it also has these two regimes. It flattens things for simple tasks. If your task is to shovel data, do some trivial compiler wrangling staring at badly designed error messages, looking through GitHub issues hunting for the comment with many tadaa emojis to fix an issue etc, those things can now be done by anyone. Just as grandpa can also drive to the grocery store. But if you're pushing at things on a higher level, now only your above-AI ability matters. If all the things that AI can do well are subtracted out, how much other expertise do you have left? This will be proportionally a bigger and bigger difference between different people.
So you're saying that it depends whether you're measuring "success at task X" (where in average everyone improves) vs comparative success (where people with knowledge can do far and away better at specific tasks).
Exactly. The post read to me as another variation of the denial that people with expertise are reaching for right now. My sense is that we as programmers went through it over a year ago already (perhaps not all of us, but at least anyone paying attention), and so it's easy to overlook that it's still new to people who do other forms of "knowledge work," i.e. people whose identity is bound up with their expertise.
My working theory at the moment is that for programmers it was relatively "clean" and took the form of an inside-out transformation of the work, where AIs directly produced the central work product more or less adequately and relatively early on, but for other forms of work it will appear as some mixture of inside-out (in which case it will appear similarly first as a tool, then as something more than mere tool) and outside-in (the things surrounding their work and the supports their work processes rely on will be progressively automated). This is going to give rise to all sorts of pathologies in the white collar world, we'll get all kinds of variations on denial/negotiation, and so on, until it fully transforms the division of labor.
One interesting point of reference here: Yuval Harari gave a talk recently about the radical changes that will take place relatively quickly, in which he noted the AIs are not quite as good at writing as he is yet, although he expects they will be relatively soon. He then gave the timeline for what he considered "soon": 10 years! So we find the denial ("I still have time, they're not as good as me yet, maybe in 10 years...") even among the most vocal "prophets," among those supposedly most wised-up to what's going on and where the capability frontier lies.
He's saying: "Expertise improves AI-assisted work, experts can steer better and get more out of models."
The obvious conclusion for anyone is "therefore experts will remain the indispensable and specially rewarded center of the production process."
That this is appearing exactly now, and in this form, strikes me as extremely suspicious. I don't doubt the author's sincerity on the surface. What I suspect is that anxiety over the possibility that the (unstated) conclusion might be false (!) motivates the argument in the first place.
I'm asking the question, "Why is this argument appearing now?" At least one reason seems to me to be, "because we're afraid of what the world could look like if it's not true."
However, I personally agree with the author and I don't think his argument is necessarily motivated out of an anxious fear. On the contrary I think it may be motivated out of a sense of extreme exhilaration and empowerment.
Because experts (like myself as a programmer for 15+ years) who are using AI in many fields are suddenly empowered and much more useful than we were before AI. My employability and value has gone up and not down, precisely because of being able to apply my expertise with AI, which people without expertise simply cannot do. I am a professional programmer and also owner of my own startup.
Let me give you a concrete example that I am dealing with at my startup. I'm a small business owner. Before AI if i wanted to produce production quality video for marketing it would taken such a huge budget and such a large team of people (or an expensive agency) that I wouldn't even have considered it due to the enormous cost. I'm talking about Apple quality video production which takes millions of dollars to produce.
Not anymore. A single competent person with AI can replace an entire marketing video production department or agency. But expertise is key here: knowledge of film terminology to be able to describe the effect you want, and ability to use video editing tools effectively, as well aesthetic taste. I as a programmer with no filmmaking experience don't even know how to write the prompt which makes the video that i want because I don't even have the terminology. But a person with that expertise has suddenly become more employable and more valuable to my business because I as a small business now have the capability to create Apple quality marketing videos.
So AI actually created a new job for an expert that would have otherwise not existed because it was outside the budget of small businesses. Previously somebody like that would have been employable to only a few large production studios but now they become employable by almost any small business.
> A single competent person with AI can replace an entire marketing video production department or agency.
This is a good example of what I was pointing to with "fully transforms the division of labor."
I'm personally in a position similar to yours, but as I watch the different moves the labs make, I see the edge we've been handed (for now) also constantly under attack from different angles. This is why it seems to me that so many who can make the most of things for the moment also feel the clock is ticking.
Have you considered the possibility that you're simply not as good as the experts, and that your experience of LLMs being capable of performing your work up to your standards doesn't imply that experts are necessarily in denial?
Being fairly at expert level in a "solved" domain (for some definition of solved) but also having near-expert level proficiency in a non-technical as-yet "unsolved" domain and watching the process repeat there (and watching how people react as inroads are made progressively deeper) is basically my own standpoint. But I am intentionally using "solved" and "unsolved" very loosely here: you can get stuck in the thicket of arguments about verifiable domains, what it means for a domain to be solved, whether a set of evals can tell us something has been solved or not, and so on. One way to avoid that (as my original comment pointed to) and focus on what matters for us is to look instead at the effects being produced in the work process and on the division of labor as a whole.
Did you maybe consider that the 10 year timeline is not denial, but actually well educated reasoning based on Yuval’s experience and understanding of the problem space? You shouldn’t dismiss people’s thoughts as denial just because they don’t match your perspective.
I would guess if asked Harari would actually revise that lower. The point was: even those arguing most strongly that AI is an autonomous historical force can slide back into the mere abstract recognition of it. In context, Harari was saying "I'm speaking to you as a writer now, but my own standpoint will eventually be undermined." I'm saying "yes, and your 10-year timeline suggests you along with all of us are not taking your own thesis seriously enough."
This works better for math because math is self-verifiable. Once you have a proof it needs no outside evidence.
Expertise is needed to evaluate model outputs where it can't verify itself, or at the very least one's expertise can help steer the model in the right direction.
However this is irrelevant if models themselves are better at evaluating/leveraging expertise/information.
Corollary to this is an important part of LLM usage is what I call pinning it to reality. That is, designing verification steps that interact with the real world in some way not easy to hallucinate or work around. This means things like having code that interacts with the physical world, round trip tests, arriving at the same result using different paths, interoperability / replication with external libraries / competing products, performance improvement projects that start with robust performance test suites, and similar sorts of things that reduce to "how do I provide evidence that's difficult to fool myself about".
This includes things like "before you start fixing this bug, write two tests that fail proving it exists".
Expertise is good, but a wise expert will set up methods for the machine to prove to itself that a desired result is achieved removing the expert from the tight development loop.
But the guy who writes the “just do it” prompt can neither formulate the conjecture in the first place, nor come up with any follow-up questions to build on the result.
Yeah, the people who say no expertise is needed for these things confuse me somewhat. This is indeed the case if you want to be a meat wrapper around an LLM, understanding neither your inputs nor your outputs. But at that point, what is the point of you versus going to the LLM myself? Expertise is necessary because it adds understanding and structure to the blob of text produced by an LLM. Progress can only be built on such understanding.
I am tempted to say (uncharitably) that the 'No knowledge needed! Just add LLMs!' byline is wishful thinking by non-experts who do not want to confront the reality that they will ultimately need to learn things.
> But at that point, what is the point of you versus going to the LLM myself?
reply
You're missing one word, and that word explains why everyone running the companies is so excited. The word is 'paying'. "But at that point, what is the point paying of you versus going to the LLM myself?"
And, yes, I think that LLMs make it a lot easier to hire a minimally trained stooge and get them productive. It's worked for me, and I appreciate being able to pretend to be productive and walk away from the job a bit early every day. I don't think software engineering is likely to be a high status, high income job for very long.
It's actually refreshing when a model is sure about something because it actually tested it and has the receipts. Opus 5 seems really good about testing its own knowledge with experiments. Scientific method ftw.
There was one math proof that was AI generated going around Twitter and the chat transcript to generate it was basically the human prompting "keep going" until it solved it.
Tao's chat was fascinating because the questions he was asking belied expert knowledge of the subject that only a handful of people could have asked.
Yup I linked that in my other comment but copy-pasted here for everyone's convenience:
The counterexample of the Dinitz-Garg-Goemans conjecture was basically just "keep going" and finally "enough of partial results. now finish with a complete unconditional counterexample"
Yes, we're in the low hanging fruit stage where folks can just say "here's the problem" and "keep going" until a result is found and it will work sometimes.
The low hanging fruit will run short. Ultimately mathematics is a field of subjective selections of problems and proofs as beautiful and interesting. Machines absolutely will struggle with what to study, what theorems are desirable, and when do be done with a proof.
> Machines absolutely will struggle with what to study, what theorems are desirable, and when do be done with a proof
And why do you think this would be the case? I'm not talking about today but in 1-2 years. For reference o1 was released less than 2 years ago, and we've had reasonable coding agents for 9 months or so.
Mathematics is ultimately an aesthetic pursuit. Outside of a well defined goal ML models don't have any sense of taste and regardless of the scaling that's been enabled in the last year or so of capability if they haven't memorized the process of doing something they have the same limitations of inability to make choices about unknowns not trained into them.
Real synthetic intelligence seems to me to be still very far away and not a matter of making models bigger or more efficient.
LLMs are a collection of biases. Humans are also a collection of biases. So we project our biases as input through the biases of an LLM and get an output. Hence why I think getting optimal output requires being an optimal person. And in that sentence there are many points of expression.
Finally, we train our LLMs on who we are. Another reinforcement of biases.
(Sorry, I'm in a crappy mood, but what on Earth are we supposed to take away from this? Everyone who disagrees with you is secretly an idiot, or worse, they're smart enough to know they're idiots but too proud to admit it?)
On a more helpful note, I think your "confusion" if honest can probably be resolved by realizing that "skeptics" are not a monolith.
I think it's the same thing as it was with googling. "Oh you're such a wizard problem solver, how did you fix my printer??" "I just typed the most important words about the problem as you told it to me, the printer's model number and 'solution' and a forum link was on the first page, searched the page for 'solved' and followed the 3 simple steps of which buttons to press."
Then you saw how other people used Google, by filling the search bar with utterly irrelevant words, missing the one key word that's most important to what they are trying to do, then not be able to evaluate the returned search results and triage for which is most "solution-shaped", and they get drawn into wrong search hits, reading a clearly irrelevant page instead of quickly backing out to the search results page to try another page etc.
Or see how people couldn't formulate questions on StackOverflow, other than dumping a huge code chunk and saying "it doesn't work".
Now, AI makes these easier. You can now really just type natural language into the textbox, not just key words, you don't have to know about quote marks and plus signs etc. You can paste the code and say it doesn't work, and the AI just might actually spot a bug.
But having general problem solving common sense will still give you very good dividends.
Or even that people are using them for the same things, but there is a chaotic element to how it works, and people are getting different outcomes due to small differences that we humans don't think ought to matter.
It's because in many cases it exactly sounds like what happened. They asked something one shot, then seeing that the AI didn't quite solve the same task that the prompter had in mind, he just closes it and decides that AI is stupid and can't even do the task and it's all hype. I say, "what you just told me right now, did you even try typing that into the chat with the AI?", and they just stare at me. You have to iterate! You have to provide context, you have to provide access to the tools you also use to do your job.
The other day someone complained here on HN that AI failed to optimize his code speed. Turns out he just pasted in the code, didn't use an agentic harness with end-to-end benchmarking ability for the model to ground its changes in and to hill-climb on. But even as a human you need to test your hypotheses and measure things, and sometimes something you thought would help actually makes it slower.
It happens over and over, but it's no skin off my nose. If they don't want to learn to use it, it's on them.
What's funny is that every other person gets a different conclusion about who these models reward/empower. I've seen people say that the generalist stands to gain the most and others say that it's the experts. Like all of life, maybe the "winner" is the person who just does stuff.