2026-04-30 Thu

• Person/Jeff Dickey is leaving Figma to go full time on Open Source - Going Full Time on Open Source — @jdx - Solo/preneur?

• Person/Andrej Karpathy: Andrej Karpathy: From Vibe Coding to Agentic Engineering - YouTube

  • Lots of interesting gems in here. I just love Karpathy's brain.

  • 1.0 -> 2.0 -> 3.0 Software

    • Software 1.0 = hand-written code
    • Software 2.0 = train AlexNet to recognize cats and dogs in images; it's basically hand-written data curation / labels turned into software
    • Software 3.0 = a shift where programming becomes prompting: instead of writing explicit instructions, you place text into a model’s context window and let it act as an interpreter, using its own embedded intelligence to carry out tasks, adapt to your environment, and debug in the loop. In this paradigm, the “program” is no longer a precise script but a piece of context given to an agent, which can reason through variability and complexity that would otherwise require large, brittle code.

      “Software 1.0 I’m writing code, software 2.0 I’m actually programming by creating data sets and training neural networks… so software 3.0 is kind of about… your programming now turns to prompting. And what’s in the context window is your lever over the interpreter, that is the LLM.”
      

  • Karpathy predicts a future flip where neural nets do most computation and CPUs become co-processors.

    • He imagines devices feeding raw video/audio into nets that render context-specific UIs and reframe data into new knowledge bases.

      You feed raw videos, like imagine a device, he takes raw videos or audio into basically what's a neural net and uses diffusion to render a UI that is kind of like, you know, unique for that Moment in a certain sense. And I kind of feel like in the early days of computing, actually, people were a little bit confused as to whether computers would look like calculators, or computers would look like Neural nets. And in the 50s and 60s, it was not really obvious which way it would go. And of course, we went down the calculator path and ended up building classical computing. And the neural nets are currently running virtualized on existing computers. But you could imagine, I think that a lot of this will flip, and that the neural net becomes kind of like the host process, and the CPUs become kind of like the co-processor.
      

  • Karpathy argues models excel in domains that are verifiable because RL training supplies verification rewards, producing jagged capabilities that peak on code and math.

    • The models now patch this, I think, but the new one is, I want to go to a car wash to wash my car, and it's 50 meters away, should I drive or should I walk? And state-of models today will tell you to walk because it's so close. How is it possible that state-of Opus 4.7 will simultaneously refactor 100,000 line code base or find zero-day vulnerabilities and yet tells me to walk to this car wash. This is insane. And to whatever extent these models remain jagged, it's an indication that number one, maybe something slightly off. Or number two, you need to actually be in the loop a little bit and you need to treat them as tools. And you do have to kind of stay in touch with what they're doing
      

  • But he points out that often times the reason why models are actually good at something is becasue the model trainers made decisions; adding chess data dramatically improved chess performance between GPT-3.5 and GPT-4.

    I think it's some kind of a combination of verifiable plus labs care. Maybe one more anecdote that is instructive is from GPT 3.5 to GPT 4, people noticed that chess improved a lot. And I think a lot of people thought, oh, well, it's just a progression of the capabilities. But actually, it's more that, I think this is public information, I think I saw it on the internet. A huge amount of data of chess made it into the pre-chaining set. And just because it's in the data distribution, basically the model improved a lot more than it would just by default. So someone at OpenAI decided to add this data, and now you have a capability that just peaked a lot more. And so that's why I think I'm stressing this dimension of it, as we are slightly at the mercy of whatever the labs are doing
    

  • profitable RL environments / domains

    • At one point, he is asked about verifiable domains that founders should look at - that is, domains other than law, medicine, coding, math, etc that are "blind spots" in today's models. He doesn't give anything away, but this is quite an interesting rorsarch test. I personally thought he might have been alluding to either education or business ... or both ...

      • Stephanie Zhan

        • What would your advice be to the founders in the audience?
          

      • Andrej Karpathy

        • verifiability makes something tractable ... Because you can throw a huge amount of RL at it. ... if you are in a verifiable setting where you could create these RL environments or examples, then that actually sets you up to potentially do your own fine-tuning, and you might benefit From that ... is fundamentally technology that just works. You can pull a lever ... I do think there are some very valuable reinforcement learning environments that people could think of that I think are not part Of the... Yeah, I don't want to give away the answer, but there is one domain that I think is very... Oh, okay, sorry. I don't mean to vague post on the stage, but there are some examples of this.
          

  • I loved this suggestion about an interview test for an Agentic Engineer

    • a lot of people ... want to hire strong agentic engineers. ... most people are still not refactored their hiring process for agentic engineer capability, right? Like if you're giving out puzzles to solve, then this is still the old paradigm. I would say that hiring has to look like, give me a really big project and see someone implement that big project.
      

    • Like let's write, say, a Twitter clone for agents, and then make it really good, make it really secure, and then have some agents simulate some activity on this Twitter. And then I'm going to use codex 5.5 x-high to try to break your website that you deployed ... and they should not be able to break it. ... so maybe it looks like that, right?
      

  • Karpathy: as intelligence becomes cheap, you can delegate thinking to agents but you still must internalize understanding.

    • He cites a tweet: "you can outsource your thinking, but you can't outsource your understanding" Quote
    • He relates this to his recent work on LLM knowledge bases / wikis:

    • because I'm still part of the system, ... Somehow, information still has to make it into my brain. And I feel like I'm becoming a bottleneck of just even knowing what are we trying to build, why is it worth doing, how do I direct my agents and so on. So I do still think that ultimately something has to direct the thinking and the processing and so on. And that's still kind of fundamentally constrained somehow by understanding. And this is one reason I also was very excited about all the LLM knowledge bases, because I feel like that's a way for me to process information. And anytime I see a different projection onto information, I always feel like I gain insight. So it's really just a lot of prompts for me to do synthetic data generation kind of over some fixed data.
      

• AI/Slop vs AI/LLM/Psychosis.
  • Beyond artificial intelligence psychosis: a functional typology of large language model-associated psychotic phenomena - ScienceDirect
  • The utility of language depends not on the assumption that the recipient knows the meaning of a word but that the recipient's use of a word passes some threshold of utility. I'm thinking of using the term "splay" to refer to the number of assumed degrees of vocabulary freedom a speaker uses. LLMs have a high degree of splay, and that is problematic for my own understanding.
  • I was randomly browsing Beads/GitHub/Issues, ran into The philosophical foundation for what you're building · Issue #3573 · gastownhall/beads by a person who calls themselves "Peter Cooper" aka Pass-the-Butter on Github and whose byline is "philosophy engineer."
    • It was a git issue advertising a Paper - The Shape of Thought and an accompanying GitHub/Repo agilemeshnet/theshapeofthought: The Shape of Thought - A Measurement Programme for the Shapes That Let Cognition Survive Substrate Transitions. Peter Cooper, April 2026..
      • I got a bit enthralled by what I saw. It reminded me of Person/Buckminster Fuller.
    • I couldn't find the original author online, though given its vast, marbled rhetoric it's possible that it's the same Peter Cooper that gave a talk Nov 9 2011 at the Institute of Mathematics and its Applications (IMA) about

      • the remarkable fact that Mount Everest, digital logic, the feminist movement, a ship’s anchor, the Renault Laguna, the Times New Roman font, The Hunt for Red October and the Japanese Art of Origami are all linked by a simple, if not humble, piece of mathematics accessible to primary school children.
        

      • Peter Cooper was also the name of the first person to build a steam locomotive in the US and that founded the Cooper Union in NYC and that was an 1876 presidential nominee. It's a very common name.
• The 'Dangerous Ideas' of Mary Everest Boole - IMA
  • 

• garddiff

  • Filed

    • DotSlash
    • Person/Jeff Dickey/Blog/26/03/02/Top 10 Features in Mise You're Not Using
    • Person/Andrej Karpathy/X/26/04/AI Psychosis

  • Updated

    • Person/Jeff Dickey