Synthèse
And when I say "verifiable," I don't mean "trustworthy," which a lot of people confuse and has always been a criticism of AI since 2023. This is verifiability in the test-driven-development sense. Or the prompting evals sense. It's testing and judging whether the output is what we asked for. I tremendously appreciate Andrej Karpathy's concept of Software 2.0, where he talks about 1.0 being about writing software, and 2.0 being about verifying software. I often think of the Scientific Method as t
And when I say "verifiable," I don't mean "trustworthy," which a lot of people confuse and has always been a criticism of AI since 2023. This is verifiability in the test-driven-development sense. Or the prompting evals sense. It's testing and judging whether the output is what we asked for. I tremendously appreciate Andrej Karpathy's concept of Software 2.
0, where he talks about 1. 0 being about writing software, and 2. 0 being about verifying software. I often think of the Scientific Method as the Last Algorithm, because it's how to pursue all others. This rhymes closely with several converging thoughts I've had over the last 10 years around goal pursual (TRIOT, 2016), hill-climbing, moving from current state to ideal state, and automation of the scientific method.
Central to all of these is Verifiability. You can't hill-climb if you don't know if you're getting higher. So the primary questions become: What does success look like, andWhat are the verifiable markers indicating that you're making progress towards it? I'm obsessed with finding ways to do this across domains. And the cross-domain bit is actually what's important.
One of the main reasons AI impacted coding first is that it's one of the more verifiable domains that exist. There are degrees of this, of course. You know if something compilesYou know if something runsYou know if something produces the output that you wanted So, what are the equivalents of these for other tasks? Like business tasks and personal tasks. That is what will start to congeal over the next year and after.
Right now it's not much of a consideration because most people aren't thinking deeply about AI or how it will be used or how it will apply to their business. They are too busy responding to the board and management telling them to implement it yesterday already.
Source originale
karpathy.ai