In particular, we’ve built up a vast amount of tooling that assists humans in writing 1.0 code, such as powerful IDEs with features like syntax highlighting, debuggers, profilers, go to def, git integration, etc. In the 2.0 stack, the programming is done by accumulating, massaging and cleaning datasets. For example, when the network fails in some hard or rare cases, we do not fix those predictions by writing code, but by including more labeled examples of those cases. Who is going to develop the first Software 2.0 IDEs, which help with all of the workflows in accumulating, visualizing, cleaning, labeling, and sourcing datasets? Perhaps the IDE bubbles up images that the network suspects are mislabeled based on the per-example loss, or assists in labeling by seeding labels with predictions, or suggests useful examples to label based on the uncertainty of the network’s predictions.
This whole post feels wild. but probably just as crazy as people working in VIM watching IDE’s take over. I do think there is going to be more training and triage for sure on future software projects. but there is a whole world of projects where software “1.0” will work “jsut” fine.
Quote Citation: Andrej Karpathy, “Software 2.0. I sometimes see people refer to neural… | by Andrej Karpathy | Medium”, Nov 11, 2017, https://karpathy.medium.com/software-2-0-a64152b37c35