Do You Trust This Computer?

OpenAI Codex is bananas, and has kind of rendered all my halfhearted attempts to learn to code pointless.

I don’t know much about AI, but in my experience as a programmer, it’s very, very hard to get this kind of thing past the “toy” stage, because if it doesn’t do exactly what you want on the first try, you still need a human to review the code, and reading and understanding code is hard enough when it was written by a human specifically with the goal of making it optimally readable.

The “make a function that does X” thing is instructive there, because generally, typing out the contents of individual functions isn’t the really hard part. The hard part is understanding what should go into functions and when they should be called and how to structure those components in a way that makes sense and is scalable, etc, etc. You still need a human to do all that hard stuff. This is just like…a more elaborate auto-complete.

It’s like the self-driving car problem. Until the computer can actually do 100% of complex use cases in a totally hands-off way, you can’t actually use it for anything more than toy use cases. I suspect this is going to be “90% of the way there”, and “just a couple years from a breakthrough” for decades.

There’s probably uses in rapid prototyping and toy/hobbyist programming though.

Sure, but it’s an elaborate autocomplete that for example knows what libraries are out there and what APIs to call and how. I’ve never really even got to the “useful toy” stage of coding, partly because I don’t know what libraries there are so I’m faced with the daunting prospect of doing almost everything from scratch. But with this it looks like I could pretty easily knock together some simple script type programmes to do useful things, without having to do a ton of background reading to figure out who has done the hard work for me.

It’s also very cool in that GPT3 is impressive on a surface level but usually generates gibberish semantically, whereas this creates something that works and is (often) what is asked for.

To be fair, a natural language to bash script converter would be pretty useful. If I have to google for what xargs does one more time…

Well, I probably will. But I’ll think about writing a python script or something instead.

Oh, we’re sending 1472 emails. I bet people love when we test this!

I would like this if it could read in my prototype R code and convert it into another language. You’d think since I have a huge amount of specificity in actual code, that would be useful. The problem would be moving functional programming ideas into an imperative language.

Very cool stuff. Bonus AI fails at the end.

That will prove very useful for photoshopping my male friend’s head onto pictures of women engaged in hardcore pornography and texting it to him with a clever comment, a thing I do on a daily basis.

Now I know where the next round of NFT “art” is coming from.

[The actual work is pretty amazing. I may have to grab the notebook tomorrow and tool around a bit.]

DeepMind not going to let OpenAI have all the fun:

I just learned about AI’s “Unreal Engine trick”.

How far we’ve come from Unreal Engine being associated with brown.

AI has mastered the Apple iPod problem!

This feels like a category error to me. The defining feature of ChatGPT and indeed all large language models is that they hallucinate facts. They produce bullshit in the classical sense. Of course you shouldn’t ask it factual questions and expect reliable answers. But, of course, it’s a tool and if you give people a tool they will misuse it. So I don’t know how you guard against what Kubacka worries about toward the end of the thread. You can put warnings on the output, but a) many people will ignore that, and b) it doesn’t help against bad actors who will eg use ChatGPT for SEO spam.