WWDC 2024 - All Apple, All the Time

Wow. So it is apparently free. That’s very impressive.

At least apple is trying to justify why they have world class laptop processors in their phones / tablets.

Huh, so Siri can use external models, starting with ChatGPT 4o. I wonder who’s paying for that, or if users need a subscription for the external models. Oh, it’s free, and privacy.

I’m really looking forward to trying this out - if it works it’s probably the largest jump in how phones work in a long time.

This seems confusing. On device, Apple cloud, and third party models?

The on device vs cloud seems like you aren’t making that decision - it’s a question of what can be run on-device. Running as much as possible on-device saves them all that compute, since you pay for hardware / electricity.

Ok, this ChatGPT stuff is impressive, especially how it is being implemented. Apple is doing a much better job than Microsoft introducing the relevance of AI in improving work flow and personal activities. Kudos to Apple … if everything works as advertised!

I think it’s internally consistent. If it can all be done on device, it is. If it can’t be done on device alone, it tries Apple’s Cloud. If Apple’s cloud can’t handle it, it can hand it off to ChatGPT (with your permission). They mentioned there’s an assessment process.

“Smart assist for Swift code generation” later today.

Yup, Apple is ready to eat everyone’s lunch.

Sure, and I get that angle. But when it prompts you whether you want to use GPT4, it just makes me wonder why Apple’s AI isn’t good enough to do what seems pretty basic. And it makes all the privacy stuff about the Apple cloud seem redundant. Either it’s something only Apple could do (as they originally implied), or that was just bullshit and anyone can, or it’s insecure. Do I trust OpenAI to run a tight privacy ship? Fuck no.

I don’t think Apple is handing off any private info to 3rd party LLMs, like chat GPT. It will simply forward the raw request. Unlike the private stuff they can do with that “Personal Context” they referred to, knowing who you are, what you look like, access all your app data, etc.

I don’t understand what “pretty basic” is here? You mean whatever they demoed?

I’d look at this as forward compatibility / integration. Whatever AI can’t do natively can get plugged in. How capable is Apple Intelligence vs GPT 4o? That’s remains to be seen.

My workplace has a “private cloud” for chatGPT 4o and for copilot. Can I send engineers to validate that it’s actually private? Nope. I think making your privacy code base verifiable (and people are going to go after it because it’s a good way to gain notoriety) seems pretty smart.

Yeah, I haven’t been able to scrub back in the live stream to be more specific, but certainly the first thing they showed seemed pretty straightforward given the claims they were making earlier in the presentation.

The way I see it is this:

Is the request about something on your device or about you or people you know, that Apple knows about because it’s stored on an iPhone? If so, Apple handles it. So, if you had a recipe for lemon chicken in your Notes app, and you ask for a recipe for supper tonight, bam. It might suggest it.

But if you don’t have any recipe on device, then it might farm out that request to ChatGPT. But it won’t tell chatGPT who you are dining with, or at what time.

Sure, I agree on that. But they’re not doing that for ChatGPT. So telling users they can only do something by giving up their privacy (at least to some extent) seems a very un-Apple thing to do.

This seems like a distinction without a difference, really, The raw request can easily have private info: “Hey Siri, where’s the closest place to town X in Texas to get a legal abortion?”. It just won’t have the context.

Yeah, my brain kind turns off when they show trivial examples.

It’ll be interesting to see how the breakdown works. There’s some possibility that some general purpose questions that are far outside of the personal-context assistant don’t work as well with Apple I - maybe they have trained smaller cheaper models that are performant enough for the assistant tasks, but maybe it’s less general.

I’m not clear if they are still doing some sort of “private cloud” with chatGPT (Apple is big enough to run their own instances of the model), or if they just ask you if it’s OK to share certain info.

I’m guessing you’re going to pretty explicitly need to invoke a 3rd party model. I don’t think the phone is going to randomly pick one for you.

That does seem to be the case, which is obviously better than the alternative. Though it wasn’t clear to me whether you can fall back to Apple’s or if it will just refuse to do the task.

And that’s why it warns you that it’s handing off to a third party and you approve if it continues or not… So you can realize the privacy implications and cancel the request if it doesn’t make privacy sense to you, personally.

Being in Canada, where cannabis is legal for recreational use, if I were to ask for a “Best strain of weed to chill and watch Deadpool with”, I’d be hypothetically ok with that going to chatGPT. But someone in a regressive regime where that would get them in trouble if intercepted might say “cancel!!”

In the ideal world, for Apple, in this scenario, I think you would have a weed recommendation app on your phone already and it would pull data from it and never go to the cloud at all.

Anyway, I’m super excited to try out the assistant type stuff (really don’t care about the writing and image generation side of things), though I wonder how good it will be in practice for me as I have an Android phone and don’t keep a lot of personal stuff on my iPad. The Mac OS implementation is probably more promising, but eg I don’t use Mail on my laptop, and don’t really use Notes for much other than as a scratchpad. I do at least import my Outlook/Google calendars, and if it can see into my Word/Excel files and PDFs that would be useful, but I wonder if MS is going to block that to favour Copilot.

For reference, this is what we have at work.

They claim:

I just watched a documentary on Netflix about Ashley Madison. I think they used the same language, hehe.

Doesn’t Apple’s finder already index all of that? I find presentations on my mac via finder and keyword search. Having better natural language search for that stuff will be amazing - instead of needing very specific and exact keywords to find something being able to describe more general context of what I’m looking for will help me dig up old analysis and talks more quickly.