Siri’s Getting New AI Smarts in iOS 18 — But Not All at Once
Credit: Wachitwit / Shutterstock
Toggle Dark Mode
There’s every reason to believe that Apple will have some significant Siri improvements to talk about when it takes the wraps off iOS 18 next month. However, it turns out that not all of those changes are going to show up right away.
While a rumored deal with OpenAI could bring ChatGPT-like features to Siri, even that’s not a certainty quite yet. OpenAI could power other things behind the scenes instead, such as summarizing voice memos and notes. After all, Siri doesn’t just need to be more conversational — it needs to be more useful.
Bloomberg first reported on the OpenAI talks earlier this month, suggesting they were centered around ChatGPT. However, a new report from The Information reveals that the two companies have actually been collaborating for some time.
Members of Apple’s machine learning team reportedly met with members of OpenAI, including CEO Sam Altman, resulting in a deal that gave Apple access to OpenAI for internal testing.
In those tests, Apple engineers connected ChatGPT to Siri, creating impressive demonstrations of Siri handling more complex queries than usual, including by better understanding the context of what users were saying.The Information
The report concurs that ChatGPT is a big part of the arrangement, but it appears that the OpenAI chatbot could be considered a supplemental capability. For instance, while Apple wants to use OpenAI technologies to make Siri more naturally responsive, it also wants to ensure that users know when ChatGPT is answering their questions. Apple’s strong focus on privacy likely means it’s looking to create a firewall to ensure no personal information is fed into the ChatGPT engine without express user consent.
It’s worth noting that Siri already integrates with ChatGPT through OpenAI’s iPhone app, allowing Siri voice queries to be directed to the chatbot. It’s a bit awkward, but it works. Presumably, whatever Apple cooks up will be much tighter.
‘Project Greymatter’
However, that’s not all that Apple has up its sleeve. While OpenAI may help Siri better understand us, it’s also working on other projects to help Siri do more of what we ask.
An exclusive report by AppleInsider’s Marko Zivkovic shares details on a project within Apple dubbed “Greymatter” that focuses on “practical benefits for the end user.” This includes the ability to summarize notifications more effectively, allowing users to get an overview of their recent notifications without having to have them all read off one by one in excruciating detail.
That’s joined by a new “smart response framework” that will allow it to be more contextually aware of the information on your iPhone, such as calendar events, locations, dates, and much more (that alone could address one of the biggest frustrations I’ve had with Apple Maps and Siri over the years — its seeming inability to understand that if I’m asking for directions to a location, I’m probably not looking for something 3,000 miles away, especially if it’s one I visit regularly).
Siri is expected to receive significantly updated response generation capabilities, through a new smart response framework, as well as Apple’s on-device LLM. When generating replies and summaries, Siri will be able to take into account entities such as people and companies, calendar events, locations, dates, and much more.Marco Zivkovic
According to Zivkovic, this will be powered by Apple’s on-device AI, which isn’t surprising considering the privacy implications of factoring in personal data. It’s also the same Ajax large language model (LLM) that’s expected to power transcriptions and summarizations in the updated Voice Memos app. However, it’s unclear how much of a role, if any, OpenAI’s technology plays here, although Zivkovic suggests it could be used to fill in the blanks for things like “answering complex video questions,” which Project Greymatter and the Ajax LLM can’t do.
AI Image Generation
Zivkovic has also heard that Apple is working on a new “Generative Playground” that will be able to create and edit images, including unique emojis. This is reportedly already integrated into iMessage, although it will likely support sharing in other apps.
In Apple’s test environments, it’s possible to generate an image through artificial intelligence and then send it off through iMessage. There are indications that the company has a similar feature planned for the end users of its operating systems.Marco Zivkovic
Similar technology is reportedly being tested to improve image editing in the Photos app, replicating features like Google’s Magic Eraser to remove objects from photos.
Letting Siri Inside Your iPhone Apps
Bloomberg’s Mark Gurman weighed in a few hours after AppleInsider’s report to add that Siri’s new AI capabilities will also include the ability to dig into your apps and perform functions like opening documents, managing emails, and asking for summaries of articles in Apple News.
The new system will allow Siri to take command of all the features within apps for the first time, said the people, who asked not to be identified because the initiative isn’t public. That change required a revamp of Siri’s underlying software using large language models — a core technology behind generative AI — and will be one of the highlights of Apple’s renewed push into AI, they said.Mark Gurman
Gurman notes that most basic AI tasks will be performed on-device, with more advanced capabilities handled in data centers filled with custom M2 Ultra-powered AI servers.
While Siri can already perform some tasks within apps, subject to what third-party developers allow through App Intents and Siri Shortcuts, Apple plans to go deeper, using AI to analyze what people are doing on their devices and empowering Siri to assist.
At the start, the new Siri will handle one command at a time, but Apple has plans to to allow users to chain commands together. For example, they could ask Siri to summarize a recorded meeting and then text it to a colleague in one request. Or an iPhone could theoretically be asked to crop a picture and then email it to a friend.Mark Gurman
When iOS 18.0 is released, these Siri app integrations will be limited to Apple’s first-party apps, as Gurman says it’s “one of Apple’s more complex AI initiatives.” Support for third-party apps isn’t coming until next year, although it’s expected to arrive in an iOS 18 point release rather than iOS 19.
[The information provided in this article has NOT been confirmed by Apple and may be speculation. Provided details may not be factual. Take all rumors, tech or otherwise, with a grain of salt.]
