The Apple Watch Has a Problem

The Apple Watch Series 12 and Apple Watch Ultra 4 look like pretty minor updates. Between the new iPhones and the rest of the Techtober noise, it would be easy to move on.

But I think something much bigger is happening here. These updates look like very specific attempts to take on two companies. The first is Whoop, and that part is pretty straightforward. The second is OpenAI, and that is where things start getting weird.

Because the most interesting question about the new Apple Watch is not how accurately it measures your heart rate. It is what happens when the device on your wrist can quietly turn conversations around you into notes.

Apple’s First Target: Whoop

The market for screenless fitness trackers has more than doubled in the past year. Whoop accounts for more than half of it, while products from Fitbit, Garmin, Amazfit, and others are chasing the same opportunity.

Apple clearly wants a piece of that. But instead of making a screenless fitness band, which I would honestly like to see, it is making the Apple Watch more competitive with those devices.

That starts underneath the Series 12. Apple has redesigned the sensor array with larger, more powerful, more efficient LEDs and photodiodes. The point is to collect measurements much more frequently without making battery life worse.

Background heart rate readings now happen every five seconds rather than every five minutes. That brings the sampling frequency into line with a lot of competitive wearables.

Apple has also published a heart rate accuracy study explaining why it believes its wearable is the most accurate, including comparisons with rings. That is Apple’s claim, but the paper is there for anyone who wants to examine the details.

The sensors, though, are not the main reason this feels like a direct shot at Whoop. The Health app is.

From collecting health data to telling you what to do

Previously, Apple Health was largely a place where information accumulated. Useful information, sure, but a lot of numbers to interpret yourself. The updated app uses AI to organize that data into an overall picture of your health, along with actions you can take to improve it.

That is a big part of the appeal of Whoop. It targets people who want to be proactive about their health and need more personalized guidance than an occasional reminder to stand up or close their rings.

The parallels are pretty hard to miss:

• Sleep and recovery: Whoop provides sleep and recovery scores to help explain how rested you are and how prepared your body is for the day. Apple Health now offers sleep and readiness scores.
• Long-term health: Whoop’s Healthspan section estimates a Whoop Age and explains the factors contributing to it. Apple’s Longevity section provides a health age with contributing factors to focus on improving.
• Blood testing: Whoop has an advanced lab section for incorporating comprehensive blood test results. Apple Health’s new Metabolic Health section similarly lets you incorporate results for a more detailed picture.

So, yes, Apple is going after the advice layer, not just the hardware.

I still think that leaves something important unanswered. Part of the screenless fitness tracker market exists because people specifically want less screen. They want the digital detox and battery life advantages that come with it. Better software on a smartwatch does not automatically address that preference.

But this is the less controversial part of the update.

Audio Intelligence Is the Bigger Story

Apple groups four capabilities under Audio Intelligence:

• Music Recognition: Identifying a song playing nearby.
• Sound Recognition: Recognizing sounds such as a doorbell, an established accessibility use case.
• Live Rewind: Turning the previous 15 seconds of speech into text after a double-tap on the Digital Crown.
• Siri Recap: Using ambient listening to create high-level notes from conversations throughout the day.

The first two are familiar. The last two are the ones I think could give Apple a lot of trouble.

Siri Recap is designed to save conversation summaries in the Siri app. Live Rewind lets you retrieve what was just said, read it on your wrist, and optionally save that text for later.

Apple’s restaurant example has a server reading a long list of specials. Rather than trying to remember everything, you double-tap the crown and get the list on your wrist.

Convenient? Maybe. But you could also just ask the server to repeat it.

It reminds me of using Visual Intelligence to identify a dog while its owner is standing right there. There is a technical solution, but there is also a very normal human interaction that already solves the problem.

More importantly, that modest convenience comes attached to a much bigger reputation problem: the new Apple Watch is listening all the time.

How Apple Tries to Keep the Audio Private

To be fair, Apple has put real work into the privacy architecture. Its assurances include that audio is not recorded or stored, that the audio cannot be accessed, that individual speakers are not identified, and that syncing is end-to-end encrypted.

Those are substantial claims. Understanding them requires looking at what Apple means by capturing, processing, and retaining audio.

The S11 chip and Secure Exclave

These capabilities are limited to the Series 12 and Ultra 4 because they use the new S11 chip, which includes a component called the Secure Exclave.

The simplest way to think about it is as an isolated section that keeps raw audio separate from the rest of the chip. The microphone continuously takes in sound, and that sound passes through a rolling buffer that is constantly overwritten.

Machine learning inside that protected area recognizes patterns associated with particular sounds. A doorbell has a recognizable signature. When the system detects it, it can trigger a notification.

That is a genuinely useful accessibility feature. Music also has recognizable patterns, which is the basic idea behind familiar experiences such as Pixel’s Now Playing or Shazam, which Apple acquired.

What happens when you use Live Rewind

Live Rewind adds an explicit action to that rolling audio processing. Here is the basic sequence:

1. You double-tap the Digital Crown.
2. The watch selects the previous 15 seconds of audio from its protected buffer.
3. It encrypts that audio and sends it to your iPhone.
4. The iPhone decrypts the audio and converts the speech into text.
5. The text goes back to the watch for display.
6. The raw audio is deleted from both devices.

That is a fairly elaborate design intended to prevent accessible, lasting audio recordings from accumulating.

But I would still argue that continuously capturing audio into a buffer counts as recording in the everyday sense, even if the rest of the chip cannot access that buffer and the contents keep disappearing. Maybe that is semantics.

For most people, the technical distinction probably will not matter anyway. Explaining that a device can constantly retrieve the last 15 seconds of speech, but is also not recording all the time, is going to be a tough sell.

Siri Recap Has a Consent Problem

Siri Recap goes beyond manually retrieving a short snippet.

In principle, the microphone keeps processing incoming audio until the system recognizes something that sounds like a conversation. That detection triggers the capture, encryption, and speech-to-text process. The resulting text is then condensed into readable notes.

At the end of the day, the Siri app can contain titled summaries and key points from conversations, meetings, and other exchanges.

Apple says this is opt-in. You can also restrict it to certain times, such as working hours, so it captures meetings and work conversations rather than running throughout your entire day.

Okay. But who else is in those meetings?

The person wearing the watch has opted in. The other people in the conversation have not.

That is the part I find invasive. Their words can still become a saved summary on someone else’s phone. Protecting the raw audio from Apple does not resolve whether those people wanted their conversation processed and preserved in the first place.

Apple has considered nearby people, too. Its stated protections include:

• No speaker attribution in the summaries.
• An attempt by Siri to exclude content that sounds sensitive.
• An audible chime when Live Rewind is activated.

But Siri Recap does not have that same audible signal. There is no chime or light telling the person next to you that the watch has detected a conversation and may be summarizing it.

Not naming speakers is a safeguard. Trying to omit sensitive information is another. Neither is the same thing as telling someone what is happening and letting them choose.

Why a Watch Is Different From a Smart Speaker

Google Home and Alexa devices already listen for recognizable patterns, including wake words. Once activated, they give an indication and start responding.

The important distinction is the setting. A smart speaker is installed in a home, in a place its owner controls. A watch comes with you into the world, into conversations with people who had no role in choosing it.

Smart glasses make a useful comparison. Meta’s glasses have attracted criticism over people recording in uncomfortable situations, along with reports of restrictions and bans.

But if I notice camera-equipped glasses and a recording light, I at least have a signal. I can decide to leave that interaction.

With a watch quietly generating recaps, I may have no reason to know anything is happening.

This is why the issue is bigger than whether Apple can hear the audio. There are two separate boundaries:

• Data security: Who can access the captured information?
• Social consent: Who gets to decide whether a conversation becomes saved information at all?

Apple can do excellent work on the first and still cross the second. Privacy is not just a one-person decision.

The Second Target: OpenAI and the AI Companion Market

So why take this risk?

My read is that Apple sees another device category forming and does not want to be caught behind it.

Portable AI note-takers and wearable pins already exist. Meanwhile, OpenAI has recruited Apple employees and is working with Jony Ive’s design firm on hardware widely expected to be some kind of portable AI companion.

The expectation, not a confirmed final specification, is a device you carry everywhere that listens, summarizes, and provides proactive assistance.

Some people believe that category could eventually replace the smartphone. If you are Apple, that is a pretty direct threat.

Apple also has a huge advantage: hundreds of millions of people already use its phones. It can add companion-like capabilities to hardware in that existing ecosystem instead of persuading people to adopt an entirely new platform.

This is the central question I explored in Apple versus the AI race: can an AI company make hardware compelling enough to pull people away from the iPhone, or can Apple make its AI good enough that people stay?

The health features look like an answer to Whoop. Audio Intelligence looks like an early answer to the AI companion.

A Quiet Launch Does Not Make the Question Smaller

At the time of this assessment, the watches are available, but these new conversation features are not live yet. They are scheduled to arrive later in beta.

I cannot know Apple’s internal reasoning, but that rollout feels cautious. The hardware is already on wrists, complete with the S11 and Secure Exclave, while the more controversial functionality arrives separately.

Apple could have put the work into bringing a similar experience to the iPhone. But attaching an always-listening reputation to its biggest product would be a much bigger risk. Starting with a delayed watch beta feels like a way to test that boundary more quietly.

There are also rumors about AirPods with cameras. Those remain rumors, and the exact hardware and capabilities are not established here. But the possible pitch raises a familiar tension: ambient awareness that helps identify landmarks and surroundings, paired with technical assurances about what is captured, processed, encrypted, or retained.

Those distinctions may matter enormously to engineers. They may be much harder to explain to everyone sharing the space around the device.

And this is not only Apple’s problem. OpenAI, Google, Amazon, and other companies pursuing ambient AI will face the same question. I am not convinced competitive pressure will make any of them more restrained.

My concern is not that Apple has ignored security. It plainly has not. It is that security is being asked to answer a question about consent.

A watch that helps you understand your own health makes sense. A watch that quietly turns other people’s conversations into your personal notes crosses a different line. When these features launch, the public reaction will tell us a lot about whether that is a trade people are actually willing to accept.