The Day the Provider Stopped Reading Your Chats

An ISOC LIVE Summary

Paper: The Day the Provider Stopped Reading Your Chats: WhatsApp Introduces Encrypted Chats with Meta AI

Authors: Theodore Christakis – University Grenoble Alpes; Peter Swire – Georgia Institute of Technology

Publication: AI-Regulation Papers 2026-05-4 (May 2026)

Overview

This paper offers a first analytical reading of Meta’s Incognito Chat with Meta AI, announced on 13 May 2026, which the authors call the most significant structural move in consumer chatbot privacy of the past two years. Rather than relying on contractual promises not to inspect user conversations, Meta has deployed an architecture designed so that it cannot read chats between users and Meta AI. The authors read this as the first real-world deployment of “Sealed Mode” — the architectural concept Christakis proposed in Part 1 of his study on consumer chatbot confidentiality (March 2026), in which a provider verifiably narrows its own access by architecture rather than by policy, from the moment of collection.

The authors are explicit about the limits of the exercise: they posted promptly after the official announcement, without having read other public commentary or having the text reviewed by other readers, and they have not conducted a technical assessment of the quality of Meta’s TEE implementation.

What has been announced

For roughly a year, Meta had been running a system called Private Processing for specific WhatsApp AI features — message summarisation and writing assistance. The announcement extends that same architecture to the chat conversation itself between a user and Meta AI. Until now, a user who summoned Meta AI inside WhatsApp saw their prompt and the response leave the end-to-end encrypted pathway and reach Meta’s servers in readable form. That is the architectural anomaly the announcement is meant to close.

Incognito Chat launches on WhatsApp and the standalone Meta AI app, with rollout expected over the coming months. Three guarantees are emphasised: the chats are protected by encryption and the TEE such that no one, including Meta and WhatsApp, can read them; the chats are not saved or stored, disappearing when the user exits, with Meta AI losing conversational context; and web search is conducted privately, de-linked from the user’s identity, with the option to disable it entirely. The feature is text-only — no image upload or generation. Meta has also pre-announced Side Chat, a forthcoming Private Processing feature allowing Meta AI to be invoked within existing WhatsApp conversations without disrupting the main thread.

Private Processing architecture

The architecture rests on three components:

• Trusted Execution Environments (TEEs) — a hardware-isolated zone inside a server, built here on AMD’s SEV-SNP chips and Nvidia’s confidential GPU platforms, where data can be decrypted and processed but remains inaccessible to the operating system, the hypervisor, and Meta’s own administrators.

• Oblivious HTTP (OHTTP) — a routing layer sending the encrypted request through an independent third-party relay (Fastly) before it reaches Meta’s gateway. The relay sees who the user is but not what is sent; the gateway sees the encrypted payload but not the identity.

• Remote attestation — before sending content, the device asks the server hardware for a cryptographic “quote” certifying that the code inside the TEE is the exact version Meta has publicly logged and auditors have reviewed. If the quote does not match, the device refuses to send.

Inside the enclave, the message is decrypted, the language model (Muse Spark in the current configuration) generates a response, the response is re-encrypted with an ephemeral key, and only ciphertext leaves. The TEE then purges the data from memory. Nothing is written to disk.

Two external audits support the trust claims. Trail of Bits published a review in early 2026 identifying 28 issues including eight high-severity findings affecting enclave integrity — among them a path by which a malicious hypervisor could have used fake ACPI tables to read protected memory. All eight were remediated before production. NCC Group conducted a 115 person-day assessment of the message-summarisation service in late 2025, confirmed the statelessness claim on the basis of source-code review, and identified a residual risk: the OHTTP anonymity guarantee depends on Meta not colluding with the third-party relay. The audits do not establish that the system is unbreakable; they establish that the architecture is real, that the trust assumptions are limited and externally documented, and that the major attack surfaces have been examined by independent specialists.

Is it equivalent to end-to-end encryption?

Not the same thing, the authors answer, but the closest functional equivalent currently available for a service requiring server-side computation.

In classical E2EE, plaintext exists only on the sending and receiving devices and the provider routes ciphertext. That works because the recipient is a human with a device that can decrypt locally. It does not work when the recipient is a large language model needing server-side GPU resources. Private Processing relocates the “endpoint” from the recipient’s device to a hardware-isolated zone inside the provider’s own data centre. Drawing on Swire, Ahmad and Specter’s forthcoming work — and on Chelsea Komlo and Britta Hale’s concept of endness — the authors describe the announcement as a deliberate redefinition of where the “end” of the encryption sits.

The protection is therefore architectural rather than absolute: it depends on the integrity of the hardware enclave, the correctness of the code inside it, and the trustworthiness of the chip manufacturer and the third-party relay. None of these dependencies exist in classical user-to-user encryption. The authors call the result “operator-blind processing” — Meta retains the computational role but renounces, by architecture, the ability to read what the service computes.

Sealed Mode, from concept to deployment

Sealed Mode emerged from Part 1’s examination of how the five most prominent consumer chatbot providers (ChatGPT, Claude, Gemini, Grok, DeepSeek) treat user conversations across four established boundaries of confidentiality — training, human review, advertising, and operational sharing — plus a fifth emerging boundary, persistent memory and longitudinal profiles. The empirical finding was that terms of service reserving broad rights to the provider are not, on their own, a credible foundation for user trust where conversations contain health, financial, legal or other intimate information.

The six elements of the proposal are: no use for training; no human review except by narrowly defined exception with user consent; siloed personalisation; no advertising signals derived from content; strict retention limits; and cryptographic hardening calibrated to the deployment environment.

The authors find that Private Processing instantiates four of the six elements directly, takes a more restrictive position than Sealed Mode envisaged on a fifth, and instantiates the sixth — cryptographic hardening — more comprehensively than any deployed consumer chatbot architecture they are aware of. Ephemeral processing with no persistent storage satisfies the retention element and structurally entails that the content cannot train future models. Human review is architecturally impossible unless a user chooses to report a message. On advertising, Meta’s public documentation does not address the point explicitly, but the conclusion follows from the architecture: content the provider cannot read cannot generate advertising signals. On personalisation, Private Processing adopts full statelessness — more protective than the siloed personalisation Sealed Mode required, at the cost of a meaningful degradation in user experience.

The authors are careful about lineage: they do not claim Meta built Private Processing in response to Christakis’s study, which appeared well after the framework was announced in April 2025. The influence runs the other way. Private Processing and Apple’s Private Cloud Compute (June 2024) are the technical infrastructure that made it possible to propose Sealed Mode as more than a conceptual aspiration. What is new is the scope of deployment.

Why deployment environment matters

Part 2 of Christakis’s study, forthcoming, argues that the appropriate level of cryptographic protection is not uniform across “chatbots” but depends on the deployment environment, because that environment carries prior architectural commitments the chatbot inherits. The paper distinguishes three:

• Standalone consumer chatbots (ChatGPT, Claude, Gemini, Grok via a dedicated interface), where no prior cryptographic commitment exists and encryption is an open product-design choice.

• Chatbots embedded in E2EE messaging environments (Meta AI in WhatsApp), where the surrounding platform has committed that the provider cannot read messages, and any departure materially weakens the platform’s overall security stance.

• Enterprise or health-specific chatbots operating under Sealed Mode, where the appropriate protection depends on the regulatory and clinical context.

Meta AI in WhatsApp falls into the second category, and the anomaly to be solved was precisely that the chatbot pathway left plaintext on Meta’s servers in apparent tension with the platform’s commitments to its users. Part 2 warns that such embedding “materially weakens the platform’s overall security posture without necessarily being visible to its users.” The May announcement, if the architecture matches the description, is the response to that concern.

Moderation inside the enclave

If a provider cannot read a conversation, it cannot moderate that conversation the way an ordinary cloud-AI provider does. The familiar pipeline — automated classifiers, human reviewers, escalation channels, law-enforcement referrals — presupposes plaintext access.

Some moderation nonetheless operates, through two mechanisms inside the secure environment, both invisible to Meta’s external systems. The first is refusal at the model level, trained through supervised fine-tuning and reinforcement learning from human feedback. The second is classifier-based moderation: Meta runs safety classifiers inside the TEE — Llama Guard 2 (harmful content in prompts and responses), Llama Code Shield (vulnerable code suggestions), and CyberSec Eval 2 (cyberattack-assistance propensity) — which can trigger a refusal independently of the model’s own behaviour. In both cases the computation happens inside the enclave with full plaintext access; what leaves is an encrypted refusal. Meta’s external systems never see the prompt, the classifier’s judgement, or the refused response. Moderation happens, but “inside the bubble.” The provider sets the threshold; the actual refusals are invisible to the institution that runs the bubble.

Two further mitigations operate the same way. For categories of harm such as suicide and self-harm, models are generally trained to direct users to expert resources or support rather than refuse outright. For repeat attempts to bypass the safety layer, the user is temporarily blocked from further chatbot responses. Neither requires the provider to see the underlying content.

The combination is not a perfect substitute. Ambiguous requests, novel jailbreaks, fictional or roleplay framings concealing genuine intent, and multi-turn manipulations will still evade classifier-plus-refusal architectures — for the same reasons they do in non-confidential deployments. Confidential deployment does not solve the underlying classification problem; it constrains where classification happens and who can see the result.

No knowledge, no liability

The authors argue that the architectural move has a legal-institutional consequence that has not yet received the analytical attention it deserves. Much of the liability framework developing around consumer chatbot providers — in the United States particularly, increasingly in the EU as well — turns on what the provider knew or should have known.

Part 2 sets out how recent litigation turns on theories of provider knowledge: the Tumbler Ridge wrongful-death and personal-injury suits filed in California federal court on 29 April 2026 against OpenAI and Sam Altman personally; the Garcia v. Character Technologies litigation, settled in January 2026; the Adam Raine suit against OpenAI, filed in August 2025; and the Florida criminal investigation of OpenAI announced by Attorney General James Uthmeier on 21 April 2026 in connection with the April 2025 FSU shooting. The structural point is a perverse incentive: the more rigorously a provider monitors its product for signs of imminent harm, the richer the evidentiary record a future prosecutor or civil plaintiff can use against it — the pattern long observed in safe-harbour debates for online intermediaries, now visible in the chatbot setting.

Private Processing reduces the provider’s actual knowledge by design. If the architecture works as advertised, Meta has no internal records of what users discussed through the encrypted pathway beyond the bare fact that a request was made. It has renounced not only the capacity to read but the capacity to be later found to have known — a substantial reduction in the liability surface. Plaintiffs would apparently continue to be able to argue that the provider “should have known” enough to act. Accountability, if it is to operate at all in this architecture, has to relocate to the design phase, where the criteria embedded in safety classifiers and refusal patterns are decided, rather than the operational phase of reviewing individual conversations.

The authors note a countervailing effect: provider-side accountability mechanisms that some in this debate (including Christakis) have called for in the proactive-disclosure context may become less effective. Referral numbers in transparency reports would presumably fall, because the provider would less often have enough knowledge to trigger a referral, and external review of referral criteria would take place without access to the content processed in the TEE.

Going Dark, in a new dimension

The authors expect concerns from law-enforcement agencies and organisations focused on detecting online harms, including child sexual abuse material. The arguments will be familiar from the Crypto Wars of the 1990s, the 2015–2016 FBI–Apple confrontation over the San Bernardino iPhone, the European debate over proposed CSAM-scanning regulation, and recurrent US proposals conditioning intermediary liability protections on content-scanning. The paper situates the announcement within what Swire, Ahmad and Specter call Round 3 of the Going Dark Debates — the period since 2015 in which E2EE became the default for large categories of consumer communications (forthcoming, Fordham International Law Journal).

Is the chatbot version a genuinely new debate or the same debate in a new setting? The case for new rests on the provider’s role: in user-to-user messaging the provider is a routing intermediary between two human parties, whereas in the chatbot setting the provider operates one end of the conversation. Two consequences follow. The object of the law-enforcement demand changes — the state is no longer asking a provider to expose third-party communications it merely routes, but to retain the capacity to read its users’ interactions with its own product. And the architecture creates a novel accountability gap: because the model runs inside the enclave and its outputs are encrypted before they leave, the provider originates content it cannot inspect, which has no real analogue in user-to-user encryption.

The case for structurally the same is that in both settings the provider chose to encrypt, the user chose the encrypted service, unencrypted alternatives remain available, and the contested act is the corporate decision to render content unreadable. On this reading the question is the same proportionality question in both: whether the investigative benefits of compelling a provider to retain readability justify the privacy, cybersecurity, and other costs imposed on the overwhelming majority of users whose conversations are of no law-enforcement interest.

Swire, Ahmad and Specter have argued across all three rounds that the case for “going dark” is weaker than law-enforcement framings suggest, notably because what they call the “Golden Age of Surveillance” has produced unprecedented investigative material through other means — location records, device metadata, social-graph data, facial recognition, behavioural databases, and content available through default cloud backup and SaaS environments. The inability to read one subset of Meta AI conversations constrains one evidentiary surface while leaving many others operative.

The Instagram counter-data point. On 8 May 2026, days before the WhatsApp announcement, Meta discontinued the optional end-to-end encryption available for Instagram direct messages since 2021. The stated reason was that very few users had opted in and that those seeking end-to-end protection would be directed to WhatsApp. Some external analyses speculated the real reason was sustained pressure from law-enforcement agencies and child-safety organisations — the FBI, Interpol, and the National Society for the Prevention of Cruelty to Children among them — arguing that default encryption on Instagram would create a “dark space” for CSAM proliferation.

Meta thus moved within the same week in two opposite directions, retreating on Instagram and advancing on Meta AI in WhatsApp. The authors read this as a dual-track strategy: hardened privacy in the utility-focused messaging platform, a more moderated environment in the social-discovery platform. Whether it is sustainable — particularly where regulatory pressure on encryption is most intense, under the United Kingdom’s Online Safety Act and the European Union’s evolving Technology Roadmap on encryption — is one of the open questions.

The authors anticipate the specific form the argument will take. Law enforcement will point to recent incidents in which chatbot conversations became evidence in serious investigations of violent threats (the Tumbler Ridge case in February 2026, the Strasbourg case in April 2026), and will argue that Meta’s move forecloses provider-initiated referrals of the kind OpenAI made — and, in the Tumbler Ridge case, controversially failed to make. Meta’s response will be that safety is still provided, at the model and in-enclave levels rather than through provider-side content monitoring, and that the trade-off is a substantive policy choice the company has now made. The debate will not settle quickly, but it will have the analytical advantage of being conducted on the basis of an actual architecture rather than a hypothetical one.

Training without access

The architecture rules out one of the standard mechanisms by which consumer AI improves: collection of interaction data for training. Meta’s whitepaper is explicit that data processed inside the TEE does not train future models.

The authors’ best guess is that Meta will rely on three mechanisms, none of which depends on such access. Alternative training data — public web sources, Facebook and Instagram content where licensing permits, licensing arrangements with publishers, and synthetic data generated by other models. User-initiated feedback — when a user reports an inaccurate or harmful answer, the user effectively breaks the encryption boundary for that specific exchange, the same mechanism that already operates for reporting ordinary WhatsApp messages. And, more speculatively, federated learning or other privacy-preserving training architectures, which the public documentation does not address but which could be added without breaking the architectural commitment.

The trade-offs are real: safety calibration, edge-case detection, and prompt-pattern analysis all become harder when the data is not directly accessible. The authors frame the broader question as normative rather than technical — it is far from clear, they argue, that the default should be that the provider has access to the content and nuance of every prompt to, and response from, a chatbot.

The price the user pays: no conversational history

Statelessness means the provider has nothing to remember from one session to the next. The price of the confidentiality guarantee is the loss of conversational history in the server’s records.

This is not a marginal limitation. Much of the value users find in current chatbots — in health, legal preparation, financial planning, and ongoing intimate-emotional engagement — depends on persistent memory of earlier exchanges. The memory features OpenAI, Anthropic, Google and others have deployed over the past year exist precisely to give the chatbot access to prior context. Part 1 identified persistent memory as the fifth and most significant emerging boundary of chatbot confidentiality, because it transforms one-off conversations into longitudinal profiles. Users would presumably be able to save sessions on their own devices, leaving room for device-side memory, but the architecture supports no such profiles on Meta’s servers.

The paper’s sharpest comparison is with the temporary or ephemeral chat modes several standalone providers already offer — ChatGPT, Claude and others. The user-facing constraint is similar, but the confidentiality posture is fundamentally different. In existing ephemeral modes the conversation is not retained in the user’s history, but the provider still receives it in plaintext, can apply content-based safety moderation, can retain it for a defined safety-monitoring window (OpenAI’s temporary chats: currently up to thirty days before deletion), and remains exposed to compelled disclosure of those logs while they exist. Incognito Chat closes all of these exposures: the provider does not receive plaintext at any point, cannot retain what it cannot read, and cannot be compelled to produce what does not exist on its servers.

Two consequences follow. The architecture creates an explicit trade-off between confidentiality and continuity — users wanting a memory-augmented experience will use a non-encrypted pathway or another provider, users wanting strong confidentiality will accept the lack of memory. The authors regard this as a useful choice architecture, and the one Part 1 argued the industry needed: explicit and visible to the user. Second, whether the architecture can be extended to support forms of confidential memory without giving the provider access is an open technical question the authors flag but do not develop.

Conclusions

The authors conclude that 13 May 2026 marks the moment the architectural turn from policy-based to architecture-based confidentiality in consumer AI ceased to be a research aspiration and became a deployed product — in a mass-market platform with billions of users, externally audited and publicly documented. The pattern Part 1 described as Sealed Mode, that Apple’s Private Cloud Compute had instantiated since June 2024, and that Private Processing had deployed in more limited form since April 2025, is now available for the most architecturally exposed kind of interaction on WhatsApp: the chat conversation itself.

The question they deliberately leave open, rather than resolve, is whether and in what ways the other major providers — OpenAI, Anthropic, Google, xAI and others — will follow. The technical building blocks do not appear to be the obstacle: Apple’s Private Cloud Compute and Meta’s Private Processing have between them demonstrated that the engineering is tractable at consumer scale. What remains unknown is which existing features could migrate to a TEE-protected pathway without disrupting the rest of the product, and whether providers will choose to do so.

The paper highlights a significant step toward architectural protections for user confidentiality in chatbot conversations, alongside multiple unresolved tensions among policy goals, and the authors signal their intention to return to these questions in subsequent work.

RESOURCES

• The Day the Provider Stopped Reading Your Chats — the paper itself, AI-Regulation Papers 2026-05-4

• Cyberspace Law eJournal — SSRN LSN journal sponsored by the Cyber and Data Security Lab, Vrije Universiteit Brussel

• Theodore Christakis — Chair on the Legal and Regulatory Implications of AI, University Grenoble Alpes

• Peter Swire — J.Z. Liang Chair, Georgia Tech School of Cybersecurity and Privacy

• You Trust Your Chatbot With Everything. Should You? Part 1 — where the Sealed Mode proposal originates, March 2026

• Part 2: Governments, Courts and the Battle Over Your Chatbot Conversations — published 8 June 2026

• Introducing Incognito Chat with Meta AI — WhatsApp Blog, 13 May 2026

• Private Processing for WhatsApp: Technical Whitepaper — Meta AI

• Private Cloud Compute: A New Frontier for AI Privacy in the Cloud — Apple Security Research, June 2024, the architectural precedent

• What we learned about TEE security from auditing WhatsApp’s Private Inference — Trail of Bits, 7 April 2026

• Public Report: Meta WhatsApp Message Summarization Service — NCC Group, August 2025