Memorization is not a failure mode of neural networks — it is their fundamental operating mechanism. The universal approximation theorem guarantees that sufficiently large networks can represent any function, including the identity function on training data. Overparameterized models (more parameters than training examples) have the capacity to store every training example verbatim, and gradient descent naturally gravitates toward solutions that memorize distinctive patterns. PII — with its structured formats, repeated appearances across documents, and unique character sequences — is precisely the kind of data neural networks are architecturally predisposed to memorize. You cannot build a model that generalizes without memorizing, because generalization IS selective memorization. The boundary between the two is mathematically blurred.

The fundamental asymmetry: extracting information from a trained model requires only clever querying, while preventing extraction requires modifying the training process itself at enormous cost. Differential privacy (the only proven defense) degrades model quality by 5-20%. Output filtering (the most practical defense) can be bypassed through novel prompts. Model inversion requires only the model weights (freely distributed for open models). Membership inference requires only API access. The defender must anticipate and block every possible extraction technique simultaneously; the attacker needs only one to succeed. This asymmetry is structural, not circumstantial — it arises from the information-theoretic fact that a useful model must encode information about its training data, and any encoded information can in principle be extracted. Defense is inherently harder than attack in the same way that proving a system is secure is harder than finding one vulnerability.

Provenance opacity is not an accidental omission — it is a structural feature of the AI training ecosystem. Common Crawl does not track per-page PII content. The Pile does not inventory per-source personal data. No AI company publishes training data manifests because the data is too large to audit (petabytes), competitive advantage depends on data secrecy, disclosure would reveal legal vulnerabilities, and the data was not inventoried at collection time. This opacity propagates through model chains: if Model A's data is unknown and Model B trains on A's outputs, B's PII content is doubly unknown. Each generation adds another opacity layer. The result is an ecosystem where billions of people's data is embedded in systems whose operators cannot identify whose data they have, where it came from, or how to remove it.

Privacy law was built for a world of databases with rows and columns — where an individual's record can be located, inspected, modified, and deleted. AI training operates in a fundamentally different paradigm: trillions of tokens processed through gradient descent, distributing each data point's influence across billions of parameters. There is no 'row' to find, no 'record' to delete, no 'index' to search. The scale of modern training data (petabytes from billions of sources) makes individual-level operations — locate this person's data, determine how it influenced the model, remove that influence — not just expensive but architecturally incompatible with the technology. This is not a scaling problem that more compute can solve. It is a categorical mismatch between a legal framework designed for databases and a technology that is fundamentally not a database. Consent at internet scale is a logical impossibility, not an engineering challenge.

Embeddings are compressed representations of meaning — and identity IS meaning. A sentence about a specific person has a specific meaning that differs from the same sentence about a different person. The embedding must capture this difference to be useful, and capturing this difference IS encoding identity information. You cannot build an embedding that preserves semantic meaning while stripping identity, because identity contributes to meaning. 'The doctor prescribed medication' means something different when the doctor is identifiable versus anonymous, and the embedding must encode this difference to function. This entanglement between identity and semantics is not a design flaw — it is an information-theoretic consequence of what embeddings are. Removing identity information from embeddings requires removing the semantic distinctions that make the embeddings useful. The utility-privacy tradeoff in embedding space is not a tunable parameter; it is a conservation law.

Consent is a temporal act — it can only be given for uses that exist at the time of giving. The web content forming the foundation of every major LLM was created in a world where AI training did not exist as a concept. A person writing a blog post in 2008 could not have consented to GPT-4 training in 2023 because GPT-4 did not exist, large language models did not exist, and 'training data' was confined to academic ML research. Retroactive consent at the scale of billions of data subjects across decades of web content is not a difficult problem — it is a logical impossibility. You cannot consent to something that does not yet exist. This temporal gap between data creation and data use is structural and permanent: every future AI capability will create new uses for already-collected data, perpetually outrunning any consent obtained today. The consent frameworks in GDPR, CCPA, and other privacy laws assume a model where the purpose of processing is known at collection time. AI training destroys this assumption.

Accountability diffusion is a social-technical problem, not purely technical or purely legal. GDPR's controller-processor framework assumes a clear chain of responsibility: someone decides what data to process (controller) and someone executes that processing (processor). In the AI training pipeline, this clarity dissolves. Common Crawl operates autonomously, scraping the web without specific data processing instructions from AI companies. Dataset curators compile data without knowing which models will use it. Pre-training organizations use datasets they did not compile. Fine-tuners modify models they did not pre-train. Deployers serve models they did not fine-tune. At each step, the entity argues it is not the responsible controller — and each has a plausible argument. The result is that PII flows through the entire pipeline with no entity accepting comprehensive responsibility. When an individual seeks to exercise GDPR rights (access, erasure, objection), there is no single entity that can fulfill the request because no entity controls the full lifecycle.