AI Feedback Analysis and Privacy: What PII Rules Apply
TL;DR
AI feedback analysis privacy comes down to three facts most vendor pages skip. Customer feedback is personal data whenever it can be tied to a person, and free text almost always can be. An AI tool that reads it adds a processor, a sub-processor, and often an international transfer. And stripping emails is pseudonymisation, not anonymisation.
Each of those has a rule behind it. Removing identifiers leaves data that is still personal under GDPR Article 4(5) and Recital 26. Handing text to an AI feedback analysis tool turns "a human skims the inbox" into a multi-party data flow, and sometimes triggers a mandatory impact assessment. "We anonymize your data" is the most abused sentence in the category.
Full disclosure: I am the founder of FeedSense, an AI feedback analysis tool, so I have skin in this game -- and I will be plain about our limitations. This is compliance education, not legal advice; if you process feedback at scale, have a privacy professional review your specific data flow.
Why Does Privacy Get Harder the Moment AI Reads Your Feedback?
Because an AI feedback analysis tool reads every message in full -- including the parts a human skimmer would have skipped -- and it usually sends that text to at least one more company to do it. A support inbox that one person glanced at becomes a multi-party data flow, and every party in that flow needs a contract, a purpose, and a deletion date.
The people building these tools know it. In Cisco's 2025 Data Privacy Benchmark Study of 2,600 privacy and security professionals across 12 countries, 64% worried about inadvertently sharing sensitive information through generative AI, and nearly half admitted to entering personal employee or non-public data into GenAI tools (Cisco, April 2025). The cost is measured, too: IBM's 2025 Cost of a Data Breach Report found that one in five organizations had a breach linked to shadow AI, that organizations with high levels of shadow AI paid an average of $670,000 more per breach than those with little or none, and that 63% of breached organizations had no AI governance policy or were still writing one (IBM, July 2025). Personally identifiable information was compromised in 53% of breaches overall, rising to 65% when shadow AI was involved.
Customers have noticed. The Stanford AI Index reports that global confidence that AI companies protect personal data fell from 50% in 2023 to 47% in 2024 (Stanford HAI, 2025), and the IAPP found that 57% of consumers worldwide see AI used to collect and process personal data as a significant threat to their privacy (IAPP, 2023). So when a security reviewer or a customer's procurement team asks what happens to feedback text once it hits an AI tool, they are asking the question a year before it gets expensive. If you are new to the category, our complete guide to feedback analysis covers what the analysis does; this post covers what it owes.
What Counts as PII in Customer Feedback?
Under the GDPR, anything in a feedback item that relates to an identified or identifiable person is personal data -- "any information relating to an identified or identifiable natural person", identifiable directly or indirectly, per Article 4(1). Under the CCPA, personal information is anything that "identifies, relates to, describes, is reasonably capable of being associated with, or could reasonably be linked" to a consumer or household, explicitly including inferences drawn to build a profile (Civil Code §1798.140(v)(1)). Neither definition cares whether the information sits in a database column or in the middle of a sentence -- and free text is where the risk lives, because you do not control what people type. Four layers hide inside a typical feedback item:
| Layer | What it looks like in a ticket or survey comment | GDPR status | CCPA status |
|---|---|---|---|
| Direct identifiers | Names, emails, phone numbers, usernames, account and order IDs, invoice numbers, IP and device IDs in metadata | Personal data (Art. 4(1)) | Personal information (§1798.140(v)) |
| Indirect identifiers | "The only clinic in Tromsø on your scheduling plan"; job title plus company; a timestamp plus a location; a bug only one account could hit | Personal data if identification is reasonably likely (Recital 26) | "Reasonably capable of being associated" with a consumer |
| Special category / sensitive | "Since my chemo started I can't use the app"; religion, union membership, ethnicity, sexual orientation; under the CCPA, also an account log-in or financial account number sent with its password | Prohibited unless an Art. 9(2) exception applies (Art. 9); account credentials are ordinary personal data under the GDPR, not special category | "Sensitive personal information" (§1798.140(ae)), which includes account log-in and financial account details combined with access credentials |
| Derived labels | Sentiment score, churn-risk flag, "angry customer" tag, theme assignments attached to a user | Personal data, and profiling if it evaluates the person (Art. 4(4)) | Inferences used to profile a consumer are personal information |
Two of those layers surprise people. Indirect identifiers are why "we don't store names" settles nothing: Latanya Sweeney showed in 2000 that 87% of the US population was uniquely identified by just ZIP code, gender, and date of birth (Sweeney, 2000). Three innocuous attributes were enough then; a support ticket routinely carries more. And derived labels mean the AI's output is personal data as soon as it is attached to an identifiable user: the churn-risk flag your tool generates is a new piece of PII you created.
Paul M. Schwartz (Jefferson E. Peyser Professor of Law, UC Berkeley School of Law) and Daniel J. Solove (Eugene L. and Barbara A. Bernard Professor of Intellectual Property and Technology Law, George Washington University Law School), in their widely cited NYU Law Review article on the topic, put the definitional problem bluntly: "Personally identifiable information (PII) is one of the most central concepts in information privacy regulation. ... At the same time, there is no uniform definition of PII in information privacy law." The practical consequence for feedback teams: assume free text contains PII until you have checked, not the other way around.
Why Is Special-Category Data the Trap in Support Tickets?
Because you do not choose what customers write, and the law treats some of what they write as radioactive. GDPR Article 9(1) prohibits processing data "revealing racial or ethnic origin, political opinions, religious or philosophical beliefs, or trade union membership, and the processing of genetic data, biometric data ... data concerning health or data concerning a natural person's sex life or sexual orientation" unless one of the ten Article 9(2) exceptions applies. None of them is "the customer mentioned it in a bug report."
A user explaining that a screen reader breaks after your update has told you about a disability. A subscriber cancelling "because of my treatment schedule" has disclosed health data. A human agent reads that, handles the ticket, and forgets it. An AI feedback analysis tool stores it, embeds it, clusters it, and possibly ships it to a model provider -- the same sentence, now processed at scale by several parties. And scale is the second half of the trap: Article 35(3)(b) makes a data protection impact assessment mandatory for "processing on a large scale of special categories of data referred to in Article 9(1)". You planned to analyze feedback; the law looks at what actually flows.
Three defenses work in combination. Do not invite it: keep free-text prompts about the product, not the person. Redact before analysis, so the model sees "since my [health] started" rather than the diagnosis. And never build segments or triggers on special-category signals, however useful "users mentioning accessibility" looks as a filter. Most vendors' data processing agreements, ours included, ask you not to send special-category data at all -- which is contractually tidy and operationally useless unless you actually redact. Support-heavy teams should read this alongside our guide to how CS teams triage feedback, because triage is where redaction has to live.
What Changes Legally When AI Processes Feedback Instead of a Human?
The law does not change; the number of obligations you trigger does. Reading a ticket yourself is processing you already do, under a lawful basis you already have. Feeding it to an AI tool adds new parties, a new purpose, and possibly an impact assessment. Here is the delta, obligation by obligation.
| Obligation | Human reads the inbox | AI feedback analysis tool reads it |
|---|---|---|
| Lawful basis (Art. 6) | Contract or legitimate interest for handling the request | The same basis must cover analysis; a compatibility test applies if analysis is a new purpose (Art. 6(4)) |
| Purpose limitation (Art. 5(1)(b)) | Purpose is "resolve the ticket" | "Product analytics" and "churn prediction" are additional purposes, specified up front |
| Recipients (Art. 28) | None outside your company | Vendor is a processor; its model provider is typically a sub-processor requiring notice and an objection right (Art. 28(2)) |
| Transfers (Chapter V) | Data stays where you are | Every hop to a non-EU/UK provider needs an adequacy decision, SCCs, or another Art. 46 safeguard |
| Transparency (Art. 13) | Standard privacy notice | Notice must name purposes, recipient categories, and any third-country transfer (Art. 13(1)(c), (e), (f)) |
| DPIA (Art. 35) | Rarely triggered | Triggered by processing "in particular using new technologies" that is likely high-risk (Art. 35(1)): systematic evaluation of people via automated processing, or large-scale special-category data (Art. 35(3)(a)-(b)). Theme extraction over pseudonymised text is a far lighter assessment than per-user churn scoring on named accounts |
| Automated decisions (Art. 22) | Not applicable | Applies if outputs alone produce decisions with legal or similarly significant effects on a person |
Do You Have a Lawful Basis for the Analysis Itself?
Usually yes, but not automatically. Handling a support request is covered by the contract with the user (Article 6(1)(b)); analyzing thousands of requests to decide what to build is a different purpose, and most teams rest it on legitimate interests (Article 6(1)(f)), which holds only where the interest is not "overridden by the interests or fundamental rights and freedoms of the data subject". When the new purpose was not specified at collection, Article 6(4) makes you test compatibility -- the link between purposes, the context, the nature of the data, the consequences for the person, and the safeguards in place -- and it names "encryption or pseudonymisation" as exactly the kind of safeguard that helps.
"We'll analyze it later" is where this fails: feedback collected for support, then repurposed into an analytics or training pipeline without notice, is the pattern regulators keep punishing. The EDPB's Opinion 28/2024 on AI models (December 2024) set out a three-step test for relying on legitimate interest in AI development and deployment, and confirmed that a model built on unlawfully processed personal data can taint its later use; France's CNIL had already published its first recommendations on developing AI systems under the GDPR in June 2024. EDPB Chair Anu Talus framed the intent: "We need to ensure these innovations are done ethically, safely, and in a way that benefits everyone." Then-FTC Chair Lina Khan said the same in fewer words when the US agencies released their 2023 joint statement on AI: "There is no AI exemption to the laws on the books, and the FTC will vigorously enforce the law to combat unfair or deceptive practices or unfair methods of competition."
Who Else Touches the Data Once a Vendor Is Involved?
Your vendor, and whoever your vendor uses. Article 28(3) requires a written contract under which the processor acts "only on documented instructions from the controller", keeps staff under confidentiality, implements Article 32 security, flows the same terms down to sub-processors, helps you answer data subject requests, deletes or returns the data when the service ends, and submits to audits. Article 28(2) adds that the processor must inform you of intended sub-processor changes so you can object.
For AI feedback analysis, the sub-processor that matters is the model provider, because it sees raw text. Ask for its retention and training terms, not just the vendor's. The major providers publish them: Anthropic states that "by default, we will not use your inputs or outputs from our commercial products (e.g. Claude for Work, Anthropic API ...) to train our models", and Google's Gemini API terms say that for paid services "Google doesn't use your prompts ... or responses to improve our products" while unpaid services are used for improvement. That paid-versus-free split is the detail to check for every provider in the chain: the same model can sit under two different data policies.
Does the Feedback Leave the EU or UK?
If any processor or sub-processor runs outside the EU/EEA or UK, yes, and Chapter V applies: a transfer to a third country "shall take place only if ... the conditions laid down in this Chapter are complied with" (Article 44), via an adequacy decision (Article 45) or appropriate safeguards such as standard contractual clauses (Article 46).
For US providers, the EU-US Data Privacy Framework adequacy decision of 10 July 2023 is the usual route. It survived its first court test when the General Court dismissed Latombe v Commission (T-553/23) on 3 September 2025, but an appeal to the Court of Justice (C-703/25 P) was lodged on 31 October 2025 and was still pending as of this writing. Keep SCCs ready in case the framework falls; the Irish DPC fined Meta €1.2 billion in May 2023 for continuing transfers under Article 46(1) clauses that no longer sufficed, the largest GDPR fine on record.
Are You Making Decisions About People?
Feedback analysis rarely does, but it can drift there. Article 22(1) gives every person "the right not to be subject to a decision based solely on automated processing, including profiling" that produces legal or similarly significant effects. Clustering tickets into themes and ranking them for your roadmap has no such effect on any individual. Automatically downgrading, pricing, or cancelling a specific user because a model flagged their tone would.
California now has its own rule. The CPPA's regulations on automated decisionmaking technology were approved on 23 September 2025 and took effect on 1 January 2026, with ADMT obligations applying from 1 January 2027. They define ADMT as any technology that processes personal information and "uses computation to replace human decisionmaking or substantially replace human decisionmaking" (11 CCR §7001(e)), and a "significant decision" as one that results in the provision or denial of "financial or lending services, housing, education enrollment or opportunities, employment or independent contracting opportunities or compensation, or healthcare services" -- advertising is expressly excluded (§7001(ddd)). Keep a human between the model's output and any decision about a named customer, and document that step.
Does the EU AI Act Apply to Sentiment or Emotion Analysis of Feedback?
Mostly no, with one obligation that is already live. The AI Act defines an "emotion recognition system" as "an AI system for the purpose of identifying or inferring emotions or intentions of natural persons on the basis of their biometric data" (Article 3(39)) -- faces, voices, gait. Text in a ticket is not biometric data under Article 3(34), so sentiment or emotion analysis of written feedback falls outside that definition, and outside the workplace and education prohibition in Article 5(1)(f) built on it. Annex III's high-risk categories cover things like recruitment and creditworthiness, not product feedback triage, and their deadlines moved to 2 December 2027 (Annex III) and 2 August 2028 (Annex I) under the Digital Omnibus on AI, Regulation (EU) 2026/1744, in force since 27 July 2026.
What did apply on schedule on 2 August 2026 is Article 50 transparency: providers must ensure people interacting with an AI system "are informed that they are interacting with an AI system". If your feedback flow includes an AI that talks back to users -- an auto-reply, a bot that acknowledges tickets -- label it. For how sentiment and emotion models work and where they fail, see our guides to AI sentiment analysis for customer feedback and emotion analysis of customer feedback.
Is Stripping Emails Anonymization?
No. Removing names and email addresses is pseudonymisation at best, and pseudonymised data is still personal data. GDPR Article 4(5) defines pseudonymisation as processing so that data "can no longer be attributed to a specific data subject without the use of additional information" that is kept separately, and Recital 26 says such data "should be considered to be information on an identifiable natural person". The EDPB's Guidelines 01/2025 on pseudonymisation (January 2025) restate it without hedging: pseudonymised data "remains information related to an identifiable natural person and is therefore still personal data."
Anonymisation is a much higher bar. The Article 29 Working Party's Opinion 05/2014 on anonymisation techniques set three tests that still govern -- singling out (can a record be isolated to one person?), linkability (can two records about the same person be joined?), and inference (can an attribute be deduced with significant probability?) -- and stated flatly that "pseudonymisation is not a method of anonymisation." Twelve years later the EDPB adopted the draft that will replace it, Guidelines 02/2026 on Anonymisation (version 1.0 adopted 7 July 2026 for public consultation until 30 October 2026), which keeps the same three tests under new names -- no record isolation, no linkage, no inference -- and warns that inferences drawn by prompting AI models can fail the third. Anu Talus, the EDPB Chair, called the guidelines "a significant milestone in clarifying the notion of anonymous data, establishing clear standards that facilitate the use of data while protecting individuals' fundamental rights."
Run "we strip emails" through those tests and it fails on the first one. The free text still says "the only clinic in Tromsø," still carries the order number, still describes a bug that one account triggered. Luc Rocher, Julien Hendrickx and Yves-Alexandre de Montjoye showed in Nature Communications that 99.98% of Americans would be correctly re-identified in any dataset using 15 demographic attributes (2019), even in datasets their sellers called incomplete. As Yves-Alexandre de Montjoye, professor of applied mathematics and computer science at Imperial College London, put it: "Companies and governments downplay the risk of re-identification by arguing that the datasets they sell are always incomplete. Our findings show this might not help."
There is one genuine nuance, and vendors will quote it at you. In EDPS v SRB (C-413/23 P, 4 September 2025), the Court of Justice held that pseudonymised data sent to a recipient may not be personal data for that recipient if it has no reasonably likely means of re-identifying anyone, while it remains personal data for the controller holding the key. The UK ICO's March 2025 anonymisation guidance takes a similar relative view with its "motivated intruder" test, and the EDPB's 2026 draft takes the same view under the label "contextual approach".
The catch: a recipient that receives raw ticket text has the means of identification in the text. The nuance helps only if what leaves your systems has actually been scrubbed of direct and indirect identifiers, and you documented the assessment, because the burden of proof sits with whoever claims the data is anonymous. The same logic reaches the models: the EDPB's Opinion 28/2024 concluded that an AI model trained on personal data is not automatically anonymous, and "the model doesn't store your data" is a claim to be evidenced, not accepted.
| Technique | What it does | Legal status (GDPR) | Protects against | Where it fails |
|---|---|---|---|---|
| Redaction | Removes or masks specific identifiers in the text (emails, phone numbers, names) | Still personal data if the person remains identifiable from what is left | Casual exposure; direct-identifier leakage to sub-processors | Indirect identifiers in free text; derived labels still linked to an account |
| Pseudonymisation | Replaces identifiers with tokens; the mapping key is held separately (Art. 4(5)) | Personal data for whoever can access the key; a recognised security measure (Art. 32(1)(a)) | Attribution without the key; limits what a processor learns | Anyone with the key, or with enough context to re-link, defeats it |
| Anonymisation | Transforms data so no one, using means reasonably likely to be used, can single out, link, or infer a person (Recital 26; WP29 05/2014) | Outside the GDPR, once demonstrated and documented | Everything above, if genuinely irreversible | Rarely achievable for free text without destroying the meaning you wanted to analyze |
The honest position for feedback text: aim for pseudonymisation done properly and say so. Claiming anonymisation you cannot demonstrate is worse than admitting you process personal data carefully. For the engineering side, ENISA's report on pseudonymisation techniques and best practices (2019) remains the reference.
Should You Redact at Collection or at Analysis, and How Long Should You Keep It?
Both, in that order, with a deletion clock on the raw text. Article 5(1)(c) limits data to what is "adequate, relevant and limited to what is necessary", Article 25(2) requires that "by default, only personal data which are necessary for each specific purpose of the processing are processed", and Article 5(1)(e) says identifiable data is kept "for no longer than is necessary for the purposes". Read together, they describe a pipeline.
At collection, stop inviting identifiers. A widget that asks "tell us what went wrong" collects less PII than one that asks "tell us about yourself." You need some link back to the user, because closing the loop is the point of feedback, but it can be a pseudonymous key held in your system of record rather than an email carried inside the text.
At analysis, redact again before the text reaches a model: detecting emails, phone numbers, card numbers, and names in free text is a solved engineering problem, and the sub-processor should receive the least identifying version of the text that still supports the analysis.
On retention, separate raw text from derived insight. Aggregated themes and trends can outlive the tickets they came from; the raw sentences and the per-user labels should not. Pick a number of days, write it into the DPA, and make sure the vendor's deletion covers its sub-processors -- Article 28(3) requires the processor to delete or return the data when the service ends, and Article 17 gives every user a right to erasure "without undue delay" that has to propagate through that chain.
Two clocks decide whether you can actually honour this. Access and deletion requests must be answered "without undue delay and in any event within one month" under GDPR Article 12(3), and within 45 days, extendable once by another 45, under CCPA §1798.130. A breach must reach the supervisory authority "not later than 72 hours" after you become aware of it (Article 33(1)), and Article 33(2) obliges your processor to tell you "without undue delay." Those windows are impossible to meet if you cannot list every place a given user's feedback text lives.
The enforcement climate makes this less theoretical every year. DLA Piper's January 2026 survey counted about €1.2 billion in GDPR fines during 2025 and €7.1 billion since May 2018, with breach notifications up 22% to an average of 443 per day (DLA Piper, 2026). Maximum penalties remain €20 million or 4% of worldwide annual turnover for breaches of the core principles, lawful-basis, data-subject-rights, and transfer rules (Article 83(5)). AI-specific enforcement is live even where it stumbles: Italy's Garante announced a €15 million fine against OpenAI in December 2024 over the legal basis for training, transparency, age checks, and breach notification; the Court of Rome annulled the fine on 18 March 2026 on one-stop-shop jurisdiction grounds without ruling on the substance, which moved the questions to the Irish DPC rather than answering them.
What Should You Ask an AI Feedback Analysis Vendor?
Ten questions, each with the answer you want to hear and the answer that should worry you. Send them before the demo, not after the contract; a vendor that has done the work answers in a day.
| # | Question | Answer you want | Answer that should worry you |
|---|---|---|---|
| 1 | Is our feedback used to train your models, or anyone else's? | "No, contractually. Here is the DPA clause and the model provider's policy." | "We use aggregated data to improve our AI." |
| 2 | Which sub-processors see raw feedback text, and where do they run? | A public list with region and purpose, plus an objection right on changes (Art. 28(2)) | "We use industry-standard providers." |
| 3 | What is the model provider's retention window for our text? | A number and a link to the provider's policy (zero retention or N days) | "We'd have to check." |
| 4 | Will you sign a DPA with Article 28(3) terms, and is it public? | Yes, self-serve, published | "DPAs are available on enterprise plans." |
| 5 | Where is data stored and processed, and what is the transfer mechanism? | Named regions; adequacy decision or SCCs named per hop | "US cloud." |
| 6 | What happens to raw text on deletion or account closure, and when? | Deletion including sub-processors, within a stated number of days | "Backups are retained indefinitely." |
| 7 | Can we pseudonymise or redact before analysis, and does the tool still work? | Yes; here is how, and here is what changes in accuracy | "The AI needs the full context." |
| 8 | Does any output drive automated decisions about individual users? | "Advisory only. Humans decide." | Auto-actions on named users with no human step |
| 9 | What happens when special-category data arrives in a ticket? | A stated policy: blocking, redaction, or an explicit instruction not to send it | Silence, or "our AI handles everything." |
| 10 | What is your breach-notification commitment to us, and what do you publish in place of certifications you lack? | An hours-based SLA in the DPA (Art. 33(2)) and an honest security page | "We follow all applicable laws." |
If you are comparing tools on more than privacy, our comparison of AI feedback analysis tools covers pricing, features, and limitations; use these ten questions as the shortlist filter first.
What Are the Red Flags in an AI Feedback Tool's Privacy Story?
The pattern behind every red flag is the same: a comforting word standing in for a document. Six that come up most:
- "Anonymised" meaning stripped names and emails. That is pseudonymisation (WP29 05/2014; EDPB 01/2025). Ask for the singling-out, linkability, and inference assessment; expect none to exist.
- No public sub-processor list, or "confidential for security reasons." Without it you cannot map transfers or use the Article 28(2) objection right. Make disclosure a condition of signing.
- Training on customer data by default. Your users never agreed, and the FTC has ordered companies to delete models and algorithms built on improperly used data (Everalbum, 2021). Require a contractual no-training clause, not a settings toggle.
- No retention numbers anywhere. Storage limitation (Art. 5(1)(e)) and erasure (Art. 17) cannot be honoured on "as long as needed". Put days in the DPA.
- "Our AI automatically decides which users to ..." Article 22 and California's ADMT rules attach to decisions about individuals. Keep a human step and document it.
- A free tier with vaguer data terms than the paid tier. Google's own API terms treat unpaid and paid services differently. Read the terms for the tier you will actually use, and the terms of the providers underneath.
Where Does FeedSense Stand on Feedback Privacy?
I am the founder of FeedSense, so read this section as a vendor answering its own questions. FeedSense is an AI feedback analysis tool: it reads feedback from the channels you already have and hands you a prioritized brief, which makes it a processor on your behalf, and everything above applies to us.
At the outcome level, here is what we commit to and publish:
- No training on your data. FeedSense does not use customer data to train any model, and we rely on the model provider's published no-retention posture for analysis traffic -- stated in our privacy policy.
- Public paperwork. A sub-processor list with a 30-day change notice and objection right, a data processing agreement, a security overview, and the privacy policy. You do not need an enterprise plan to read or sign any of it.
- Published retention and rights timelines. Account data is deleted within 30 days of account closure; access and deletion requests are answered within 30 days under GDPR and KVKK (Turkey's data protection law) and within 45 days under CCPA. Data export and deletion are self-serve.
- Advisory output. The AI surfaces themes, classifications, and recommendations; it does not make decisions with legal or similarly significant effects on your users. Your team decides what ships.
And the limitations, because a privacy page without them is not worth much:
- We do not hold SOC 2 or ISO 27001 certifications, and we have not yet commissioned a third-party penetration test. Our DPA says so in as many words; we do not claim compliance we have not achieved. Weigh a small vendor without third-party audits accordingly.
- Special-category data. We do not knowingly process it, and our DPA asks you not to send it -- the industry-standard clause, with the industry-standard weakness that it only works if you redact before the ticket reaches us.
- We are one processor in your chain, not the whole answer. Your lawful basis, your privacy notice, and your DPIA are yours to write. We can give you documents to reference; we cannot do the assessment for you.
If those trade-offs work for you, start on the free plan and read the four legal pages before you connect a channel. If you need certified infrastructure today, say so in the vendor questions above and shortlist accordingly.
Frequently Asked Questions About AI Feedback Analysis and Privacy
Is customer feedback personal data under GDPR?
Yes, whenever it relates to an identified or identifiable person, which is almost always for support tickets and often for survey comments. GDPR Article 4(1) covers information relating to a person directly or indirectly, so a ticket that names nobody but describes one account's bug is still personal data if the account can reasonably be identified.
Does GDPR apply if we only use AI to summarize feedback?
Yes. Summarizing is processing, and the AI tool becomes a processor under Article 28 the moment it receives text about identifiable people. You need a lawful basis that covers the analysis purpose, a written DPA with the vendor, a privacy notice that mentions the purpose and recipient categories, and a transfer mechanism if the vendor or its model provider is outside the EU/UK.
Is removing names and email addresses enough to anonymize feedback?
No. That is pseudonymisation, and pseudonymised data remains personal data under Article 4(5) and Recital 26. Anonymisation requires that nobody, using means reasonably likely to be used, can single out, link, or infer a person from the data -- the three tests in Article 29 Working Party Opinion 05/2014 and the EDPB's July 2026 draft guidelines. Free text rarely passes.
Do we need a DPIA before using AI on customer feedback?
Often, yes. Article 35(3) requires one for systematic and extensive evaluation of people based on automated processing, and for large-scale processing of special-category data such as health information. Theme extraction on pseudonymised text is lower risk than per-user churn scoring on named accounts, but the DPIA is where you establish which one you are doing and what safeguards apply.
Can we rely on legitimate interest to analyze feedback with AI?
Usually, for product improvement, if you document the balancing test and the analysis would not surprise a reasonable user. Article 6(1)(f) requires that the interest not be overridden by the individual's rights, and Article 6(4) requires compatibility with the original collection purpose. The EDPB's Opinion 28/2024 sets out a three-step test for AI use cases; special-category data cannot rest on legitimate interest alone.
Does the EU AI Act ban sentiment or emotion analysis of customer feedback?
No. The AI Act's definition of an emotion recognition system (Article 3(39)) is limited to inferences from biometric data such as faces and voices, so analyzing written feedback is outside it and outside the workplace and education prohibition in Article 5(1)(f). What does apply since 2 August 2026 is Article 50: users interacting with an AI system, such as an auto-reply bot, must be told it is AI.
Is an AI vendor's model provider a sub-processor?
Yes, if it receives your feedback text to perform the analysis. Under Article 28(2) the vendor must inform you of sub-processors and changes to them so you can object, and Article 28 requires the same contractual protections to flow down. Ask specifically for the model provider's retention window and training policy, because those terms, not the vendor's marketing page, govern what happens to raw text.
How long can we keep raw feedback text?
As long as necessary for the stated purpose and no longer, per Article 5(1)(e), which means you must pick a number and defend it. A workable pattern is a fixed retention period for raw text and per-user labels, with aggregated themes and trends kept longer because they no longer identify anyone. Write the number into your DPA and confirm the vendor's deletion reaches its sub-processors.
What does CCPA require when AI analyzes customer feedback?
CCPA treats feedback as personal information when it is reasonably linkable to a consumer or household, and it counts inferences drawn to profile someone as personal information too (Section 1798.140(v)(1)). You must honour access and deletion requests within 45 days, extendable once, and from 1 January 2027 California's ADMT regulations add notice and opt-out duties where automated technology makes significant decisions about people.
Sources
- GDPR Article 4 - Definitions (personal data, profiling, pseudonymisation)
- GDPR Recital 26 - Not applicable to anonymous data
- GDPR Article 5 - Principles relating to processing of personal data
- GDPR Article 6 - Lawfulness of processing
- GDPR Article 9 - Processing of special categories of personal data
- GDPR Article 12 - Transparent information and modalities (one-month response)
- GDPR Article 13 - Information to be provided at collection
- GDPR Article 17 - Right to erasure
- GDPR Article 22 - Automated individual decision-making
- GDPR Article 25 - Data protection by design and by default
- GDPR Article 28 - Processor
- GDPR Article 32 - Security of processing
- GDPR Article 33 - Notification of a personal data breach
- GDPR Article 35 - Data protection impact assessment
- GDPR Article 44 - General principle for transfers (Chapter V)
- GDPR Article 83 - Administrative fines
- Article 29 Working Party - Opinion 05/2014 on Anonymisation Techniques (WP216)
- EDPB - Opinion 28/2024 on AI models (press release, December 2024)
- EDPB - Guidelines 01/2025 on Pseudonymisation (news, January 2025)
- EDPB - Guidelines 02/2026 on Anonymisation, version 1.0 for public consultation (news, 8 July 2026)
- Court of Justice of the EU - Press Release 107/25, EDPS v SRB (C-413/23 P), 4 September 2025
- ICO - Anonymisation guidance (28 March 2025; under review following the Data (Use and Access) Act 2025)
- CNIL - First recommendations on the development of AI systems (June 2024)
- California Civil Code §1798.140 - CCPA definitions
- California Civil Code §1798.130 - 45-day response requirement
- CPPA - CCPA updates: ADMT, risk assessment, and cybersecurity audit regulations
- Alston & Bird - California finalizes new and amended CCPA regulations (September 2025)
- CPPA - Approved text of the ADMT, risk assessment and cybersecurity audit regulations (OAL approval 23 September 2025)
- EU AI Act - Article 3 (definitions), Article 5 (prohibited practices), Article 50 (transparency), Annex III (high-risk systems)
- EU AI Act - Implementation timeline
- Cloud Security Alliance - EU AI Act high-risk deadline deferred by Regulation (EU) 2026/1744 (2026)
- European Commission - EU-US Data Privacy Framework adequacy decision (10 July 2023)
- IAPP - General Court dismisses Latombe challenge, upholds EU-US DPF (3 September 2025)
- WilmerHale - Court of Justice to review Latombe appeal (December 2025)
- Irish Data Protection Commission - Conclusion of inquiry into Meta Ireland, €1.2 billion fine (22 May 2023)
- Garante - €15 million fine on OpenAI (20 December 2024)
- PPC Land - Court of Rome annuls OpenAI fine on jurisdiction grounds (2026)
- FTC - Joint statement on AI enforcement, Chair Khan quote (25 April 2023)
- FTC - Everalbum settlement requiring deletion of models and algorithms (11 January 2021)
- Cisco - 2025 Data Privacy Benchmark Study (April 2025)
- IBM - 2025 Cost of a Data Breach Report press release (30 July 2025)
- Stanford HAI - AI Index Report 2025, Public Opinion chapter
- IAPP - Privacy and Consumer Trust Report (March 2023)
- DLA Piper - GDPR Fines and Data Breach Survey: January 2026
- Sweeney, L. - Simple Demographics Often Identify People Uniquely (2000)
- Rocher, Hendrickx & de Montjoye - Estimating the success of re-identifications in incomplete datasets using generative models, Nature Communications (2019)
- Imperial College London - Anonymising personal data 'not enough to protect privacy' (July 2019)
- Schwartz & Solove - The PII Problem, NYU Law Review 86(6) (2011)
- ENISA - Pseudonymisation techniques and best practices (December 2019)
- Anthropic - Is my data used for model training?
- Google - Gemini API Additional Terms of Service (paid vs unpaid services)
Tags:
Stay in the loop
Get the latest insights on customer feedback, product management, and building better SaaS products. No spam, unsubscribe anytime.
By subscribing, you agree to our privacy policy. We respect your inbox.