Data privacy and AI: what to know before using customer data

Someone on your team has probably pasted a customer spreadsheet into an AI chatbot to save an hour. It worked, the output was great, and nobody asked whether they were allowed to. For SMB owners and operations leaders, that is the most common and most avoidable risk in AI adoption. The good news: privacy law does not forbid using AI with customer data. It asks for discipline.

There is no single US AI privacy law, and that is the catch

The United States regulates privacy through a patchwork. Depending on where your customers live and what you collect, you may face state laws such as the California Consumer Privacy Act (as amended by the CPRA), and comparable statutes in states like Virginia, Colorado and Texas. Sector rules add more: HIPAA for health information, GLBA for financial data, FERPA for education records, and COPPA for children under 13. The FTC also treats misleading privacy claims and unreasonable data security as unfair or deceptive practices, whether or not AI is involved.

If you sell to people in the EU or UK, GDPR can apply too. This is general information, not legal advice, so confirm which regimes cover you with counsel.

What these laws have in common

• Notice: tell people what you collect and how you use it. If AI processes their data, your privacy policy should not contradict that.
• Purpose and minimization: use only what the task needs, and do not quietly repurpose data collected for something else.
• Vendor accountability: when a vendor processes data for you, you typically need a contract that limits what they can do with it, often a data processing agreement.
• Security: reasonable safeguards, proportional to the sensitivity of the data.
• Individual rights: access, deletion, correction and opt-outs, including limits on certain automated decision-making in some states.

The AI-specific risks

• Consumer-grade tools: free tiers and personal accounts may retain prompts or use them to improve models. Business and API plans usually come with stronger commitments. Read the actual terms.
• Shadow AI: employees using unapproved tools is how most leaks happen, not through sophisticated attacks.
• Prompts and logs: personal data ends up stored in chat histories and application logs that need access controls and retention limits.
• Over-permissioned assistants: an AI connected to your CRM or ERP can surface records a given employee should never see.
• Wrong answers about real people: hallucinated claims about a customer can feed bad decisions, especially in hiring, credit, pricing or support priority.

A practical checklist before you start

1. Map the data: what personal data enters the AI workflow, where it comes from and where it goes.
2. Classify sensitivity: health, financial, children’s and biometric data need extra care or should stay out entirely at first.
3. Minimize and mask: replace names, emails and account numbers with IDs when the task does not need them.
4. Pick the right plan: choose business-tier or API access with written commitments on retention and no training on your data. Get it in the contract, not the marketing page.
5. Sign the paperwork: a data processing agreement, and a business associate agreement if HIPAA applies.
6. Respect permissions: the AI should see only what the requesting user may see.
7. Set retention: decide how long prompts, outputs and logs live and who can delete them.
8. Keep a human in the loop for decisions that materially affect individuals.
9. Update your privacy notice and make sure you can honor deletion and access requests across the AI tools you use.
10. Prepare for incidents: state breach notification laws have deadlines, so know who calls whom.

Start low-risk, then expand

Hypothetical example: a 40-person distributor begins with summarizing internal SOPs and drafting emails, then moves to analyzing support tickets after stripping names and account numbers, and only later connects the assistant to its CRM with role-based access. Each step has an owner, a written rule and a quarterly review. Nobody was blocked, and nothing sensitive went through an unvetted tool.

One more habit worth building: keep a simple register of every AI tool in use, what data it touches, who approved it and when the vendor terms were last reviewed. It takes an hour to start, and it turns a vague worry into something you can audit, explain to a customer and hand to your lawyer.

Privacy done well is a speed advantage. Customers and enterprise buyers increasingly ask how you handle their data in AI workflows, and having clear answers shortens sales cycles and security reviews.

Want to put this to work? Convertty’s senior team runs an AI assessment of your workflows and delivers an implementation plan with 30-day quick wins. Book a call.