The Case for an Agent-User Privilege
Ask a lawyer whether you’re liable and the question stays secret. Ask a chatbot and it can be read to a jury. Georgia should close that gap, with a statute built to answer its critics.
On New Year’s Day 2025, according to a federal criminal complaint, a man on a trail in the Santa Monica Mountains typed a question into ChatGPT during his calls to 911: “Are you at fault if a fire is lift [sic] because of your cigarettes.”1 Prosecutors say that fire kept burning underground and, a week later, became the Palisades Fire.1 The question went into the complaint. At trial in June 2026, prosecutors used his ChatGPT logs against him, the jury split 10–2 for the defense, and the judge declared a mistrial.2 His lawyer argued that he had not started the fire and had called the authorities when he saw it break out.2 Prosecutors are trying him again. A second jury was seated in early November 2026,3 and in mid-November the judge paused the retrial to look into the government’s claim that the defense had withheld material it was required to share.4
Set the arson case aside. Look only at the question. “Am I legally responsible for this?” is the most ordinary question a person can ask a lawyer. Had he asked it of a lawyer in confidence, it would ordinarily have been privileged. The privilege protects advice about past conduct. The crime-fraud exception takes it away when the advice sought concerns “future wrongdoing.”5 He asked a machine instead. So the question became evidence.
This post argues for a fix: a statutory agent-user privilege for people who consult an AI system about their legal problems. It should be modeled on the attorney-client privilege, held by the user, and drawn narrowly enough to answer its serious critics. We think Georgia should write one.
1. What the law says now
The first court to decide the question said no. In United States v. Heppner, a securities-fraud defendant who had already received a grand jury subpoena used Anthropic’s Claude, on his own, to prepare “reports that outlined defense strategy.”6 FBI agents seized the documents with a search warrant. Judge Jed Rakoff of the Southern District of New York held in February 2026 that neither the attorney-client privilege nor the work-product doctrine protected them.6
The court gave three reasons. First, Claude is not a lawyer, and “the discussion of legal issues between two non-attorneys is not protected by attorney-client privilege.”6 Second, the conversations were not confidential, because Anthropic’s privacy policy reserved the right to disclose data to “third parties,” including “governmental regulatory authorities.”6 Third, Claude itself disclaims giving legal advice. Asked by the government, it answered: “I’m not a lawyer and can’t provide formal legal advice or recommendations.”6 On work product, the documents were not prepared “by or at the behest of counsel.”6
Other courts have gone the other way, at least in civil cases. A federal magistrate judge in Michigan refused to order a self-represented employment plaintiff to produce her AI materials. He called ChatGPT and similar programs “tools, not persons, even if they may have administrators somewhere in the background.”7 A Colorado magistrate judge agreed that a self-represented plaintiff’s AI use could be work product. She asked a question that is hard to answer: “Does that mean that anyone with a Gmail account has forfeited all rights to confidentiality and privacy?”8 A New York trial court quashed subpoenas served on OpenAI for a self-represented defendant’s prompts and outputs.9 The Texas Business Court protected a party’s AI chat log under the state’s work-product rule, even though he “was not a lawyer or acting as his own counsel.”10
Read together, these decisions protect less than they seem to. Work product covers material prepared “in anticipation of litigation.” It does not cover the tenant who asks about a deposit before any lawsuit exists, or the person who wonders, at 2 a.m., whether something they did was a crime. It is qualified, so an opponent can overcome it with a showing of need.9 Even the Colorado court ordered the plaintiff to name the AI tools he had used with the other side’s confidential documents, and it barred him from putting those documents into “any mainstream AI tool like standard ChatGPT, Claude, Gemini, or similar platforms.”8
Then there is the government’s side door. In June 2026, a federal judge in Manhattan refused a criminal defendant’s request to block a warrant served on OpenAI for his records. Under the Stored Communications Act, she wrote, “The statute does not authorize a customer or subscriber to move to quash a warrant.”16 The user whose words are at stake may not even get a hearing before they are handed over.
Civil litigants can reach these conversations too, including the conversations of people who are not parties at all. In the copyright fight between news organizations and OpenAI, the court ordered production of a sample of 20 million de-identified ChatGPT logs. It reasoned that users’ privacy interests are weaker because they “voluntarily disclosed” their conversations to OpenAI, which “retains in the normal course of its business” the logs.17 Judge Rakoff cited that order in Heppner.6
So a person who asks an AI system about a legal problem today cannot know whether the conversation is protected. The answer depends on whether a lawsuit had started, whether the person had a lawyer, whether the case is civil or criminal, which court hears it, and which company’s privacy policy they clicked through. The Supreme Court has warned against exactly this: “An uncertain privilege, or one which purports to be certain but results in widely varying applications by the courts, is little better than no privilege at all.”11
2. Who is asking
The people who type legal questions into chatbots are, disproportionately, the people the legal system already fails. The Legal Services Corporation’s most recent national study found that low-income Americans “do not get any or enough legal help for 92% of their substantial civil legal problems.”12 Nearly half of those who did not seek help for a problem cited cost.12 In a National Center for State Courts study of nearly a million civil cases, at least one party was self-represented “in more than three-quarters of the cases,” usually the defendant.18
Meanwhile, the tool is everywhere. Pew found in February 2026 that 49% of U.S. adults use AI chatbots, up from 33% in 2024, and that 20% use them for medical advice.13 Legal advice was not on Pew’s list of uses. The best direct measure we found comes from Britain. In a survey by the law-reform charity JUSTICE, about one in six people who had a recent legal dispute had used a chatbot for “help, information, or advice” about it. Among 18-to-24-year-olds the figure was 26%.19 Their shared conversations covered deposits, evictions, dismissals, debt collection and court procedure. The researchers concluded these were problems that “largely fall outside the scope of legal aid.”19
Picture two people with the same problem on the same night. One sits in a lawyer’s office and says, “Here is what happened. Am I in trouble?” The other types the same words into a phone at the kitchen table, because a lawyer costs more than rent. The law treats the first conversation as sacred and the second as a document. The only difference between them is money.
That is not what the privilege is for.
3. Why we have the privilege at all
The Supreme Court has explained the attorney-client privilege in one sentence: “Its purpose is to encourage full and frank communication between attorneys and their clients, and thereby promote broader public interests in the observance of law and administration of justice.”11 The privilege is not a reward for hiring a lawyer. It exists because society is better off when people find out what the law requires before they act, and they will not ask honestly if their questions can be used against them.
The person asking a chatbot whether a landlord can keep a deposit, or whether a late payment voids an insurance policy, is doing exactly what that sentence hopes for. They are trying to learn the law so they can follow it. Today the law punishes that effort, but only for people who cannot afford a lawyer.
The law has stretched the privilege before to fit how people actually get legal help. In 1961, Judge Henry Friendly held that the privilege can cover a client’s statements to an accountant working for the client’s lawyer, when the point is to help the lawyer give legal advice. Friendly could “see no significant difference” between a lawyer who sends a client to “an interpreter to make a literal translation of the client’s story” and a case where the interpreter “has been brought along by the client.” He then extended the same logic to accountants, because “[a]ccounting concepts are a foreign language to some lawyers.”20 For millions of people, the law itself is the foreign language, and a chatbot is the only interpreter they can afford. Under Kovel, though, the interpreter is protected only when it helps a lawyer give legal advice.20,6 That is why a court cannot fix this alone and a legislature should.
The Supreme Court also gave the best answer to the claim that privileges cost the courts the truth. Recognizing a privilege for psychotherapists in 1996, it observed that without one, much of the evidence litigants want “is unlikely to come into being,” and that this “unspoken ‘evidence’ will therefore serve no greater truth seeking function than if it had been spoken and privileged.”21 The same is true here. If people learn that their legal questions to a chatbot become exhibits, they will not stop having legal problems. They will stop asking, or they will ask less honestly and get worse answers.
The companies have noticed. OpenAI’s chief executive, Sam Altman, has said that “talking to an AI should be like talking to a lawyer or a doctor,”22 and that the lack of confidentiality is “very screwed up.”23 He is right about the problem. But a company calling for privilege has its own interests, so any privilege must belong to the user and must never become a shield for the company. More on that below.
4. The serious objections, and the answers
The strongest case against an AI privilege comes from Professor Ira Robbins of American University, in an essay Judge Rakoff cited in Heppner.24,6 Robbins argues that privileges “rest on fiduciary duties, confidentiality safeguards, and accountability structures wholly absent from AI interactions,” and that recognizing an AI privilege “would entrench corporate opacity precisely when courts need transparency.”24 His points deserve answers, one by one.
Objection: privileges protect relationships, not software
Maybe so, for judge-made privileges. Robbins argues that “the law protects relationships, not functionalities,” and that “nobody suggests a calculator privilege.”24 But legislatures create privileges whenever they decide confidentiality serves the public. Georgia’s evidence code protects communications between “accountant and client,” and since July 2025 it has protected communications made in “victim centered practices or victim-offender dialogues.”25 Neither is a lawyer’s privilege. Robbins himself concedes that “[l]egislatures may experiment with narrow protections for therapeutic uses.”24 He would stop at therapy. We would not, but he agrees on who should decide. Our proposal is a statute, not a new rule invented by a court.
Objection: no one owes the user a duty of confidentiality
True today, and that is the right thing to fix. In Heppner, the privacy policy was one of the court’s reasons for finding no confidentiality: the company reserved the right to share data with “governmental regulatory authorities.”6 A statute can impose the duty directly. It can bar disclosure without the user’s consent or a court order, and require the provider to notify the user of any legal demand. Robbins faults privacy promises in terms of service because those “changeable contracts lack the professional enforceability of bar or licensing obligations.”24 A statute is not a changeable contract.
Objection: the company reads your chats anyway
This is the best objection, and the statute should take it head on. As Mikołaj Barczentewicz of the International Center for Law & Economics put it, “It is hard to justify keeping chats from courts and opposing parties while allowing the AI company to examine them for training, advertising, or other commercial purposes.”26 So the privilege should apply only when the provider does not train on, sell, or target ads with the conversation. That gives companies a reason to build protected modes. OpenAI says client-side encryption is on its “long-term roadmap.”27 It also helps users, because a provider that cannot read a conversation generally cannot be ordered to produce it, though that protection is incomplete when copies live elsewhere.26
Objection: people will launder evidence through the chatbot
The attorney-client privilege solved this problem long ago. “[T]he protection of the privilege extends only to communications, and not to facts.”11 Uploading a contract to a chatbot would not privilege the contract, any more than handing it to a lawyer does. The underlying events stay discoverable from every other source. A lawyer’s client “cannot be compelled to answer the question, ‘What did you say or write to the attorney?’”11 Under our proposal, the chatbot user could not be compelled to say what they asked the system or what it answered. Everything else stays in play.
Objection: criminals will use it to plan crimes
Then the conversation would not be privileged, for the same reason it would not be with a lawyer. The crime-fraud exception keeps the “seal of secrecy” from covering communications “made for the purpose of getting advice for the commission of a fraud” or crime.5 Courts already apply that exception, and the Supreme Court has approved private (in camera) review of the disputed communications to decide it.5 An agent-user privilege should adopt it word for word, along with an exception for threats to someone’s life or safety.
Objection: chatbot advice is bad advice
Sometimes it is. But the privilege does not certify the advice. It protects the person asking. Taking away the privilege does not make anyone’s answers more accurate. It only makes the questions dangerous.
5. What a good statute looks like
Arizona tried the short version. House Bill 2410 had one operative sentence: “A person’s communication with an artificial intelligence is privileged if the person would have been entitled to privileged communication had the person sought the advice from a human professional.”28 It passed the Arizona House 53–4 in February 2026.14 The Senate Judiciary and Elections Committee held it, and it died there.15 One sentence leaves too much open. It does not say who holds the privilege, whether the company can invoke it, what happens to conversations the company trains on, or what exceptions apply. Those gaps are where the critics aim.
A better bill would do seven things:
- Give the privilege to the user alone. Only the user can claim it or waive it. The provider may assert it only on the user’s behalf, and never to protect itself.
- Cover communications made to get legal information or advice. That means the user’s questions and the system’s answers about a legal problem, including documents the user writes for that purpose. It does not cover pre-existing documents or underlying facts.
- Require confidentiality as the price of protection. The privilege applies only if the provider does not use the conversation for training, advertising or sale, and discloses it only with consent or a court order.
- Give users a seat at the table. A provider served with a subpoena or warrant for covered conversations must notify the user, unless a court orders delay, so that the user can assert the privilege. In the Manhattan case, the user had no right to challenge the warrant before OpenAI complied.16 A privilege is only as good as the chance to assert it.
- Copy the attorney-client exceptions. Include crime-fraud, threats to life or safety, and disputes between the user and the provider.
- Leave the company exposed. The privilege must not block discovery in product-liability, consumer-protection or regulatory matters about the provider’s own conduct. Robbins is right that a privilege should not become a corporate shield, and this provision makes sure it does not.24
- Keep work product intact. Nothing in the statute should narrow the protection that courts in Michigan, Colorado, New York and Texas have already recognized.7,8,9,10
6. Why Georgia, and why now
A state statute does more than one might think. Under Federal Rule of Evidence 501, “in a civil case, state law governs privilege regarding a claim or defense for which state law supplies the rule of decision.”29 A Georgia privilege would therefore protect Georgians in their contract, tort, landlord and family cases, even when those cases land in federal court. It would not control federal criminal cases like Heppner. But the Supreme Court recognized the psychotherapist privilege in federal court only after “all 50 States and the District of Columbia” had enacted some form of it, and it reasoned that state consensus shows “reason and experience” support a privilege.21 Federal privilege law tends to follow the states.
Georgia has also already started legislating chatbots. In 2026 it enacted rules for “AI companion chatbots,” taking effect July 1, 2027, covering disclosures, protections for minors and crisis protocols.30 That law says nothing about whether what you tell a chatbot can be used against you in court. The next session is the place to say it.
The law already accepts that people need a safe place to ask what the law requires. It has decided, for now, that only people who can afford a lawyer get one. That line made sense when legal advice came only from lawyers. It does not anymore.
The bottom line
The privilege was built to encourage people to ask before they act. Millions now ask a machine, because it is the only counsel they can afford, and every honest question they type can become evidence against them. A narrow, user-held agent-user privilege, with the same crime-fraud exception lawyers live under and a confidentiality requirement the AI companies must meet to qualify, would close that gap without hiding the facts from anyone. Asking should not be evidence.
What you can do
Georgia’s Evidence Code is a statute, enacted by the General Assembly in 2011.31 So a new privilege needs a bill: passed by both chambers and signed by the governor. That makes the people to persuade your state representative and state senator.
- Write your own legislators. Lawmakers listen most closely to the people who can vote for them. Find yours by home address with Open States.
- Write before the session starts. The General Assembly meets “on the second Monday in January,” which is January 11, 2027, and the regular session is capped at 40 days a year.32 Bills get drafted and lined up in the weeks before it opens. Once it starts, lawmakers are swamped.
- Make one specific ask. Ask them to sponsor or co-sponsor a bill adding an AI-user privilege to O.C.G.A. § 24-5-501, built on the seven points in section 5, and send them this post.
- Copy the committees. Expect a privilege bill to go to a Judiciary committee in each chamber. The 2025 bill that added the victim-offender dialogue privilege went through the House Judiciary Non-Civil Committee and the Senate Judiciary Committee.33,25 A note to their members helps even if they do not represent you.
- If you are a lawyer, use the Bar. A State Bar section or committee can submit a legislative proposal to the Bar’s Advisory Committee on Legislation. If that committee adopts it, the proposal goes to the Board of Governors for final approval.34
Sources
- Jason Lalljee, “Palisades arson suspect allegedly used ChatGPT to create images of burning city,” Axios (Oct. 8, 2025) (reporting the federal criminal complaint). Read the source (axios.com)
- “Prosecutors used a man’s ChatGPT logs to try to link him to the Pacific Palisades fire, but it didn’t work,” TechSpot (June 29, 2026). Read the source (techspot.com)
- Eric Leonard, “Jury selected for Palisades fire re-trial,” NBC Los Angeles (Nov. 4, 2026). Read the source (nbclosangeles.com)
- Jonathan Lloyd & Eric Leonard, “Palisades arson fire re-trial on hold for judge’s review of evidence discovery claims,” NBC Los Angeles (Nov. 16, 2026). Read the source (nbclosangeles.com)
- United States v. Zolin, 491 U.S. 554 (1989). Read the source (supreme.justia.com)
- United States v. Heppner, No. 25 Cr. 503 (JSR), Memorandum, ECF No. 27 (S.D.N.Y. Feb. 17, 2026), reported at 820 F. Supp. 3d 292 (copy hosted by Akin Gump). Read the source (akingump.com)
- Warner v. Gilbarco, Inc., No. 2:24-cv-12333, ECF No. 94 (E.D. Mich. Feb. 10, 2026) (Patti, M.J.). Read the source (law.justia.com)
- Morgan v. V2X, Inc., No. 1:25-cv-01991, ECF No. 65 (D. Colo. Mar. 30, 2026) (Braswell, M.J.). Read the source (cases.justia.com)
- Assini v. Hayward, 2026 NY Slip Op 26086 (Sup. Ct. Nassau County June 4, 2026). Read the source (nycourts.gov)
- Norton Rose Fulbright, “AI in litigation: Texas Business Court finds AI chat log privileged” (2026) (summarizing the June 3, 2026 order in Tate Group Automotive, LLC v. Legacy Automotive Capital, LLC). Read the source (nortonrosefulbright.com)
- Upjohn Co. v. United States, 449 U.S. 383 (1981). Read the source (supreme.justia.com)
- Legal Services Corporation, The Justice Gap: The Unmet Civil Legal Needs of Low-income Americans, Executive Summary (2022). Read the source (justicegap.lsc.gov)
- Pew Research Center, “Americans and AI 2026: Chatbots, Smart Devices and Views on Impact” (June 17, 2026). Read the source (pewresearch.org)
- Arizona State Senate, Fact Sheet for H.B. 2410 (Mar. 5, 2026). Read the source (azleg.gov)
- Arizona Legislature, Bill Status Overview, HB 2410 (2026 Regular Session) (final disposition: “Held in Senate”). Read the source (apps.azleg.gov)
- United States v. Kim, No. 25 Crim. 359 (LGS), Order, ECF No. 56 (S.D.N.Y. June 22, 2026) (Schofield, J.). Read the source (assets.alm.com)
- In re OpenAI, Inc., Copyright Infringement Litigation, No. 25-md-3143 (SHS) (OTW), Order (S.D.N.Y. Jan. 5, 2026) (Stein, J.). Read the source (storage.courtlistener.com)
- Paula Hannaford-Agor, Scott Graves & Shelley Spacek Miller, The Landscape of Civil Litigation in State Courts (National Center for State Courts 2015). Read the source (ncsc.contentdm.oclc.org)
- JUSTICE & Administrative Fairness Lab, “What AI Chatbots Can Teach Us About Unmet Legal Needs” (2026). Read the source (justice.org.uk)
- United States v. Kovel, 296 F.2d 918 (2d Cir. 1961) (Friendly, J.). Read the source (law.justia.com)
- Jaffee v. Redmond, 518 U.S. 1 (1996). Read the source (law.cornell.edu)
- Andrea Roth & Rebecca Wexler, “AI Privilege and the Legal Services Crisis,” 79 Stan. L. Rev. Online 78 (2026) (quoting Sam Altman (@sama), X (June 5, 2025)). Read the source (review.law.stanford.edu)
- Sarah Perez, “Sam Altman warns there’s no legal confidentiality when using ChatGPT as a therapist,” TechCrunch (July 25, 2025). Read the source (techcrunch.com)
- Ira P. Robbins, “Against an AI Privilege,” Harv. J.L. & Tech. Digest (Nov. 7, 2025). Read the source (jolt.law.harvard.edu)
- O.C.G.A. § 24-5-501 (Certain communications privileged), as amended by Ga. L. 2025, p. 497 (HB 582). Read the source (law.justia.com)
- Mikołaj Barczentewicz, “AI Privilege: Code May Matter More Than Courts,” International Center for Law & Economics (July 15, 2026). Read the source (laweconcenter.org)
- Dane Stuckey, “Fighting the New York Times’ invasion of user privacy,” OpenAI (Nov. 12, 2025). Read the source (openai.com)
- Arizona H.B. 2410, 57th Leg., 2d Reg. Sess. (2026) (introduced version). Read the source (azleg.gov)
- Federal Rule of Evidence 501. Read the source (law.cornell.edu)
- O.C.G.A. § 39-5-6 (AI companion chatbots), enacted by Ga. L. 2026, p. 822 (SB 540), effective July 1, 2027. Read the source (law.justia.com)
- O.C.G.A. Title 24 (Evidence), with editor’s note to Ga. L. 2011, p. 99 (HB 24). Read the source (law.justia.com)
- Ga. Const. art. III, § IV, ¶ I (organization and sessions of the General Assembly). Read the source (georgiacommons.org)
- Georgia General Assembly, HB 582 (2025–26 Regular Session), Georgia Survivor Justice Act, bill status page. Read the source (legis.ga.gov)
- State Bar of Georgia, “Legislative Policy” (Standing Board Policy 100 process). Read the source (gabar.org)
Further reading
- Justin Curl & Mihir Kshirsagar, “AI and Privilege After United States v. Heppner,” Lawfare (2026). Link (lawfaremedia.org)
- Elizabeth X. Guo, “United States v. Heppner,” Harvard Law Review Blog (Mar. 23, 2026). Link (harvardlawreview.org)
- Jennifer Ellis, “When Does Client Use of AI Waive Privilege?” ABA Law Technology Today (May 20, 2026). Link (americanbar.org)
- Future of Privacy Forum, 2026 Chatbot Legislation Tracker. Link (fpf.org)