Ask AI for a lawyer, and 95% of the firms we sampled aren’t in the answer

Each engine was asked “Who are the best [practice] lawyers in [city]?” for family, employment, real estate, wills and estates, corporate, criminal, immigration, personal injury, tax, and charity and nonprofit law. For the Bay Area the question said “the San Francisco Bay Area”. A firm counts as named if, in at least one of three runs, its name or domain appeared in the answer or its website was cited as a source. Counting only names and domains written in the answer text gives 12 firms instead of 13.

Each answer names only a handful of firms, while each metro has more than a thousand. Most firms will be absent from any one list, so the 95% describes how few places there are in AI answers, not a failing of the firms. Many of the sources the engines cite are directories and rankings; a data brief on where AI gets its lawyer recommendations follows in October.

The answers aren’t stable. Of the 11 times ChatGPT named one of our sampled firms for a question, the firm appeared in all three runs only twice; four times, it appeared in just one. Perplexity returned the same sources on all three runs of each question, so its repeated runs may not be independent. Google showed no AI Overview at all for 48% of runs (57 of 119). City differences are too small to detect with this sample.

We counted seven terms (best, top, leading, premier, #1, expert, specialist) when a firm used them about itself without naming a source, such as an award or a directory. “Expert” was the most common in every city. The differences between cities are exploratory (chi-square p = 0.06). Testimonials, named awards, certified-specialist statements and phrases such as “best interests of the child” weren’t counted.

The rules differ by place. The Law Society of Ontario treats superlatives such as “best” and “#1” as rankings, which must be genuine and provable, and reserves “specialist” for Certified Specialists. Florida’s Bar says “the best” is generally not objectively verifiable, and allows “expert” only when it can be. New York and California prohibit false or misleading communications and warn that unsubstantiated comparisons may mislead. They allow an uncertified lawyer to say they specialize. Dropping “specialist” from the US cities changes the count from 93 to 91 firms (35.5% to 34.7%), so the rounded figure stays 35%.

A term on this list isn’t a violation. Only a regulator decides that, and a claim may be true and provable.

Firms that used these terms were named slightly more often: 8% of them (7 of 93), against 4% of the rest (6 of 169). The difference isn’t statistically significant (Fisher’s exact p = 0.23; adjusted odds ratio 1.70, 95% CI 0.51–5.60). With only 13 firms named, the data are compatible with the terms helping, making no difference or hurting. This study can’t tell which, and it doesn’t test cause and effect.

New York’s rules required a firm’s home page to carry the label “Attorney Advertising” until June 1, 2026, when the requirement was repealed (Joint Order of the Appellate Division, May 27, 2026, effective June 1, 2026, listed on nycourts.gov; the change is summarized by the New York State Bar Association). Four months later, 34% of New York City firm home pages in our sample still carry it (24 of 70; 95% CI 24–46%). We can’t tell whether firms haven’t updated their sites or keep the label for other jurisdictions where they practice. Counting the variant “Attorney Advertisement” gives 36%. A check of a random 20% of pages matched the automatic count. A full re-scan of all 70 pages found and fixed one label the parser had missed; it is included in the 34%.

• Sample: 22,406 Google Business listings for lawyers and law firms in four metros. From the 5,313 eligible firms with their own website, we drew a seeded random sample of 80 per metro (320). Listings were retrieved up to a cap per metro (1,500; 1,800 in New York City), in Google’s internal listing-ID order, which we assume is unrelated to firm quality. The cap covered about 46% of listings with a website (25% in New York City). Listings without a website were excluded. 262 of the 320 sites (82%) were analysable. The other 58 blocked our crawler (24), redirected to another domain (11), had little or script-only text (12), or returned errors (11). The sample was drawn from listings, not search rankings.
• Websites: each firm’s home page and up to four linked pages, fetched politely (robots.txt respected, one request per second per site).
• Classification: 1,603 term mentions (1,602 unique phrases), each classified independently by two AI models (Gemini 3.8 Flash and GLM 5.3 Flash). Against a 200-item reference set of Ontario firms, the two agreed on 193 items and were correct on 96.9% of those. On a 150-item US reference set (New York, Miami and the Bay Area), the labels accepted automatically were 99.3% accurate (weighted to the full set; 147 of 150 unweighted), and all three errors over-counted claims. The 53 mentions where they disagreed or had low confidence were decided by a third AI model (Claude), with a written reason for each. If every one of those 53 had gone the other way, the self-claim share would range from 34% to 39% (it is 35%). The term list, the rule and the classifier prompt were developed in a Toronto pilot on September 29, 2026 (57 firms), whose data are not pooled.
• AI answers: 479 of 480 planned answers collected on October 1, 2026 through DataForSEO (ChatGPT and Gemini app scrapers, Perplexity Sonar, Google AI Overviews). Firms were matched by domain and normalized name. “Gemini” is DataForSEO’s Gemini app scraper, which reports a Flash-Lite model label; “Perplexity” is the Sonar API. Both may differ from what a person sees in the apps.
• Method fixed in advance: the core method (questions, terms, the claim rule and the sampling approach) was written and hashed (SHA-256) 10 minutes before data collection began (02:44 and 02:54 UTC, October 1, 2026). The hash wasn’t registered with a third party at the time. Details the method didn’t specify were filled in afterwards, and they and every other change are listed in the deviations log. Not released: the engine code and the reference sets, because the reference sets quote firms’ websites.

The pre-registration, deviations log and rule-wording review are in the research pack.

• Coverage: Google Business listings miss firms without a profile.
• Volatility: AI answers change with time, location and wording. We asked one question per practice area, on one day.
• Location: ChatGPT could only be set to a country, not a city, so the city was in the question. Perplexity takes no location.
• Size bands: firm size could be detected for only 58 of 262 firms.
• Classifier accuracy: it was measured on 200 Ontario mentions and 150 US mentions, both labelled by AI models rather than people. Its errors on US sites all over-counted claims, so the self-claim figures may be slightly high. Because we searched for only seven terms, the counts are a lower bound for self-description of this kind (words such as “elite” or “foremost” weren’t counted).
• AI review, not human: the reference set, the disputed mentions and the New York check sample were reviewed by AI models, not people.
• Rule descriptions: checked against primary sources by an AI, not reviewed by a licensed lawyer. This page is general information, not legal advice. Rules checked October 1, 2026.

Innovara Studios is a brand, website and search studio. We sell services to law firms, so we have an interest in this topic. No firm or person is named in this study, its charts or its dataset, and no firm paid for or saw it before publication. How we research and correct our work is set out in our editorial standards.

Updated October 2, 2026. After an independent methods review, we clarified wording: the claims result (from “no sign they help” to “the data can’t say”); what counts as “named”; the listing cap; the 58 excluded sites; the source for New York’s rule change; and other limits. One sensitivity figure was corrected: dropping “specialist” leaves the rate at 35% (34.7%), not 34%. No other figure changed. The anonymized dataset now bins one column (mention counts) to lower the risk of identifying a firm. Details are in the deviations log.