How we ran this study

We compared answers from ChatGPT, Gemini, Google AI Mode and Google AI Overviews: which insurers they recommend, how they explain policy features and trade-offs, what help they offer after purchase, where their information comes from, and whether selected factual statements are right.

The study covers car insurance. Home and contents questions are outside its question set and results.

The twelve consumer situations include a first-time buyer, a household under budget pressure, a rural driver, an imported-car owner and someone who wants help over the phone. They are researcher-authored situations, not a sample of New Zealand consumers.

The questions cover understanding the need for insurance, exploring options, choosing cover, comparing insurers, checking the details, buying and switching, and ongoing support. These are groups of questions, rather than real people observed over time. Buying factors describe the priorities in the questions and the reasoning in the answers, rather than how common a preference is among consumers.

Regional questions name where the consumer lives or keeps the car. Other questions describe particular claims, repair problems and policy decisions. We retain the original wording of each question.

Onsomble collected the answers through the four consumer applications. Collection completed on 11 October 2026 at 7:48 pm New Zealand daylight time. Each of the 183 car-insurance questions was planned for each application.

All collection settings were New Zealand. Region names are part of the question text; these are not captures taken from sixteen different regional access locations. Failed executions do not enter the collected-answer denominator.

Conditional recommendations count with their conditions retained. An insurer included in a shortlist need not be the first or strongest recommendation. Ties in frequency remain ties. Collected answers with no qualifying insurer stay in the denominator.

We distinguish insurers and customer-facing insurance brands from brokers, underwriting groups, citations, neutral references and escalation organisations. Bank insurance brands remain distinct options when actually offered as choices. AA Roadservice membership is distinct from AA Insurance. An incidental name does not count as a recommendation.

Support concerns an existing policy or claim: renewal, switching, a parked-car excess, damage around a move, a disputed repair or write-off, and transport to a distant repairer.

Asking a new insurer about claim updates and repair guarantees before signing up is acquisition validation. Its four answers are excluded from actual support. A renewal shortlist is shopping advice, rather than a recommendation for another insurer to handle an existing claim.

Six consumer situations have no eligible question in the persona-assigned actual-support group. This is an absence of evidence, not a zero-performance result. Additional claims scenarios inform the interpretation without being assigned to invented consumer support cells.

Consumer selection uses twelve questions answered by all four applications: four answers for each situation, 48 in total. Guidance, general benchmark questions, priority variants and support questions do not increase this purchasing denominator.

The regional comparison uses sixteen region-named selection questions, each answered by all four applications. Every region contributes four answers, giving 64 in total. Platform comparisons use the same twelve consumer questions per application, or the same sixteen regional questions per application, without pooling the two groups.

Additional regional scenarios have different facts. They are used to interpret the answers, rather than to rank insurers across regions on unlike questions.

The sentiment dataset contains 1,926 saved insurer claims across the 48 consumer-selection answers and 64 regional-selection answers. Each record is one unique claim about one insurer within one answer. Repetition across narratives does not add another claim.

The stance is the saved claim stance, not an inference from recommendation inclusion or a brand score. Strong-positive and positive stances are grouped as positive; hedged-mixed as mixed; neutral as neutral; negative and strong-negative as negative. Different claims in one answer can express different stances about the same insurer.

The bars show positive, mixed, neutral and negative claims in the selected answer group. Opening a matrix cell shows the breakdown and original passages. Brokers, comparison services, corporate groups and AA Roadservice are excluded from insurer sentiment.

The source dataset contains 6,056 answer, domain and URL associations, with a complete index of all 728 car-insurance answers, including those with no stored sources. These are the references recorded against the saved answers.

In the source tables, a domain counts once for each answer associated with it, however many times it appears. An answer can contribute to several websites. Publisher types describe those websites, such as insurer websites, comparison tools, consumer groups and news.

“Had sources” counts answers with a recorded source association. “Showed you a link” counts answers containing a web URL in the captured text. Saved associations and literal answer URLs remain separate: an associated source need not appear as a URL in the saved answer.

Each topic was reviewed across four platform answers, giving 200 review cells. Sources were checked on 12 October 2026, one day after collection. The review considers context, an explicit year or version, and material policy conditions; dated historical documents retain their dates.

The selected assertion is checked in each corresponding answer, including any direct conflicting statement. Supported, inaccurate, materially incomplete, unresolved and not-stated outcomes remain separate. Paragraph references identify the exact evidence; they do not assign a verdict automatically.

104 cells did not state the selected assertion. The accuracy section shows the remaining 96 statements: 81 correct, seven wrong and eight unclear. These are statement-and-platform counts, not counts of independent answers with errors.

The shortlist was selected for consequential claims and began with thirty ChatGPT assertions. It is not a random sample of every statement. A supported-share calculation excludes unresolved and not-stated cells; the audit does not provide a population-wide accuracy estimate.

The consumer situations are not a population sample, and the journey groups do not track real consumers over time. Regional differences describe the answers to region-named questions with a New Zealand collection setting.

Recommendation frequency is not market share, actual quotation prices or measured claims performance. The study shows what the assistants said, with the original answers available to inspect; the factual audit covers selected statements rather than every claim.