Political polls shape election narratives, but opaque panels, hidden funders, and low scientific standards mean their results are rarely reliable, Associate Professor Ben Gray argues.
Commentary: This general election will be close and opinion polls may make a significant difference to the outcome. We must know whether we can trust them.
A recent political poll by Curia Research had both the major parties National and Labour polling under 30 percent – on the face of it an alarming prospect for either party weeks out from an election, but is it accurate?
Conducting a trustworthy scientific survey is expensive and time consuming.
The New Zealand Health Survey is a good example of a high-quality survey. It randomly selects households from census data, representing the total population by age, sex, region, ethnicity (Māori, Pacific, Asian) and deprivation.
Interviews are in person with multiple call backs resulting in a 75 percent response rate.
Interpreters are used for the 5 percent of the sample not fluent in English. The methodology is approved by an independent Research Ethics Committee and complies with National Ethics Advisory Committee Research Ethics Guidelines.
By comparison there are several reasons why political polls may be unreliable.
Political polls all use ‘panels’ (online or telephone) from which they claim to select a random sample. None provide any information about how their panel is recruited.
Standards from the European Society for Opinion and Marketing Research (ESOMAR) preclude random recruitment, so all members of panels should have chosen to be part of a panel. Some are paid for filling out the survey.
There are companies, for example Dynata, that develop and maintain these panels. Dynata’s panel comprises 297,000 individuals who have opted in to participate in survey research. The panel is stratified according to factors like age, sex, region. A sample of the full panel is selected.
Without knowing the panel size used in political polls and how it is stratified, we cannot judge whether it could be reliable. Only the agency Verian (previously known as Kantar Public and Colmar Brunton) provides weighting for ethnicity and education.
Do the other panels have that information? Without it, the panel could have few Māori or Pacific people. Research finds that these non-probability panel members are more likely to be unemployed, male-heavier internet users. Some political surveys will be part of an ‘omnibus’ survey including questions from other clients on other topics.
Landlines are now uncommon and will be skewed towards the elderly or business numbers. Cell phone numbers may be randomly generated but there are now many people who do not answer calls from numbers they do not recognise.
None report a response rate.
We know that they exclude at least 5 percent of the population because they do not use interpreters.
A random sample of a panel does not produce a random sample of the population.
In contrast, NZ Health Survey protocols are externally quality controlled. Most pollsters are members of their industry bodies, Research Association of New Zealand (RANZ) or Marketing Research Association and claim compliance between their codes and the code of ESOMAR.
There is no assessment of compliance with the RANZ code, although there is provision for complaints to those bodies. Roy Morgan has accreditation from the International Organization for Standardization, which is likely more substantial.
Curia Research, responsible for the recent damning political poll mentioned above, resigned from RANZ in 2024 following three complaints for practice that breached the polling code for non-political polls, but they claim compliance with ESOMAR.
All polls cite a maximum margin of error around 3 percent. However, this calculation is based on the presumption that the sample is a random sample; which as discussed, is not the case. The less random the sample, the higher the margin of error that it reflects the full population.
All panels say they weight results, but none describe how. One approach is to assume that Māori who answer represent all Māori and put extra weight on that response. This will add further uncertainty.
Who pays for the poll is also important. One News and Radio NZ have a transparent reason for funding. Both Infrastructure New Zealand and The Taxpayers’ Union must feel the investment is worthwhile.
Talbot Mills polls just say ‘corporate client’ although they are known to do polls for the Labour Party, and Roy Morgan says nothing.
Would it matter if these polls were paid for by the Chinese Communist Party? By calling their poll ‘Taxpayers’ Union Curia Poll’ there is an automatic bias, created by people who distrust the Taxpayers’ Union not responding to a survey.
These problems can be mitigated by transparency. Research Association of NZ gives advice to media outlets to consider whether to publish a poll. They should request full availability of the poll data so they can decide whether the poll is sufficiently reliable to publish.
Verian and Freshwater Strategy provide more information to the public than the others. There is no information about Talbot Mills polls. Ideally, full data should be publicly available, as in the example of the health survey.
Political polls are research, and to be trustworthy they need to be conducted ethically. None tell us who they poll, there is no external quality control, some are funded by vested interests, and there are many ways a poll can be skewed either inadvertently or deliberately to influence the result. Without more scrutiny and better information, we cannot know whether they can be trusted.
Associate Professor Ben Gray has convened the course in Professional Practice for undergraduate medical students in Wellington for the past 20 years. The course involves appraisal of evidence, research ethics, cultural competence and health Law. He is an affiliate of the Department of Bioethics at University of Otago.
This piece was first published on Newsroom, 14 September 2026.