Author Topic: Polling isn't getting worse  (Read 319 times)

0 Members and 1 Guest are viewing this topic.

Offline Michael Hardner

  • Hero Member
  • *****
  • Posts: 12463
Polling isn't getting worse
« on: May 30, 2018, 10:51:42 am »
https://fivethirtyeight.com/features/the-polls-are-all-right/

Quote
You read that right. Polls of the November 2016 presidential election were about as accurate as polls of presidential elections have been on average since 1972. And polls of gubernatorial and congressional elections in 2016 were about as accurate, on average, as polls of those races since 1998. Furthermore, polls of elections since 2016 — meaning, the 2017 gubernatorial elections and the various special elections to Congress this year and last year — have been slightly more accurate than average.

Anyone here surprised ?

I liked how vitriolic fivethirtyeight was when - after giving a 30% for Trump to win or something - people told them YOU SAID TRUMP WOULD LOSE !

:D

Share on Facebook Share on Twitter


Offline cybercoma

  • Full Member
  • ***
  • Posts: 2956
Re: Polling isn't getting worse
« Reply #1 on: May 31, 2018, 11:34:18 am »
The polls are accurate. The polls, however, do not measure public opinion in the same way elections are held. They would have to sample every single riding to give a more accurate picture of the election outcome. That would be prohibitively expensive, so they make inferences from popular opinion and extrapolate based on the characteristics of the populations in each riding. Most of the time they don't take that second step and just report on popular opinion, sometimes adjusted for voting intention and past voting behavior in relation to demographic characteristics. Another challenge with polling is that you're dealing with human beings. We are rational creatures, which means the publication of results from polls leads people to think about and react to them, thereby changing the results almost immediately upon publication. Nevertheless, the polls that we have a far better predictors than drawing names out of a hat or flipping a coin. The Trump and Hillary polls is also a perfect example that the electoral system does not work based on the will of the voters. The polls were very accurate. In fact, Hillary Clinton did beat Donal Trump in the election, based on the number of votes each received. That's, however, not how you get elected in the United States. That's not how a premier is elected in Ontario either. You only need a plurality of votes across various sub-divisions of the population. But at the end of the day when you aggregate the vote, ignoring ridings and "electoral college" the polls are very close.

Offline Michael Hardner

  • Hero Member
  • *****
  • Posts: 12463
Re: Polling isn't getting worse
« Reply #2 on: May 31, 2018, 02:19:09 pm »
They do have seat projections from Ontario this election.  Let's see how it turns out.

Offline cybercoma

  • Full Member
  • ***
  • Posts: 2956
Re: Polling isn't getting worse
« Reply #3 on: May 31, 2018, 07:03:00 pm »
They do have seat projections from Ontario this election.  Let's see how it turns out.
"Projections" being the operative word. Those are still based on popular opinion, then translated through algorithms based on demographic characteristics in the ridings. They're not riding by riding opinion polls, which would be far more accurate and FAR more expensive.
Agree Agree x 1 Informative Informative x 1 View List

Offline Michael Hardner

  • Hero Member
  • *****
  • Posts: 12463
Re: Polling isn't getting worse
« Reply #4 on: May 31, 2018, 07:21:02 pm »
"Projections" being the operative word. Those are still based on popular opinion, then translated through algorithms based on demographic characteristics in the ridings. They're not riding by riding opinion polls, which would be far more accurate and FAR more expensive.


Wow.  That sounds right.  How'd you know that ?  I am impressed...

Offline cybercoma

  • Full Member
  • ***
  • Posts: 2956
Re: Polling isn't getting worse
« Reply #5 on: June 02, 2018, 09:23:27 am »
I follow these things closely.  :P

Offline Michael Hardner

  • Hero Member
  • *****
  • Posts: 12463
Re: Polling isn't getting worse
« Reply #6 on: June 02, 2018, 09:30:35 am »
I follow these things closely.  :P

You nudged me into looking into all the 'projections' and I'm starting to think we are in a fool's paradise when it comes to projections.

Everyone seems to be posting TWO poll trackers, neither of which seem 100% clear to me.  For example, why does the CBC tracker put the chance of an NDP majority high above that of a minority ?  And the other tracker is not done by a statistician.

I am starting to suspect we will see a BIG SURPRISE on election day.

PS: This may be the rebirth of intelligent 'public' discussion I have been waiting for, for 15-20 years or so.  I never suspected that the re-emergence of intelligent public discussion would start with error-prone lone guns and build on that but it makes sense.

Offline Squidward von Squidderson

  • Full Member
  • ***
  • Posts: 5630
Re: Polling isn't getting worse
« Reply #7 on: June 03, 2018, 09:38:14 am »
Quote
For example, why does the CBC tracker put the chance of an NDP majority high above that of a minority ?

For the NDP to get a minority the Liberals would need to win seats that would have gone to the PCs.  That doesn’t look at all realistic. Hence, a minority NDP gov’t is very unlikely.

Offline Michael Hardner

  • Hero Member
  • *****
  • Posts: 12463
Re: Polling isn't getting worse
« Reply #8 on: June 03, 2018, 09:52:49 am »
For the NDP to get a minority the Liberals would need to win seats that would have gone to the PCs.  That doesn’t look at all realistic. Hence, a minority NDP gov’t is very unlikely.

I don't follow this.  Why can't the NDP win seats that would have gone to the PCs ?  That is what happened in 1990.  When I read the methodology of the other tracker it seemed that they were basing results on past results which seems experimental to me.

Offline Squidward von Squidderson

  • Full Member
  • ***
  • Posts: 5630
Re: Polling isn't getting worse
« Reply #9 on: June 03, 2018, 10:09:31 am »
I don't follow this.  Why can't the NDP win seats that would have gone to the PCs ?

Yes, that would be a more likely scenario....   but then it would be a majority NDP gov’t.   

What you don’t seem to be getting is that for an NDP minority, the Libs would have to win seats unexpectedly.

Offline Michael Hardner

  • Hero Member
  • *****
  • Posts: 12463
Re: Polling isn't getting worse
« Reply #10 on: June 03, 2018, 10:23:24 am »
Yes, that would be a more likely scenario....   but then it would be a majority NDP gov’t.   

What you don’t seem to be getting is that for an NDP minority, the Libs would have to win seats unexpectedly.

'fewer' but ok...

Look the Liberals are already in 3rd.  If the NDP win X seats they will have a minority.  If they win Y seats, where Y > X, then it's a majority.  Why is it is easier to win way more seats than just a few more.

Offline cybercoma

  • Full Member
  • ***
  • Posts: 2956
Re: Polling isn't getting worse
« Reply #11 on: June 03, 2018, 11:19:55 am »
You nudged me into looking into all the 'projections' and I'm starting to think we are in a fool's paradise when it comes to projections.

Everyone seems to be posting TWO poll trackers, neither of which seem 100% clear to me.  For example, why does the CBC tracker put the chance of an NDP majority high above that of a minority ?  And the other tracker is not done by a statistician.

I am starting to suspect we will see a BIG SURPRISE on election day.

PS: This may be the rebirth of intelligent 'public' discussion I have been waiting for, for 15-20 years or so.  I never suspected that the re-emergence of intelligent public discussion would start with error-prone lone guns and build on that but it makes sense.
The problem with these private firms is that their specific methods and algorithms are proprietary. If you believe in the free market then competition between them means that they all want to have that one magic formula that is the most accurate. They tend to be so.

In any case, all statistical projections are just that. The formula is accurate, it’s just that the formula includes a measure of unaccounted variance. We can never know how big that’s going to be. And as I said earlier, it’s a moving target. Polls are publish and people react to them, which then changes the results as soon as it’s announced. Each poll taken is simply a snapshot in time using the best prediction method available. They try to translate it into riding projections but it’s still fundamentally based on popular vote. Seats are won by popular vote, except on a riding by riding basis, whilst the polling typically contacts about 1200 people across different riding. Why’re pretty good at inferring ridings from that most of the time, but it’s obviously imperfect. Incumbents tend to be sticky, so they account for that. Start power also helps candidates, so they try to account for that. Voter fatigue can play a role, so they also try to account for that. Then there’s demographic composition of the riding and how peopemwith those characteristics usually vote. So there’s a lot they consider.

Offline Michael Hardner

  • Hero Member
  • *****
  • Posts: 12463
Re: Polling isn't getting worse
« Reply #12 on: June 03, 2018, 12:04:15 pm »
The problem with these private firms is that their specific methods and algorithms are proprietary. If you believe in the free market then competition between them means that they all want to have that one magic formula that is the most accurate. They tend to be so.

The CBC Tracker strategy is owned by a private firm ?

Quote
In any case, all statistical projections are just that. The formula is accurate, it’s just that the formula includes a measure of unaccounted variance. We can never know how big that’s going to be. And as I said earlier, it’s a moving target. Polls are publish and people react to them, which then changes the results as soon as it’s announced. Each poll taken is simply a snapshot in time using the best prediction method available. They try to translate it into riding projections but it’s still fundamentally based on popular vote. Seats are won by popular vote, except on a riding by riding basis, whilst the polling typically contacts about 1200 people across different riding. Why’re pretty good at inferring ridings from that most of the time, but it’s obviously imperfect. Incumbents tend to be sticky, so they account for that. Start power also helps candidates, so they try to account for that. Voter fatigue can play a role, so they also try to account for that. Then there’s demographic composition of the riding and how peopemwith those characteristics usually vote. So there’s a lot they consider.

Sure, but perfect is the opposite of good.  That is to say: just because predictions can't be perfect doesn't mean we should not talk about accuracy.  Such discussions are part of fomenting a reasonable public IMO, since these tools are used to rationalize policy.

Offline cybercoma

  • Full Member
  • ***
  • Posts: 2956
Re: Polling isn't getting worse
« Reply #13 on: June 03, 2018, 07:46:04 pm »
I’m not sure who CBC uses these days, but they usually contract these things out to private firms.

Offline Michael Hardner

  • Hero Member
  • *****
  • Posts: 12463
Re: Polling isn't getting worse
« Reply #14 on: December 26, 2020, 03:29:47 pm »
Old thread but I think it's a propos.

The polls AREN'T all right.

This is the listen of the year for me, and changed my mind.

https://fivethirtyeight.com/features/politics-podcast-how-to-make-polls-better/