NationStates Jolt Archive

Inspired by another thread, but this deserves it's very own.

My question is, if you take a "random sampling" of a population of people, ask their opinion, is it accurate to say that the results apply to the entire population? For example, a poll of 1,000 people being used to describe the opinions of 300,000,000 people.

My secondary question is, if so, do you believe in democratic processes like voting?

IF it is true that a random sampling of a tiny minority of the population is an accurate portrayal of the characteristics of the entirety of the population, THEN it must be true that if a random sample of only a tiny minority of the popuulation must be a legitimate and accurate way to determine the vote of the nation at large.

Thus, it would be possible and acceptable to randomly sample 1,000 people, have them vote, and then that would be the presidential election. Right?

60% of the time they're right every time.

60% of the time they're right every time.

Hmm I thought it was 55% of the time statistics were right... Maybe that was 55% of the time Santa Barbara was right. Damn cant keep track of all this statistical data. ;)

Hmm I thought it was 55% of the time statistics were right... Maybe that was 55% of the time Santa Barbara was right. Damn cant keep track of all this statistical data. ;)

I actually have a 55% chance of being 100% right 76.56% of the time.

Thus, it would be possible and acceptable to randomly sample 1,000 people, have them vote, and then that would be the presidential election. Right?
Let's do it!

It's all going to turn out bad anyway...why not make it cheaper?

Thus, it would be possible and acceptable to randomly sample 1,000 people, have them vote, and then that would be the presidential election. Right?

Depending on where you take the sample, you could ensure that either no Democrat, or no Republican was ever elected.

Hmm I thought it was 55% of the time statistics were right... Maybe that was 55% of the time Santa Barbara was right. Damn cant keep track of all this statistical data. ;)
I was only refering to the statistics for getting laid while wearing sex panther cologne.

Brian Fantana: Sex panther by Odion. It's made with real bits of panther so you know it's good. This stuff is illegal in 9 countries.
Ron Burgundy: Yes, it's quite pungent, it stings the nostrils... in a good way though. Brian, I'm gonna be honest with you: that stuff smells like pure gasoline.
Brian Fantana: They've done studies you know, they say 60% of the time, it works every time.
Ron Burgundy:That doesn't make any sense.

Inspired by another thread, but this deserves it's very own.

My question is, if you take a "random sampling" of a population of people, ask their opinion, is it accurate to say that the results apply to the entire population? For example, a poll of 1,000 people being used to describe the opinions of 300,000,000 people.

My secondary question is, if so, do you believe in democratic processes like voting?

IF it is true that a random sampling of a tiny minority of the population is an accurate portrayal of the characteristics of the entirety of the population, THEN it must be true that if a random sample of only a tiny minority of the popuulation must be a legitimate and accurate way to determine the vote of the nation at large.

Thus, it would be possible and acceptable to randomly sample 1,000 people, have them vote, and then that would be the presidential election. Right?


You've asked quite a few questions and then made a few logical leaps. What you are talking about is a survey. While there is alot of statistical information that can be culled from a survey, it needs a context. Surveys are often done in research in order to get a general idea about a subject, but without a fairly rigorous scoriing procedure and statistical analysis (which is alot more than just saying "these people agree with question x 10% more than these people) you have alot of information that means nothing.

Now, to the meat of the issue. A sample survey would be bad for deciding an election. In order to build a truely representative sample, you would need a screening process that would take months to complete and alot of money to support. It would require you to first discern how many individual groups there are in the electorate (democrats, republicans, greens, socialists, anarchaists, male/female/black/white/hispanic/asian/household earnings/education/etc/state/county/city for each individual group). Once you had figured out how many "cells" you have (i.e. the number of distinct groups that need to be represented), you would need to figure out their percentage of the general population so you could weight each cell's value. Once you had finished all of that, you would need to find enough individuals in each group to ensure some level of external validity (bare minimum to even be considered for publishing is 20). The sample size you ended up with would be much larger than 1000 people. The process would be complicated and unwieldy. Its easier, and cheaper, to just treat everyone as an individual and give them a vote. Trying to do a sample study to determine leaders instead of a simple vote is a classic example of a Goldbergian idea.

Depending on where you take the sample, you could ensure that either no Democrat, or no Republican was ever elected.
If you survey me and my 1,000 100%, REAL, GENUINE FAUX FRIENDS who have identical view points you would realize that the person who the American People really want for President is: Me.
Oddly enough, they also seem to want me as their State Senator, Mayor, Governor, Publicly Elected Judge, Sheriff, Local Representative and Dog Catcher. Normally I'd turn down such a wide array of powers to abuse, but if the people demand that I be given a private jet and a $5 Million a year salary (plus Health benefits and a personal Sex Slave) then I don't see how I could deny them and thwart the Democratic process.

You've asked quite a few questions and then made a few logical leaps. What you are talking about is a survey. While there is alot of statistical information that can be culled from a survey, it needs a context.

I agree it does, and it seems most newspapers and magazines and any other published, nonscientific articles that use results from surveys lack that context. They say "X amount of Americans are X" based off a survey of a minority. And then the readers go "OMG! That's a LOT of people who are X!" or "Whew! At least there are that many people who are X. Thank God, and thank the media for telling me."


Now, to the meat of the issue. A sample survey would be bad for deciding an election. In order to build a truely representative sample, you would need a screening process that would take months to complete and alot of money to support. It would require you to first discern how many individual groups there are in the electorate (democrats, republicans, greens, socialists, anarchaists, male/female/black/white/hispanic/asian/household earnings/education/etc/state/county/city for each individual group). Once you had figured out how many "cells" you have (i.e. the number of distinct groups that need to be represented), you would need to figure out their percentage of the general population so you could weight each cell's value. Once you had finished all of that, you would need to find enough individuals in each group to ensure some level of external validity (bare minimum to even be considered for publishing is 20). The sample size you ended up with would be much larger than 1000 people. The process would be complicated and unwieldy. Its easier, and cheaper, to just treat everyone as an individual and give them a vote. Trying to do a sample study to determine leaders instead of a simple vote is a classic example of a Goldbergian idea.

I suppose, but this doesn't really answer my need to know why its simpler to treat everyone as an individual for a political process, yet for published media sources and the general public it's simpler to treat everyone as an automaton whose opinion can be calculated based off a sample.

Depending on where you take the sample, you could ensure that either no Democrat, or no Republican was ever elected.
Psst thats why it has to be a truly random (well as close as you can get) sample of the entire population

If you have selection bias you have statistical bias

But on that note is our system not already something simmilar with a larger sample?

<snip>
why its simpler to treat everyone as an individual for a political process, yet for published media sources and the general public it's simpler to treat everyone as an automaton whose opinion can be calculated based off a sample.
Because polls are taken to make news on days when none exists.

Yes, but 23.43% of all statistics are made up on the spot, and 89.4% of the time, statistics are just ways of proving things. Remember, you can use facts to prove anything that's even remotely true.

Yes, but 23.43% of all statistics are made up on the spot, and 89.4% of the time, statistics are just ways of proving things. Remember, you can use facts to prove anything that's even remotely true.
Ha not going to be fooled twice ... simpsons :) lol

Psst thats why it has to be a truly random (well as close as you can get) sample of the entire population

If you have selection bias you have statistical bias

But on that note is our system not already something simmilar with a larger sample?

Shhh. We only sample people who actually care to vote. A self-selecting sample.

There would be a lot of argument about what was a truly random sample.

A properly done survey is accurate. A survey of 1,000 people yields a margin of error of +/-3%. The size of the population does not matter because it simply isnt one of the variables that goes into the formula for determining margin of error.

Doing a survey to elect leaders would result in the election of a leader who is more representative of what the average person wants than someone elected in a traditional optional voting system. However, it brings up a lot of problems. I wont bring up all of the problems because I dont want to turn this into a big post. I will simply mention the biggest one, perception. Even though a well educated person will be well aware that the leader chosen accurately reflects the desires of the population 97% of the time, most people dont have even a basic understanding of statistics. Even if you educate people about the process, most people will still feel like they are not a part of the electoral process. Having a population that feels like they are marginalized (not a part of the process) is the worst thing that could happen to a democracy.

If you wanted the added accuracy and other benefits that elections by survey give you without the major drawback of having people feel marginalized, the better option is to implement mandatory voting laws.

Shhh. We only sample people who actually care to vote. A self-selecting sample.

There would be a lot of argument about what was a truly random sample.
Yes agreed

... the statistics minor in me does not like the stats bias in the sample

I would rather see a more stisticaly fair sample with the conclusion adjusted then an initial bias sample

But thats the stats in me talking

Inspired by another thread, but this deserves it's very own.

My question is, if you take a "random sampling" of a population of people, ask their opinion, is it accurate to say that the results apply to the entire population? For example, a poll of 1,000 people being used to describe the opinions of 300,000,000 people.

It depends on how well the survey is done.

Most statistical analyses are based on the assumption that the data is a "Simple Random Sample" of the population - the fact that every person in the population has an exactly equal chance of being selected for the sample.

This is actually really difficult. How do you go about obtaining random samples? If you stand on the street and ask the first 1000 people to pass by, you're clearly not getting a random sample at all - you'll only get people that happen to live near that street corner. Not a simple random sample.

Most (reputable) polling companies go the route of making phone calls - basically, pick a random phone number to get a household, and pick a random person from the household... which can get pretty close to a simple random sample, except that it's getting less reliable now that people have multiple phone numbers, and it's skewed anyway by the fact that households have different numbers of people.

A foolproof way of getting a good sample, of course, is to simply have a list of every person in the population and draw names out of a hat :-p. Anyhow, there are a lot of ways of getting pretty good samples, but almost no way of getting a perfect random sample... and as long as you don't have one of those, there's always the chance that the results could be off in a systematic way that can't be compensated for by calculations.

My secondary question is, if so, do you believe in democratic processes like voting?

IF it is true that a random sampling of a tiny minority of the population is an accurate portrayal of the characteristics of the entirety of the population, THEN it must be true that if a random sample of only a tiny minority of the popuulation must be a legitimate and accurate way to determine the vote of the nation at large.

Thus, it would be possible and acceptable to randomly sample 1,000 people, have them vote, and then that would be the presidential election. Right?

It will be a pretty good approximation of the presidential election, yes. It will have some error built in from the fact that it's a sample and not a comprehensive survey - polls do (or, at least, should) come with a statement of what the 95% confidence interval is, and it'll be generally within 3-4% for a 1000-2000 size sample.

But think about that for a moment. A 95% confidence interval of +/-3% means that "there's a 95% chance that the true value for the population is within 3% of the value that our sample gave." That means there's a 5% chance that the true value is over 3% away from the value the survey gives. And a very high chance that the survey is off from the true value by 1-2% or so, no matter how good the survey is. (And that's already assuming that the sample has ONLY the error that comes from it being a sample, and assuming the sample is perfectly random with no bias.)

All that said, a survey simply is too inaccurate for an election. Even if it was done well, there would be a high enough margin of error so that close elections (ones won by less than five percent or so) would always be in doubt.

And even neglecting that, there's the issue of selecting a sample. How would you make sure that it's unbiased? It's pretty hard to do even when there isn't any partisan pressure. Imagine how much controversy there would ALWAYS be about the selection of a sample.

"WHAT?!?! They included X number of people from minority Y! That's not representative of the poplulation! We need to redo the sample!"


To summarize - a poll can be a pretty good estimation of what people think, but only if done very carefully; and I give this carefulness almost NO chance of standing up to political pressure. A poll is only as good as it is unbiased, and it is only unbiased so far as the people running it don't have any vested interest in the outcome; and while it is possible for reputable polling companies to have no vested interest in a lot of polls they do, I don't think it is possible for ANYONE to not have a vested interested in the presidential election, and hence it would be a very bad idea to decide something that important by a poll instead of the more traditional method of having EVERYONE who wants to vote.

My question is, if you take a "random sampling" of a population of people, ask their opinion, is it accurate to say that the results apply to the entire population? For example, a poll of 1,000 people being used to describe the opinions of 300,000,000 people.

yeah, pretty much. assuming that the sample really was random, it will almost certainly be pretty close to what you'd get if you got a full survey of everybody. you'd probably miss at least some minority opinions, of course (i know that there are thousands of anarchists in the united states, but we never show up in polls), but that's because those minority opinions are essentially the equivalent of statistical noise in the population as a whole. though there is always a slight chance that your random sample happened to turn up a bizarrely distorted sample by sheer dumb luck.

My secondary question is, if so, do you believe in democratic processes like voting?

when necessary. more of a 'rationally discuss and compromise' consensus kind of guy, myself.

IF it is true that a random sampling of a tiny minority of the population is an accurate portrayal of the characteristics of the entirety of the population, THEN it must be true that if a random sample of only a tiny minority of the popuulation must be a legitimate and accurate way to determine the vote of the nation at large.

which is why we use exit polls to determine whether election results are legitimate in sketchy situations - in countries that aren't the u.s., anyway.

Thus, it would be possible and acceptable to randomly sample 1,000 people, have them vote, and then that would be the presidential election. Right?

except for that whole thing about everybody getting to have roughly equal say in such matters, yes. no less possible or acceptable than any of the various other methods for choosing leaders. but lots of people do kind of like the 'roughly equal say' thing, so they probably wouldn't find it particularly acceptable.

what would be better than that is using a random sample of the population for the legislature. not to vote for the legislature. to be the legislature.

I agree it does, and it seems most newspapers and magazines and any other published, nonscientific articles that use results from surveys lack that context. They say "X amount of Americans are X" based off a survey of a minority. And then the readers go "OMG! That's a LOT of people who are X!" or "Whew! At least there are that many people who are X. Thank God, and thank the media for telling me."



I suppose, but this doesn't really answer my need to know why its simpler to treat everyone as an individual for a political process, yet for published media sources and the general public it's simpler to treat everyone as an automaton whose opinion can be calculated based off a sample.


.....because published media sources are buisnesses that aim to make money through entertainment? Apples and oranges.

"Are statistics really accurate?"

Statistics don't lie, but statisticians surely do.

Actually 25% of Americans think that statistics lie and 47% say statistics don't lie. The remaining 28% either said "Fuck off," or threw things at the pollsters. :D

A properly done survey is accurate. A survey of 1,000 people yields a margin of error of +/-3%. The size of the population does not matter because it simply isnt one of the variables that goes into the formula for determining margin of error.

Thats something of an obfuscation. Standard error is the square root of (population variance/number of subjects). While the overall population from which the sample is derived does not factor into the actual equation, it does factor into the logic behind it. Why does the standard error decrease as the number of subjects in your sample increases? Because the more subjects you have, the smaller the chance that your sample is a statistical oddity because it represents a larger portion of the general population.

I agree that a properly done survey is accurate, I just don't feel that most of the surveys you see in this country count as "properly done." Leading questions are the norm, badly written form responses are everywhere, and the less said about sampling procedures the better (the vast majority of studies you see coming out of university labs today have a population pool consisting solely of individuals taking an introduction psych course participating for course credit).


Doing a survey to elect leaders would result in the election of a leader who is more representative of what the average person wants than someone elected in a traditional optional voting system. However, it brings up a lot of problems. I wont bring up all of the problems because I dont want to turn this into a big post. I will simply mention the biggest one, perception. Even though a well educated person will be well aware that the leader chosen accurately reflects the desires of the population 97% of the time, most people dont have even a basic understanding of statistics. Even if you educate people about the process, most people will still feel like they are not a part of the electoral process. Having a population that feels like they are marginalized (not a part of the process) is the worst thing that could happen to a democracy.

If you wanted the added accuracy and other benefits that elections by survey give you without the major drawback of having people feel marginalized, the better option is to implement mandatory voting laws.

Wow, that was painfully populist. The US was designed not to give "power to the people" but to protect individual liberty. Our government isn't there to "reflect the desires of 97% of the population," its there to hold tyranny at bay, especially to protect that 3% from the other 97%. Mandatory voting is a bad option as well. Not only do you criminalize consensual behavior (and accept all of the costs associated with enforcement) but you end up with people who don't know about the issues and don't care flocking to the polls. We have enough of that crap already.

*snip*

Thank you, I was just about to break out the old Stats textbook and try to explain it properly. You did it better, now I don't have to, whew!

"Are statistics really accurate?"

Statistics don't lie, but statisticians surely do.

Actually 25% of Americans think that statistics lie and 47% say statistics don't lie. The remaining 28% either said "Fuck off," or threw things at the pollsters. :D
Now that's my type of pole! :D

Wow, that was painfully populist. The US was designed not to give "power to the people" but to protect individual liberty.
Elections exist to give people the power to elect their leaders. I am only talking about the electoral process here, I am not concerned with the constitution and protection of individual rights in this post which is only about the electoral process, which is entirely about allowing people to elect their leaders.

Mandatory voting is a bad option as well. Not only do you criminalize consensual behavior (and accept all of the costs associated with enforcement) but you end up with people who don't know about the issues and don't care flocking to the polls. We have enough of that crap already.

I dont support mandatory voting laws either, I was simply stating that mandatory voting laws give the advantages (or disadvantages, depending on how you view it) that a survey system does without marginalizing anyone. I wasnt saying it would be better than what currently exists, but that it would be better than a survey based system.

Now that's my type of pole! :D
Heh! Is that anything like "pole-dancing?" :D

or there is 3 types of people in the world, people who can do math, and people who can't.

.....

Statistics don't lie, but statisticians surely do.


......
Damn! And there was me thinking a large part of my job was telling users of my statistics to stop ****ing twisting them to their own purpose!

There are lies, damn lies, and the spin put on statistics by marketing and politics.

Thats something of an obfuscation. Standard error is the square root of (population variance/number of subjects). While the overall population from which the sample is derived does not factor into the actual equation, it does factor into the logic behind it. Why does the standard error decrease as the number of subjects in your sample increases? Because the more subjects you have, the smaller the chance that your sample is a statistical oddity because it represents a larger portion of the general population.

.......
Not quite. It depends what formula you're using. When sampling from small populations, the proportion sampled is factored in to the calculation of the standard population. When sampling from a large population, the proportion is assumed to approximate to zero.[/PEDANTRY]

A properly done survey is accurate. A survey of 1,000 people yields a margin of error of +/-3%. The size of the population does not matter because it simply isnt one of the variables that goes into the formula for determining margin of error.Not sure where you got that math from. If your population is also 1,000 people, your margin of error is +/- 0.00000%


Doing a survey to elect leaders would result in the election of a leader who is more representative of what the average person wants than someone elected in a traditional optional voting system. However, it brings up a lot of problems. I wont bring up all of the problems because I dont want to turn this into a big post. I will simply mention the biggest one, perception. Even though a well educated person will be well aware that the leader chosen accurately reflects the desires of the population 97% of the time, most people dont have even a basic understanding of statistics. Even if you educate people about the process, most people will still feel like they are not a part of the electoral process. Having a population that feels like they are marginalized (not a part of the process) is the worst thing that could happen to a democracy.
The great bulk of the population is marginalized.

If you wanted the added accuracy and other benefits that elections by survey give you without the major drawback of having people feel marginalized, the better option is to implement mandatory voting laws.And what should be done with those like me who would refuse to vote?

<snip>

Spot on. To which I can only add that the act of sampling in this context would itself be likely to introduce bias in the sample. A decision would need to be made whether or not to tell those to be sampled that they have been selected for the sample.

If people do not know in advance, and the chance of being selected is something like one in a several thousand or more, then, for most people, the holding of a political position becomes something of an academic exercise. The chances are it will never have any practical impact, so motivation to research or think through personal politics will be reduced, and so people are likely to hold different views to a more fully considered position.

If those to be sampled do know in advance, they are likely to think a lot more about how they will vote, and may well come to a more considered (read rationalised) postion than otherwise. (A similar bias is observed in focus groups, where the fact that participants know that their views are having a disproportionate impact tends to lead to them moving away from the general position of the populaiton they are supposed to represent.)

Not sure where you got that math from. If your population is also 1,000 people, your margin of error is +/- 0.00000%

....
You're both right. There are two forms of the formula for determining the (so-called) 'margin of error'. Implicitly, you're using the maths for small populations; whereas LazyHippies is using that for large populations, where the proportion sampled is removed from the equation.

Not sure where you got that math from. If your population is also 1,000 people, your margin of error is +/- 0.00000%

If the population were 1,000 people, then that would be a census which is an entirely different animal. We are talking about the electoral process here, we already know that there are more than 1,000 in the population. In fact, there are more than a million.

Inspired by another thread, but this deserves it's very own.

My question is, if you take a "random sampling" of a population of people, ask their opinion, is it accurate to say that the results apply to the entire population? For example, a poll of 1,000 people being used to describe the opinions of 300,000,000 people.

My secondary question is, if so, do you believe in democratic processes like voting?

IF it is true that a random sampling of a tiny minority of the population is an accurate portrayal of the characteristics of the entirety of the population, THEN it must be true that if a random sample of only a tiny minority of the popuulation must be a legitimate and accurate way to determine the vote of the nation at large.

Thus, it would be possible and acceptable to randomly sample 1,000 people, have them vote, and then that would be the presidential election. Right?
No, I don't think they are accurate, nor objective for that matter.
I used to ponder when I saw these population maps of Bosnia in an old National Geographic: before and after lines of control in 1995. They were on the same scale, but, as you'd expect, the top one was a mosaic, while the bottom one was clear-cut (more or less).
Anyway, I kept looking at the top one and wondering: Bosnia is really small, and the map is really intricate - would I be right to assume that the colours are given by the likes of several families? And don't we see here the intrinsic error: were you to change the scale or the grid, wouldn't you get completely different colour arrangements?

My question is, if you take a "random sampling" of a population of people, ask their opinion, is it accurate to say that the results apply to the entire population? For example, a poll of 1,000 people being used to describe the opinions of 300,000,000 people.

No random sample is ever perfect, but for any given population we can identify the margin of error on random samples of various sizes. So it is always a question of "how good", and never "exactly right." Generally speaking, a sample of 1,000 is pretty good.

However, this sample is only representative of the population it represents. There are all sorts of decisions that go into how to obtain the sample... and I doubt we will ever really be satisfied that a particular sampling method actually represents the entire United States (for instance).