Data collection and sampling, Study notes of Data Mining

Download Data collection and sampling and more Study notes Data Mining in PDF only on Docsity!

Data Collection and Sampling

OPRE 6301

Recall...

Statistics is a tool for converting data into information:

Data

Statistics

Information

But where then does data come from? How is it gath- ered? How do we ensure its accurate? Is the data reliable? Is it representative of the population from which it was drawn? We now explore some of these issues.

Surveys...

A survey solicits information from people; e.g. Gallup polls; pre-election polls; marketing surveys.

The Response Rate (i.e. the proportion of all people selected who complete the survey) is a key survey para- meter.

Surveys may be administered in a variety of ways, e.g.

• Personal Interview,
• Telephone Interview, and
• Self-Administered Questionnaire.

Questionnaire Design

Over the years, a lot of thought has been put into the science of the design of survey questions. Key design principles:

1. Keep the questionnaire as short as possible.
2. Ask short, simple, and clearly worded questions.
3. Start with demographic questions to help respondents get started comfortably.
4. Use dichotomous (yes | no) and multiple choice ques- tions.
5. Use open-ended questions cautiously.
6. Avoid using leading-questions.
7. Pretest a questionnaire on a small number of people.
8. Think about the way you intend to use the collected data when preparing the questionnaire.

Sampling Plans...

A sampling plan is just a method or procedure for spec- ifying how a sample will be taken from a population.

We will focus our attention on these three methods:

• Simple Random Sampling,
• Stratified Random Sampling, and
• Cluster Sampling.

Details...

Simple Random Sampling...

A simple random sample is a sample selected in such a way that every possible sample of the same size is equally likely to be chosen.

Drawing three names from a hat containing all the names of the students in the class is an example of a simple random sample: any group of three names is as equally likely as picking any other group of three names.

Stratified Random Sampling...

A stratified random sample is obtained by separating the population into mutually exclusive sets, or strata, and then drawing simple random samples from each stra- tum.

Strata 1 : Gender

Male

Female

Strata 2 : Age

Strata 3 : Occupation

professional

clerical

blue collar

other

We can acquire about the total population, make inferences within a stratum or make comparisons across strata

After the population has been stratified, we can use sim- ple random sampling to generate the complete sample:

If we only have sufficient resources to sample 400 people total, we would draw 100 of them from the low income group…

…if we are sampling 1000 people, we’d draw 50 of them from the high income group.

Sample Size...

This is an important issue. Numerical techniques for de- termining sample sizes will be described later, but suffice it to say that the larger the sample size is, the more ac- curate we can expect the sample estimates to be.

Sampling and Non-Sampling Errors...

Two major types of error can arise when a sample of observations is taken from a population: sampling error and non-sampling error.

Sampling error refers to differences between the sample and the population that exist only because of the obser- vations that happened to be selected for the sample.

Non-sampling errors are more serious and are due to mistakes made in the acquisition of data or due to the sample observations being selected improperly.

Details...

Non-Sampling Error...

Non-sampling error are more serious and are due to mistakes made in the acquisition of data or due to the sample observations being selected improperly.

There are three types of non-sampling errors:

• Errors in data acquisition,
• Nonresponse errors, and
• Selection bias.

Increasing the sample size will not reduce this type of error.

Details...

Errors in Data Acquisition

... arises from the recording of incorrect responses, due to:

— incorrect measurements being taken because of faulty equipment, — mistakes made during transcription from primary sources, — inaccurate recording of data due to misinterpretation of terms, or — inaccurate responses to questions concerning sensitive issues.

Selection Bias

... occurs when the sampling plan is such that some members of the target population cannot possibly be se- lected for inclusion in the sample.