SURVEY METHODOLOGY AND DATA SOURCES, Schemes and Mind Maps of Business

The margin of sampling error for this study overall is +/- 2.61% (95% confidence level). Phone: Within the phone portion of this study, BRI ...

Typology: Schemes and Mind Maps

2022/2023

Uploaded on 02/28/2023

ekadant
ekadant 🇺🇸

4.3

(32)

267 documents

1 / 5

Toggle sidebar

This page cannot be seen from the preview

Don't miss anything!

bg1
SURVEY METHODOLOGY AND DATA SOURCES
The Pennsylvania K–12 & School Choice Survey project, funded by EdChoice, developed by EdChoice and
Commonwealth Foundation, and conducted by Braun Research Incorporated (BRI), interviewed adults (age
18+) who are Registered Voters in the State of Pennsylvania. A total of 1,407 interviews were conducted using an
online methodology from February 23, 2020 through March 11, 2020. The total N size includes a statewide base
and oversamples in Allegheny County, the City of Philadelphia, and Harrisburg/Dauphin County. The margin of
sampling error for this study overall is +/- 2.61% (95% confidence level).
Phone:
Within the phone portion of this study, BRI conducted a total of n=137 telephone interviews, from February 23, 2020
through March 11, 2020, by means of both landline and cell phone.
The margin of sampling error for the entire phone sample (N=137) of interviews is +/- 8.37% (95% confidence level).
BRI’s live callers conducted all phone interviews.
• Adult Voters 18+ Respondents:
o 19,846 in total – 10,976 landline; 8,870 cell
o Of these calls 6,483 [3,574 landline, 2,909 cell] were unusable phone numbers
(disconnected, fax, busy, non-residential, or non-answers, etc.);
o 13,086 [7,523 landline, 5,833 cell] were usable numbers but eligibility unknown
(including refusals and voicemail); and
o 34 [15 landline, 19 cell] people did not complete the survey.
o The average response rate of the landline interviews was 1.1%.
o The average response rate of the cell phone interviews was 0.5%.
Details on sample dispositions, landline, and cell phone response rates, and weighting are discussed in following
sections.
Sample Design
A combination of landline and cellular random digit dial (RDD) samples was used to represent Adults 18+ in Dauphin
County/Harrisburg, and in the ‘Eastern region’ of Pennsylvania, who have access to either a landline or cellular
telephone. Both samples were provided by Dynata (formerly Survey Sampling International, LLC, or SSI) according
to BRI specifications.
Dynata starts with a database of all listed telephone numbers, updated on a four- to six-week rolling basis, 25
percent of the listings at a time. All active blocks—contiguous groups of 100 phone numbers for which more than
one residential number is listed—are added to this database. Blocks and exchanges that include only listed business
numbers are excluded.
pf3
pf4
pf5

Partial preview of the text

Download SURVEY METHODOLOGY AND DATA SOURCES and more Schemes and Mind Maps Business in PDF only on Docsity!

SURVEY METHODOLOGY AND DATA SOURCES

The Pennsylvania K–12 & School Choice Survey project, funded by EdChoice, developed by EdChoice and

Commonwealth Foundation, and conducted by Braun Research Incorporated (BRI), interviewed adults (age

18+) who are Registered Voters in the State of Pennsylvania. A total of 1,407 interviews were conducted using an

online methodology from February 23, 2020 through March 11, 2020. The total N size includes a statewide base

and oversamples in Allegheny County, the City of Philadelphia, and Harrisburg/Dauphin County. The margin of

sampling error for this study overall is +/- 2.61% (95% confidence level).

Phone:

Within the phone portion of this study, BRI conducted a total of n=137 telephone interviews, from February 23, 2020

through March 11, 2020, by means of both landline and cell phone.

The margin of sampling error for the entire phone sample (N=137) of interviews is +/- 8.37% (95% confidence level).

BRI’s live callers conducted all phone interviews.

  • Adult Voters 18+ Respondents:

o 19,846 in total – 10,976 landline; 8,870 cell

o Of these calls 6,483 [3,574 landline, 2,909 cell] were unusable phone numbers

(disconnected, fax, busy, non-residential, or non-answers, etc.);

o 13,086 [7,523 landline, 5,833 cell] were usable numbers but eligibility unknown

(including refusals and voicemail); and

o 34 [15 landline, 19 cell] people did not complete the survey.

o The average response rate of the landline interviews was 1.1%.

o The average response rate of the cell phone interviews was 0.5%.

Details on sample dispositions, landline, and cell phone response rates, and weighting are discussed in following

sections.

Sample Design

A combination of landline and cellular random digit dial (RDD) samples was used to represent Adults 18+ in Dauphin

County/Harrisburg, and in the ‘Eastern region’ of Pennsylvania, who have access to either a landline or cellular

telephone. Both samples were provided by Dynata (formerly Survey Sampling International, LLC, or SSI) according

to BRI specifications.

Dynata starts with a database of all listed telephone numbers, updated on a four- to six-week rolling basis, 25

percent of the listings at a time. All active blocks—contiguous groups of 100 phone numbers for which more than

one residential number is listed—are added to this database. Blocks and exchanges that include only listed business

numbers are excluded.

Numbers for the landline sample were drawn with equal probabilities from active blocks (area code + exchange +

two-digit block number) that contained three or more residential directory listings. The cellular sample was not

list-assisted but was drawn through a systematic sampling from dedicated wireless 100-blocks and shared service

100-blocks with no directory-listed landline numbers.

Contact Procedures

Interviews were conducted from February 23, 2020 through March 11, 2020. As many as eight attempts were

made to contact every sampled telephone number. Sample was released for interviewing in replicates, which are

representative subsamples of the larger sample. Using replicates to control the release of sample ensures that

complete call procedures are followed for the entire sample. Calls were staggered over times of day and days of

the week to maximize the chance of contact with potential respondents. Each phone number received at least one

daytime call.

BRI has noticed over the last several years, response rates have been declining for consumer polls. Generally, running

survey over a longer period of time will boost these response rates. However, lower response rates do not lead to

lower reliability of the data.

The margin of error is the largest 95% Confidence Interval for any estimated proportion based on this sample – is

8.37%. This means that in 95 of every 100 samples drawn using the same methodology, estimated proportions based

on the entire sample will be no more than 8.37 percentage points away from their true values in the population.

It is critical to note that the Mean Squared Error (MSE) is higher when considering the number of respondents for

a given demographic subgroup. For example, the MSE for a subgroup of 150 respondents is ± 8.0 percentage points.

In addition to sampling error, question wording, ordering, and other practical difficulties when conducting surveys

may introduce error or bias into the findings of public opinion research.

Call Dispositions and Response Rates

Full dispositions for all sampled landline and cell phone numbers are located on the following pages.

A total of N=1,407 Registered Pennsylvania Voters aged 18+ participated in the online survey relevant to and

targeted toward this group from an initial outreach of 3,609 persons emailed from 23 February – 11 March 2020.

These individuals were randomly selected from the opt-in nonprobability online pool of panelists; n=619 persons

terminated as disqualified.

For the online part of this study BRI utilized their partner Fulcrum to recruit and complete online. In turn Fulcrum

works with their various partners to send sample. More information about Fulcrum may be found via the following

web site: https://luc.id/fulcrum

Weighting Procedures and Analysis

Weighting is generally used in survey analysis to compensate for sample designs and patterns of non-response that

might bias results.

** BRI weighted this study on the following factors: Age, Race, Ethnicity, Income, Gender, County, Community Type,

and Party ID.

Online

Please note that Paul Braun has written his perspective on the current environment of online surveys and how they

are being employed more and more. A ‘sampling’ of that text is below.

Online Studies

Online surveys offer unique challenges to ensure quality and honesty. For most surveys, email blasting to panelists are

unchecked and depend on the first responders to complete a survey. Further, there are abuses in many panel companies

where people sign up under multiple names, and even live in other countries!

The established procedures being used for the best possible results in online surveys are always being reviewed. There

are several companies where panelists are drawn from random telephone samples and recruited to participate in periodic

surveys for cash and prizes. Our concerns include the proportional sample balance, number of surveys a respondent stops/

starts, frequency of respondent solicitation, and frequency of the panel company to draw upon competitor databases to

complete low incidence projects.

Unlike telephone surveys, online surveys generally attract more males than females. Further, the number of respondents

over the age of 60 are disproportionally thin versus their population. Some panel companies have achieved some balance

by providing seniors with a limited action internet-ready device at the panel company’s expense. Still, for general

population surveys, seniors are solicited repeatedly far more than their younger counterparts.

Over the last several years conversations have been started through the media and, indeed, within national conferences

(such as that of AAPOR) regarding putting the science behind online data collection, though there are no immediate

plans fully to support projecting studies collected online being presented in the media.

Many leading media and corporate entities do not recognize online surveys as a projectable method of data collection,

but some online surveys compare nicely to similar telephone surveys. Braun Research is an active participant in these

discussions to help mark the way forward.

No one panel directly controls all panelized respondents. More than half of all respondents are members of multiple

panels, and, typically, on average, six percent of that population will be panelized. That population is generally younger,

more connected, slightly less than average income and has other underlying differences (slightly higher unemployed/

uninsured) and more liberal.

ABOUT BRAUN RESEARCH, INC. (BRI)

The Braun Research network of companies, founded in 1995 , engages in data collection via telephone, and

internet for various survey research firms, government and advertising agencies, local community organizations,

local and national business groups, foundations, universities and academic entities, as well as religious organizations.

In twenty-five (25) years Braun Research has conducted over 11,000 research projects by telephone, internet, and

mail worldwide.

Braun Research is a well-respected firm employing techniques and standards approved by various survey research

academic organizations and other affiliations including those with whom Braun is an active member, including

AAPOR (The American Association for Public Opinion Research).

Nationally-known research firms have hired Braun Research, including the Gallup Organization, the Pew Research

Center, the Eagleton Poll, Mathematica Policy Research, and the Washington Post. Braun Research has worked for

the New Jersey Department of Health and Human Services, as well as other government agencies including the

United States Departments of the Treasury and Defense, and the Center for Disease Control.

The work Braun Research accomplishes for other research firms requires them to perform all work up to standards

required by the various research organizations where Braun Research enjoys membership and, in some cases,

participate actively. Paul Braun is recognized as a leader in the field by colleagues who asked him to serve on these

committees. He has served as President of the New Jersey Chapter of AAPOR, and he has been a member of the

International Association for the Measurement and Evaluation of Communication (AMEC) in North America.