



















































































Study with the several resources on Docsity
Earn points by helping other students or get them with a premium plan
Prepare for your exams
Study with the several resources on Docsity
Earn points to download
Earn points by helping other students or get them with a premium plan
Section 14.2 contains information about the sources of country sampling frames and their coverage of the target population.
Typology: Study notes
1 / 91
This page cannot be seen from the preview
Don't miss anything!




















































































Leyla Mohadjer, Tom Krenzke and Wendy Van de Kerckhove, Westat
This chapter presents information about the PIAAC Main Study sample design and selection results. Participating countries were required to develop their sample design and selection plans according to the standards provided in the PIAAC Technical Standards and Guidelines (TSG) and to submit their plans to the Consortium for approval. The sample design plans included information about sampling frames and their coverage, providing descriptions of the national sample designs that included stages of sampling, probabilities of selection, sampling units and sample sizes. The sample selection plans included detailed information about the processes for sample selection at each stage of sampling. In addition, the countries were required to complete and submit quality control sample selection forms to the Consortium to verify that the sample selection was conducted in an unbiased and randomized way consistent with PIAAC standards.
The target population for PIAAC consists of all noninstitutionalized adults between age 16 and 65 (inclusive) who reside in the country (meaning their usual place of residency is in the country) at the time of data collection. Countries were allowed to expand the target population to include additional subpopulations of interest to the country as long as they followed the TSG on such supplementation. Section 14.1 provides more detail on the PIAAC target population and the national target populations if expanded beyond the PIAAC standard definition. Section 14. contains information about the sources of country sampling frames and their coverage of the target population.
The TSG allowed each country to choose a sample design and selection approach that is most optimal and cost effective as long as the design applies full selection probability methods to select a representative sample from the PIAAC target population. Descriptions of the standard PIAAC and national sample designs and probabilities of selection are given in section 14.3. The definition of sampling units and sample selection methods are provided in section 14.4. Section 14.5 contains the PIAAC target sample sizes and describes the process applied to determine the initial sample sizes. Sample selection results and a summary of the sampling quality control procedures are given in section 14.6 and section 14.7, respectively. Finally, section 14.8 provides a brief description of the incentive plans for PIAAC.
A clear and precise definition of the target population is necessary to ensure that the population of interest is adequately covered by each participating country and to maintain consistency and comparability across countries. The PIAAC target population consists of all noninstitutionalized adults between age 16 and 65 (inclusive) who reside in the country (usual place of residency is in
the country) at the time of data collection. Adults were to be included regardless of citizenship, nationality or language (standard 4.1.1). The target population excludes adults in institutional collective dwelling units (or group quarters) such as prisons, hospitals and nursing homes, as well as adults residing in military barracks and military bases. However, full-time and part-time members of the military who do not reside in military barracks or military bases are included in the target population.
Adults in other noninstitutional collective dwelling units (or group quarters), such as workers’ quarters or halfway homes, are also included in the target population. This includes adults living at school in student group quarters such as a dormitory, fraternity or sorority. Adults who were unable to complete the assessment because of a hearing impairment, blindness/visual impairment or physical disability are considered in scope; however, they were excluded from PIAAC response rate calculations because the assessment does not accommodate such situations.
The target population does not cover the entire geography area for the following countries:
Some countries expanded the target population to include additional subpopulations of interest to the country. These country-specific supplemental samples, approved by the Consortium, are presented in Table 14-1 below.
Table 14-1: Country-specific samples
Country Specific samples Australia Persons aged 15 and 66- 74 Denmark PISA 2000 survey respondents aged 26- 27
Some countries elected to oversample portions of the target population. The oversamples approved by the Consortium are presented in Table 14-2 below.
(^1) Please refer to notes A and B regarding Cyprus in the Note to Readers section of this report.
(^2) Please refer to the note regarding the Russian Federation in the Note to Readers section of this report.
However, some countries’ lists of residents used for the study did not completely cover the PIAAC target population (e.g., the lists may have excluded nonnationals/noncitizens), complicating their use as a sampling frame. See Table 14-3 for the full list of sampling frames employed by countries with population registry samples.
Table 14-3: Sampling frames for countries with population registry samples
Country
Sampling frame Stage 1 Stage 2 Stage 3 Austria Population registry, 2011 Denmark Population registry, 2011 Estonia Population registry, 2011 Finland Statistics Finland’s population database (based on the Central Population Register), 2011 Flanders (Belgium) Population registry, 2011 Germany German Census Bureau frame of communities, 2011
Local population registries, 2011
Italy National Statistical Institute of Italy frame of municipalities, 2011
Household registries held by municipalities, 2011
Population registries, 2011; combined with field enumeration Japan Resident registry, 2011 Resident registry, 2011 Netherlands Population registry, 2011 Norway Population registry, 2011 Poland Population registry, 2011 Population registry, 2011 Slovak Republic Population registry, 2011 Population registry, 2011 Spain Population registry, 2011 Population registry, 2011 Sweden Population registry, 2011 indicates there is no such stage in the country’s sample design.
Some countries have access to master samples used for national surveys. For example, Australia has a master sample of dwelling units (DUs) already in use by governmental surveys that was also used for PIAAC. Similarly, Australia and France have master samples of area primary sampling units (PSUs). See Table 14-4 for more information on how master samples were employed by participating countries.
Table 14-4: Sampling frames for countries using master samples
Country
Sampling frame Stage 1 Stage 2 Stage 3 Stage 4 Australia Bureau of Statistics population survey master sample, 2006
Bureau of Statistics population survey master sample, 2006
Bureau of Statistics population survey master sample, 2006
Field enumeration
France Master sample from census data file, 1999
Individual taxation file, 2011
For multistage area sample designs in which a registry is not being used, listing procedures are necessary to create a frame of households within the selected geographic clusters. A frame of geographic clusters can be formed by combining adjacent geographic areas, respecting their population sizes and taking into consideration travel distances for interviewers. Table 14- contains sampling frames for the remaining countries without registries using area sample designs for PIAAC. The exception is that Cyprus^3 is included in Table 14-5 among the countries without population registries, even though it did not use an area sample design, Cyprus did not require listing procedures because its sample frame for the first stage was a list of households from the Statistical Service Census 2001, updated with information from the 2010 Electricity Authority Household Registry.
(^3) Please refer to notes A and B regarding Cyprus in the Note to Readers section of this report.
considerations such as excluding persons in hard-to-reach areas. The Consortium asked that each country identify to the extent possible exclusions before sample selection. Adjustments for any noncoverage of the target population in each country was made through benchmarking during the weighting process (see Chapter 15). A complete list of exclusions for countries using population registries is presented in Table 14-6; Table 14-7 includes a similar list for countries not using population registries.
In addition to PIAAC eligible persons not included in sampling frames, persons that were included in the frame but in practice were impossible to be interviewed were treated as exclusions conditional on the total exclusion rate staying at or below 5%. Chapter 16 provides more information about this group, with Table 16-2 showing the overall exclusion rate for each country.
Table 14-6: Portion of target population not covered by Main Study sampling frames for countries using population registries
Country
Percentage of target population not covered Group not covered* Austria 0.6% Undocumented immigrants Denmark < 0.1% Undocumented immigrants Estonia 2.8%+ Persons without a detailed address; undocumented immigrants (no estimate provided) Finland 0.2% Undocumented immigrants; asylum seekers Flanders (Belgium) 1.0% Undocumented immigrants Germany 0.5% Undocumented immigrants Italy 0.8%+ Adults in noninstitutional group quarters; undocumented immigrants (no estimate provided) Japan 2.2% Nonnationals; undocumented immigrants Netherlands 0.9% Undocumented immigrants Norway 0.4% Undocumented immigrants Poland 0.8% Foreigners staying in Poland fewer than 3 months; nonregistered immigrants Slovak Republic 0.1% Undocumented immigrants Spain 0.0% None Sweden < 1.0% Undocumented immigrants
Table 14-7: Portion of target population not covered by Main Study sampling frames for countries not using population registries
Country
Percentage of target population not covered Group not covered* Australia 3.3% Persons living in very remote areas, discrete indigenous communities (DIC), or noninstitutional special dwellings; non-Australian diplomats, their staff and household members of such; members (and their dependents) of non- Australian defense forces Canada 1.8% Residents of smallest communities in the northern territories; residents of remote and very low population density areas in provinces; and persons living in noninstitutional collective dwellings, other than students in residences. Cyprus 6 < 2.0% Persons living in houses built after December 2010 Czech Republic 1.8% Professional armed forces; municipalities with < 200 habitants England/Northern Ireland (UK)
2.0% Individuals living in private residences that are not listed on the “residential” version of the Postal Address File (PAF) or, in Northern Ireland (UK), not listed on the NI(POINTER) database France < 2.6% Young adults who have never claimed any income and are not attached to their parents households; undocumented immigrants Ireland 0.4% Some mobile dwellings Korea 2.4% Small islands residents Russian Federation^7 1.5% Chechnya region United States 0.1% (^) People in large gated communities
The PIAAC standard sample design is a self-weighting design of persons (or of households, for countries without person registries). A self-weighting design is achieved when each sample person (or household, if sampling dwelling units) has an equal probability of selection (standard 4.4.3). For countries that are geographically large, the typical sample design is a stratified multistage clustered area sample. For participating countries that are geographically small, the sample design had less clustering and fewer stages of sampling. Also, several countries had lists of households or persons already available from national registries or registries managed by municipalities.
(^6) Please refer to notes A and B regarding Cyprus in the Note to Readers section of this report.
7 Please refer to the note regarding the Russian Federation in the Note to Readers section of this report.
Austria was the only country that adapted a one-stage sample design with no explicit stratification.
For a one-stage stratified sample design, let
𝑛 (^) ℎ = number of persons to be sampled in stratum ℎ; and
𝑁ℎ = number of eligible persons in stratum ℎ.
Further, let 𝑟 = 𝑛/𝑁, then the probability of selecting person 𝑙 in strata ℎ is
𝑃ℎ𝑙 = 𝑟.
The sample size is allocated to strata as
𝑛 (^) ℎ = 𝑃ℎ𝑙 × 𝑁ℎ = 𝑟 × 𝑁ℎ.
Seven countries used a one-stage stratified sample design: Flanders (Belgium), Denmark, Estonia, Finland, Netherlands, Norway and Sweden.
Two-stage stratified probability proportionate to size designs
The formulae for the standard PIAAC selection probabilities for each stage are given below.
For the first-stage sample of primary sampling units (PSUs) in the remaining countries, let
𝑚 (^) ℎ = number of PSUs to be sampled in stratum ℎ;
𝑀𝑂𝑆ℎ𝑖 = measure of size for PSU 𝑖 in stratum ℎ; and
𝐼𝑝𝑠𝑢ℎ^ = sampling interval for the selection of PSUs in stratum ℎ.
The probability of selecting PSU 𝑖 in stratum ℎ is
For the second-stage sample of persons, let
𝑛 = total number of persons to be sampled;
𝑁 = total number of eligible persons;
𝑛 (^) ℎ𝑖 = number of persons to be sampled in PSU 𝑖 of stratum ℎ; and
𝑁ℎ𝑖 = number of eligible persons in PSU 𝑖 of stratum ℎ.
Let 𝑟 = 𝑛/𝑁, then the conditional probability of selecting person 𝑙 in PSU 𝑖 of stratum ℎ is
The overall probability of selecting person 𝑙 in PSU 𝑖 of stratum ℎ is
𝑃ℎ𝑖𝑙 = 𝑃ℎ𝑖 × 𝐶𝑃ℎ𝑖𝑙 = 𝑟.
The sample size in PSU 𝑖 of stratum ℎ is
Seven countries used a two-stage stratified sample design: Cyprus,^9
Three-stage stratified probability proportionate to size (PPS) designs
France, Germany, Japan, Poland, Slovak Republic and Spain. Poland’s weights varied due to oversampling and by applying an alternative design implementation strategy. France used a different approach that followed balance sampling (Deville & Tillé, 2004 and Tillé, 2006) that resulted in varying base weights. Germany’s design included deep stratification in the context of Cox (1987) and included simulated values for probabilities of selection due to a sampling-related problem. Spain’s weights varied due to applying an alternative design implementation strategy.
In a three-stage stratified PPS design, PSUs are selected with a probability proportionate to a measure of size as described below.
For PSU selection in the training countries, let
𝑚 (^) ℎ = number of PSUs to be sampled in stratum ℎ;
𝑀𝑂𝑆ℎ𝑖 = measure of size for PSU 𝑖 in stratum ℎ; and
𝐼𝑝𝑠𝑢ℎ^ = sampling interval for the selection of PSUs in stratum ℎ.
The probability of selecting PSU 𝑖 in stratum ℎ is
For the second stage sample of dwelling units (DUs), let
𝑑 = total number of housing units to be sampled;
𝐷 = total number of housing units in the sampling frame;
𝑑ℎ𝑖 = number of housing units to be sampled in PSU 𝑖 of stratum ℎ; and
𝐷ℎ𝑖 = number of housing units in PSU 𝑖 of stratum ℎ.
Let 𝑟 = 𝑑 𝐷⁄ , then the conditional probability of selecting housing unit 𝑘 from PSU 𝑖 in stratum ℎ is
(^9) Please refer to notes A and B regarding Cyprus in the Note to Readers section of this report.
For SSU selection, let
𝑞 = total number of SSUs to be sampled;
𝑀𝑂𝑆ℎ𝑖𝑗 = measure of size for SSU 𝑗 of PSU 𝑖 in stratum ℎ; and
𝐼𝑆𝑆𝑈 = sampling interval for the selection of SSUs.
The conditional probability of selecting SSU 𝑗 from PSU 𝑖 in stratum ℎ is
𝑀𝑂𝑆ℎ𝑖𝑗 𝑃 (^) ℎ𝑖 � ∑ (^) � 𝑀𝑂𝑆ℎ𝑖𝑗 ℎ𝑖𝑗 𝑃 (^) ℎ𝑖 �
For DU selection, let
𝑑 = total number of housing units to be sampled;
𝐷 = total number of housing units in the sampling frame;
𝑑ℎ𝑖𝑗 = number of housing units to be sampled in SSU 𝑗 of PSU 𝑖 of stratum ℎ; and
𝐷ℎ𝑖𝑗 = number of housing units in SSU 𝑗 of PSU 𝑖 of stratum ℎ.
Let = 𝑑 𝐷⁄^ , then the conditional probability of selecting housing unit 𝑘 from SSU 𝑗 of PSU 𝑖 in stratum ℎ is
The overall probability of selecting housing unit 𝑘 in SSU 𝑗 of PSU 𝑖 of stratum ℎ is
𝑃ℎ𝑖𝑗𝑘 = 𝑃ℎ𝑖 × 𝐶𝑃ℎ𝑖𝑗 × 𝐶𝑃ℎ𝑖𝑗𝑘 = 𝑟
The DU sample size in a SSU is
For person selection, let
𝑛 (^) ℎ𝑖𝑗𝑘 = number of persons to be sampled from housing unit 𝑘 of SSU 𝑗 in PSU 𝑖 within stratum ℎ; and
𝑁ℎ𝑖𝑗𝑘 = total number of eligible persons in housing unit 𝑘 of SSU 𝑗 in PSU 𝑖 within
stratum ℎ.
The conditional probability of selecting person 𝑙 from housing unit 𝑘 of SSU 𝑗 in PSU 𝑖 within stratum ℎ is
The overall probability of selecting person 𝑙 from housing unit 𝑘 of SSU 𝑗 in PSU 𝑖 within stratum ℎ is
Australia, the Czech Republic, the Russian Federation,^10
the England design stratum of the United Kingdom, and the United States used a four-stage stratified PPS sample design. The Czech Republic conducted oversampling and also implemented a sequential selection design strategy that caused excessive variation in the resulting base weights. England (UK) had variation in its base weights due to implementing a selection process that is different from the one outlined with the above formulae.
14.4.1 Sample units
Because Austria, Flanders (Belgium), Denmark, Estonia, Finland, Netherlands, Norway and Sweden all implemented a one-stage sample design, they have only one sample unit: persons. The sampling units for countries with two-, three-, and four-stage sample designs are shown in Tables 14-8 to 14-10, respectively.
Table 14-8: Main study sample units for countries with two stages of sampling
Country Stage 1 Stage 2 Cyprus 11 Households Persons France Area PSUs Persons Germany Communities Persons Japan Cho/Chome/Aza administrative districts
Persons
Poland Urban Towns/Cities Persons Rural Towns/Villages Persons Slovak Republic Municipalities Persons Spain Area PSUs Persons Note: “Area PSUs” indicates primary sampling unit covers a geographic area not defined by a generic geographic terminology (towns, villages, etc).
(^10) Please refer to the note regarding the Russian Federation in the Note to Readers section of this report.
(^11) Please refer to notes A and B regarding Cyprus in the Note to Readers section of this report.
Table 14-12: Main Study selection methods for countries with two stages of selection
Country Stage Description Cyprus 13 1 Systematic random from a sorted list within explicit strata 2 SRS of 1 person per household via pre-assigned selection grid France 1 Systematic random from master sample IAAs (master sample selected using the balanced sampling algorithm, the “Cube” method, PPS (number of main residences in the IAA)) 2 Systematic random from a sorted list Germany 1 Stratified, PPS (target population) with allocation by controlled rounding 2 Two-phase sample.
All countries with three- or four-stage designs selected samples of dwelling units before the enumeration and selection of persons within households. Although the goal was to select one person per household, the selection of more than one person per household was preferred for countries with a large variation in household size (standard 4.4.4). These include the Russian Federation^14 and the United States. Details regarding the selection methods for countries with three- or four -stage designs are presented in Tables 14-13 and 14-14, respectively.
(^13) Please refer to notes A and B regarding Cyprus in the Note to Readers section of this report. (^14) Please refer to the note regarding the Russian Federation in the Note to Readers section of this report.
Table 14-13: Main Study selection methods for countries with three stages of selection Country Stage Description Canada 1 Systematic PPS (2006 population counts) from a sorted list within explicit strata with Census Metropolitan Areas sampled with certainty 2 Systematic random from a sorted list within explicit strata 3 SRS of 1 person per household via pre-assigned hash number Ireland 1 Stratified PPS (total dwellings) 2 SRS 3 SRS of 1 person per household Italy 1 Systematic PPS (target population) from a sorted list within explicit strata 2 Systematic random from a sorted list 3 SRS of 1 person per household via selection grid is used if the household composition is different from the register; otherwise SRS from registry. Korea 1 Systematic random sample from a sorted list within explicit strata 2 Systematic random from a sorted list 3 SRS of 1 person per household Note: “SRS” indicates simple random sampling.
Table 14-14: Main Study selection methods for countries with four stages of selection Country Stage Description Australia 1 Systematic PPS (number of DU clusters) from a sorted list within explicit strata (subsample from master sample) 2 Systematic PPS (number of DU clusters) from a sorted list (subsample from master sample) 3 Systematic random from a sorted list 4 SRS of 1 person per household Czech Republic 1 Systematic PPS (number of inhabitants aged 16-65) from a sorted list within explicit strata 2 Systematic PPS (number of address points) 3 SRS; selected a “basic” sample of households to achieve the 5,000 completes plus an additional sample of households in which only 16- to 29-year-olds were sampled. 4 SRS of 1 person per household England (UK) 1 Systematic PPS (PAF single occupancy count) from a sorted list within explicit strata 2 Systematic random from a sorted list 3 SRS of 1 household at the sampled address using the Kish grid 4 SRS of 1 person per household using the Kish grid Northern Ireland (UK) 1 Systematic random from a sorted list 2 SRS of 1 household at the sampled address using the Kish grid 3 SRS of 1 person per household using the Kish grid Russian Federation^15 1 Systematic PPS (population in the region) from a sorted list within explicit strata 2 Systematic PPS (target population) from a sorted list 3 Systematic random from a sorted list 4 SRS of 1 person for household sizes up to 4 (otherwise 2 persons) via pre- assigned selection grid United States (USA) 1 Systematic PPS (population) within explicit strata 2 Systematic PPS (number of DUs) from a sorted list 3 Systematic random from a sorted list 4 SRS of 1 person for household size up to 3 (otherwise 2 persons) Note: “SRS” indicates simple random sampling.
(^15) Please refer to the note regarding the Russian Federation in the Note to Readers section of this report.
Table 14-16: Main Study stratification/sorting variables and methods for countries with two stages of selection
Country Stage Description Cyprus 16 1 Strata: district, urban/rural classification Within strata: sort by geographic location 2 None France 1 Strata: administrative region (for master sample) Balancing variables: number of main residences, total income, number of DUs in rural, peri-urban, and urban areas. 2 Stratified by housing (synthetic variable differentiating ordinary housing and communities) and sorted by department (administrative district). Germany 1 Strata: region, urban/rural status (BIK) – approximately 1,000 strata cells 2 None in Phase 1. In Phase 2, stratified by age group and gender, sorted by age. Japan 1 Strata: region, urban/rural status; Sort by regional code 2 Sort by address Poland Urban 1 Strata: size class 2 Strata: age (19-26, other) Rural 1 Strata: region and size class 2 Strata: age (19-26, other) Slovak Republic 1 Strata: region, municipality size; Within strata: sort by number of age 16-65 in municipality 2 Sort by gender and age Spain 1 Strata: categories of municipality size Within strata: sort by population size 2 Sort by gender and age
(^16) Please refer to notes A and B regarding Cyprus in the Note to Readers section of this report.
Table 14-17: Main Study stratification/sorting variables and methods for countries with three stages of selection
Country Stage Description Canada 1 Stratify by province, urban/rural; sort by geographic order of PSUs and 2006 population counts 2 Stratified by province/territory and urban/rural. Sort by geographic order (province/territory code, urban/rural, PSU ID, Census collection unit ID) 3 None Ireland 1 Strata: urban/rural status, and educational profile Within strata: sort by size of SAs 2 None 3 None Italy 1 Strata: geographic regions of equal size Within strata: sort by the target population count of the PSUs 2 None 3 Random sort if selection from registry. If the household composition is different from the registry, persons are sorted by gender and age and the selection grid is used. Korea 1 Strata: administrative districts Within strata: sort by enumeration district characteristics, such as townhouse versus apartment, percentage of 1-person household, education level, average age, percentage of people who are older than 60 2 Sort by address 3 None