Download NWCA Project Evaluations Exam and more Exams Technology in PDF only on Docsity!
Question 1. Which evaluation purpose focuses on improving program implementation while it is still underway? A) Summative evaluation B) Formative evaluation C) Ex post evaluation D) Impact evaluation Answer: B Explanation: Formative evaluations are conducted during program delivery to provide feedback for ongoing improvement, unlike summative evaluations which assess final outcomes. Question 2. Aligning evaluation goals with an organization’s mission primarily ensures which of the following? A) Higher statistical power B) Relevance to stakeholder priorities C) Faster data collection D) Reduced cost of evaluation Answer: B Explanation: When evaluation goals reflect the organization’s mission, the findings are more relevant and useful to stakeholders who care about mission fulfillment. Question 3. A key evaluation question (KEQ) that asks “What unintended consequences occurred?” is an example of what type of question? A) Process‑oriented KEQ B) Outcome‑oriented KEQ C) Impact‑oriented KEQ D) Contextual KEQ Answer: D
Explanation: Unintended consequences relate to the broader context and external factors, making it a contextual KEQ. Question 4. Which research design provides the strongest evidence of causality? A) Non‑experimental survey B) Quasi‑experimental pre‑post design C) Randomized controlled trial (experimental) D) Case study Answer: C Explanation: Randomized controlled trials randomly assign participants to treatment and control groups, minimizing bias and establishing causality. Question 5. In a mixed‑methods evaluation, the primary advantage of integrating quantitative and qualitative data is: A) Reducing the need for sampling B) Enhancing triangulation of findings C) Eliminating the need for statistical analysis D) Shortening the evaluation timeline Answer: B Explanation: Combining methods allows researchers to cross‑validate results (triangulation), providing richer, more credible insights. Question 6. When determining sample size for a quantitative survey, which factor most directly influences the required number of respondents? A) Number of interviewers available B) Desired confidence level and margin of error C) Length of the questionnaire
C) Focuses only on financial inputs D) Is a visual diagram only Answer: B Explanation: Theory of Change articulates the causal reasoning and assumptions behind how activities will achieve impact, often presented as a narrative. Question 10. Which of the following is an external assumption in a project logic model? A) Staff have the required skills B) Funding will continue for two years C) Participants will attend workshops D) Data collection tools are reliable Answer: B Explanation: External assumptions are factors outside the project’s control, such as continued funding. Question 11. The primary purpose of pilot testing an instrument is to: A) Increase the sample size B) Identify ambiguous wording and potential bias C) Conduct full data analysis D) Replace the need for validity testing Answer: B Explanation: Pilot testing reveals unclear items, leading to revisions that improve validity and reliability. Question 12. Content validity of a survey is best assessed by: A) Test‑retest reliability B) Cronbach’s alpha
C) Expert review of items against the construct domain D) Factor analysis Answer: C Explanation: Content validity ensures that items fully represent the construct, typically judged by subject‑matter experts. Question 13. Which reliability coefficient indicates internal consistency of a multi‑item scale? A) Pearson’s r B) Kappa statistic C) Cronbach’s alpha D) ICC (intraclass correlation) Answer: C Explanation: Cronbach’s alpha measures the extent to which items on a scale are correlated, reflecting internal consistency. Question 14. In digital data collection, which technology most enhances real‑time data quality checks? A) Paper‑based questionnaires B. Spreadsheet entry after fieldwork C) Mobile data collection apps with built‑in validation rules D) Email‑based survey distribution Answer: C Explanation: Mobile apps can enforce required fields, range checks, and skip logic instantly, reducing entry errors. Question 15. Informed consent is primarily intended to protect participants’:
Question 18. Which descriptive statistic best summarizes the central tendency of a highly skewed income variable? A) Mean B) Median C) Mode D) Standard deviation Answer: B Explanation: The median is less affected by extreme values and therefore better represents central tendency for skewed data. Question 19. A p‑value of 0.03 in a hypothesis test indicates: A) The null hypothesis is definitely false B) There is a 3% probability that the observed effect is due to random chance, assuming the null is true C) The effect size is large D) The confidence interval includes zero Answer: B Explanation: A p‑value of 0.03 means there is a 3% chance of obtaining the observed data (or more extreme) if the null hypothesis were true. Question 20. In regression analysis, multicollinearity is problematic because it: A) Increases the sample size needed B) Inflates standard errors, making coefficient estimates unstable C) Guarantees a higher R‑square D) Violates the independence assumption Answer: B
Explanation: Multicollinearity (high correlation among predictors) reduces precision of coefficient estimates, leading to inflated standard errors. Question 21. Which inferential test compares the means of three independent groups? A) Paired t‑test B) One‑way ANOVA C) Chi‑square test D) Mann‑Whitney U test Answer: B Explanation: One‑way ANOVA assesses whether there are statistically significant mean differences among three or more independent groups. Question 22. In thematic analysis, the process of grouping similar codes into broader categories is called: A) Open coding B) Axial coding C) Selective coding D) Pattern coding Answer: B Explanation: Axial coding links codes around a central axis, forming themes that explain relationships among categories. Question 23. An evaluator wants to measure participants’ perceptions of program relevance using open‑ended interview questions. Which qualitative technique is most appropriate for analyzing the data? A) Factor analysis B) Grounded theory coding C) Logistic regression
C) Detailed methodological dissertation D) Long narrative report without graphics Answer: B Explanation: Executives prefer high‑level summaries, visualizations, and actionable recommendations rather than exhaustive technical details. Question 27. Heatmaps are particularly useful for visualizing which type of data? A. Time series trends B. Categorical frequencies across two dimensions C. Geographic locations only D. Single numeric values without comparison Answer: B Explanation: Heatmaps display intensity of values across rows and columns, making it easy to spot patterns in categorical cross‑tabulations. Question 28. Delivering “negative” findings constructively involves which of the following actions? A) Hiding the results from stakeholders B) Presenting them alongside evidence‑based recommendations for improvement C) Blaming external factors exclusively D) Ignoring them in the final report Answer: B Explanation: Constructive communication frames negative results as opportunities for learning and provides actionable steps. Question 29. According to the NWCA Code of Ethics, an evaluator must disclose: A) Only conflicts that affect financial compensation
B) Any actual or potential conflict of interest, regardless of magnitude C. Personal opinions about the project’s success D. The identities of all participants to the sponsor Answer: B Explanation: Transparency requires disclosing all real or perceived conflicts to maintain objectivity and trust. Question 30. A conflict of interest arises when: A. The evaluator receives a salary from the sponsoring organization B. The evaluator has no prior relationship with the project team C. The evaluator’s personal values align with the project’s mission D. The evaluator uses open‑source software for analysis Answer: A Explanation: Receiving compensation from the sponsor can bias the evaluator’s judgments, constituting a conflict of interest. Question 31. Social Return on Investment (SROI) differs from traditional ROI by: A. Ignoring monetary benefits B. Incorporating social, environmental, and economic value into a single ratio C. Using only qualitative narratives D. Excluding cost data from the calculation Answer: B Explanation: SROI translates social outcomes into monetary equivalents, allowing comparison of total value generated versus investment. Question 32. For long‑term sustainability assessment, which indicator is most relevant? A. Number of participants in the first month
A. Independent‑samples t‑test B. Mann‑Whitney U test C. Paired t‑test D. Pearson correlation Answer: B Explanation: The Mann‑Whitney U test compares distributions of ordinal data between independent groups. Question 36. Which of the following best describes “construct validity”? A. Consistency of scores over time B. The degree to which a test measures the theoretical construct it intends to measure C. Agreement between two raters D. The proportion of missing data in a dataset Answer: B Explanation: Construct validity assesses whether an instrument truly captures the abstract concept it purports to measure. Question 37. A researcher plans to use an observation checklist to assess classroom engagement. To ensure inter‑rater reliability, she should: A. Use a single observer for all sessions B. Train multiple observers and calculate Cohen’s kappa across them C. Randomly change the checklist items each day D. Avoid any training to keep observations natural Answer: B Explanation: Inter‑rater reliability is measured by statistical agreement (e.g., Cohen’s kappa) after standardized training.
Question 38. Which data‑collection method is most vulnerable to social desirability bias? A. Anonymous online survey B. Structured telephone interview with a known interviewer C. Direct observation without participants’ knowledge D. Document review of administrative records Answer: B Explanation: When respondents interact with an identifiable interviewer, they may answer in ways they think are socially acceptable. Question 39. In an evaluation report, a “technical appendix” typically contains: A. High‑level findings for executives B. Detailed methodology, data tables, and statistical outputs C. Graphic design elements only D. Personal anecdotes from the evaluator Answer: B Explanation: Technical appendices provide the methodological depth and raw results for readers who need full transparency. Question 40. Which of the following best illustrates a “negative” finding presented constructively? A. “The program failed completely; we recommend termination.” B. “Attendance was lower than expected; consider adjusting outreach strategies to improve reach.” C. “No data were collected, so we cannot comment.” D. “Stakeholders expressed dissatisfaction; we will ignore their feedback.” Answer: B
Answer: B Explanation: Changes in knowledge (a short‑term outcome) can predict longer‑term impacts, making it a leading indicator. Question 44. In a cost‑effectiveness analysis, the incremental cost‑effectiveness ratio (ICER) is calculated as: A. (Cost of intervention – Cost of control) ÷ (Effect of intervention – Effect of control) B. (Total benefits ÷ Total costs) C. (Effect of control ÷ Cost of control) D. (Cost of control ÷ Effect of control) Answer: A Explanation: ICER compares the additional cost to the additional effect of one intervention versus another. Question 45. Which of the following best describes “external validity”? A. The extent to which findings can be generalized to other settings, populations, or times B. The precision of measurement within the study sample C. The statistical significance of results D. The reliability of the instrument used Answer: A Explanation: External validity concerns the applicability of results beyond the specific study context. Question 46. A researcher uses a “snowball” sampling technique. This method is most appropriate when: A. The target population is well‑defined and easily listed B. The population is hidden or hard to reach (e.g., illicit drug users)
C. Random sampling is required by the funding agency D. The sample size must be exactly 100 respondents Answer: B Explanation: Snowball sampling leverages referrals from initial participants to reach hidden populations. Question 47. In a dashboard designed for project managers, which visual element best conveys progress toward a numeric target? A. Pie chart showing market share B. Gauge or progress bar indicating % of target achieved C. Scatter plot of unrelated variables D. Word cloud of stakeholder comments Answer: B Explanation: Gauges or progress bars directly illustrate how close the actual value is to the predefined target. Question 48. Which of the following is an ethical consideration when publishing evaluation findings? A. Excluding any negative results to protect the sponsor’s reputation B. Providing attribution to all data sources and respecting confidentiality agreements C. Using only the evaluator’s personal interpretation without participant input D. Publishing raw data without anonymization Answer: B Explanation: Ethical reporting requires proper attribution, confidentiality, and balanced presentation of results. Question 49. When calculating a confidence interval for a mean, increasing the sample size will:
Question 52. Which of the following best describes “response bias” in surveys? A. Random sampling error B. Systematic tendency of respondents to answer in a particular way (e.g., socially desirable) C. Technical malfunction of the survey platform D. Missing data due to non‑response Answer: B Explanation: Response bias occurs when respondents’ answers are systematically skewed, affecting validity. Question 53. An evaluator wants to assess whether a new teaching method improves test scores compared to the traditional method, controlling for prior achievement. Which statistical technique is most appropriate? A. Independent‑samples t‑test without covariates B. Paired t‑test on pre‑post scores only C. ANCOVA (Analysis of Covariance) with prior scores as covariate D. Chi‑square test of independence Answer: C Explanation: ANCOVA adjusts for baseline differences (prior achievement) while comparing group means. Question 54. In a cost‑benefit analysis, assigning a monetary value to improved health outcomes is an example of: A. Direct cost estimation B. Valuation of intangible benefits (willingness‑to‑pay) C. Discounting future cash flows D. Sensitivity analysis
Answer: B Explanation: Monetizing health improvements often uses willingness‑to‑pay or market‑based proxies, treating them as intangible benefits. Question 55. Which of the following best captures the purpose of a “logic gap analysis”? A. Identifying missing data points in the dataset B. Comparing expected logical pathways with actual implementation to spot disconnects C. Calculating the financial deficit of the project D. Measuring participant satisfaction levels Answer: B Explanation: Logic gap analysis examines where the planned theory of change diverges from real‑world execution. Question 56. When reporting statistical results, which of the following is considered best practice? A. Reporting only p‑values without effect sizes B. Providing effect sizes, confidence intervals, and p‑values together C. Using vague terms like “significant” without numbers D. Omitting any mention of sample size Answer: B Explanation: Comprehensive reporting includes effect magnitude, precision (CI), and significance to aid interpretation. Question 57. A “dashboard” that updates automatically from a cloud‑based data source exemplifies which evaluation principle? A. Static reporting B. Real‑time data utilization for decision‑making
