Statistical Methodology

Census Data Journey

When the people of Scotland return their digital or paper census questionnaires to National Records of Scotland (NRS), there is a massive body of work that takes place to transform the millions of individual responses into high quality census outputs.

Understanding the flow of census data through its end-to-end journey is vital for ensuring appropriate methodologies, including the development of meaningful quality indicators, are in place and being followed. The development of the statistical methodology that underpins this work for Scotland’s Census 2022 is well underway.

The journey that census data takes once it arrives at NRS can be summarised as follows:

Capture and Coding The process by which responses given by an individual or household are converted into electronic data and given consistent numerical codes.
Data Cleansing A collection of processes we apply to census data to account for specific errors, and prepare the data so it's suitable for later statistical processes.
Administrative Data We use other data sources that are independent of census to help with the quality assurance process.
Edit and Imputation Ensures the data is complete and consistent at record level.
Census Coverage Survey (CCS) A survey that happens after census day to provide a secondary source of data to the census which allows us to estimate the total Scottish population.
Estimation and Adjustment Estimation produces overall population and household estimates, Adjustment creates new records for the missed population and both combined gives a census dataset for the whole population.
Validation of Population Estimates We use expert panels and other data sources that are independent of the census to ensure our population estimates are correct, accurate and what we would expect.
Statistical Disclosure Control To prevent the release of disclosive or confidential information about an individual or household.
Outputs To produce the statistical outputs, supporting information and analysis for release to the public.

It is vital that the statistical methods that NRS develop for each step in the data processing journey are trustworthy and aid in producing high quality data that are of value to the people of Scotland. In addition, NRS are aiming to produce first outputs within one year of Census day, meaning all of the processes listed above are time critical. To ensure our methodology at each stage in the data journey is fit for purpose, our statistical methodology undergoes rigorous peer review and follows a strict governance and testing schedule prior to live census.

When developing statistical methodology for the 2022 census, there are several factors that are generally considered by our statisticians:

  • What is the purpose of the process or test?
  • What was done in the previous census?
  • What lessons have been learned?
  • How have things changed?
  • How do changes in one part of the census design affect downstream processes and methodologies?
  • Are the methods harmonised with the other UK censuses?
  • What are the lessons learned from international censuses?
  • Is it cost effective?
  • Does it improve the quality of the data?
  • Is it achievable within IT/Operational timescales?

Peer Review and Governance

Internal Peer Review (IPRG)     An internal NRS panel of 15 statisticians that review and critique the statistical methodology for each step of the data journey.

For all proposed statistical steps, methodology papers are created that outline the fine detail of how each statistical step will occur and what the expected outcome will be, with any testing and modelling included. Papers are reviewed and agreed within the teams responsible for delivery of the statistical steps and these papers are submitted to the Internal Peer Review Group (IPRG). The IPRG is made up of 15 statisticians from across the census programme and NRS population and demography teams. The group meets twice a month to critique proposed statistical methodology and provide feedback on statistical options. The IPRG are regularly asked to assure that the statistical methodologies proposed are fit for purpose.

External Methodology Assurance Panels (EMAPs)    

A short-life working group of external subject matter experts and academics who review the statistical methodology after internal peer review and provide advice on whether proposed methodology is fit for purpose.

Following revisions to methodologies after internal peer review, key methodologies that are fundamental to the census programme are submitted to External Methodology Assurance Panels (EMAPs) for review. The EMAPs comprise seven subject matter experts and academics that meet monthly between May and October 2020, to review the proposed statistical methodology for Scotland’s Census. The group is chaired by one of the external panel members ensuring a fully independent peer review process.

Panel members review and discuss each proposed methodology and are asked to reach a conclusion on whether each proposed methodology is fit for purpose. A summary report, along with the methodology papers presented for review by the panel is then published on the Scotland’s Census website following each panel.

More information on the methodologies reviewed and EMAPs members can be found on the External Methodology Assurance Panel web page.

Board approval                           

After both internal and external peer review, statistical methodologies are sent to the internal governance boards for final approval.

Statistical methodologies are then be presented to Census governance boards for final sign off. Methodologies are also being rigorously tested using rehearsal data and further testing will take place during end to end testing of the integrated census systems that will occur prior to live census.

Methodology Stakeholder Events

In February 2020, NRS ran Statistical Methodology Stakeholder events aimed at the general public and data users. These events gave attendees a high level overview of what happens to census data from when NRS receives census responses through to producing the Census 2022 outputs. During these events, we sought feedback to help to further develop our plans to ensure the highest quality of outputs for our users. Slides from the event are published on the event page of the Scotland’s Census 2022 website.

Statistical Methodology Rehearsal 2020

From April to June 2020, we ran a rehearsal of several statistical methodologies for Scotlands Census 2022.

You can read the evaluation reports we produced from the rehearsal.