Do you use census data? We'd like your feedback.

Scotland’s Census 2022 Filter Rules – Final Methodology

Background

Filter Rules is part of the fourth step in the census data processing journey.

For Scotland’s Census 2022, some respondents were routed past certain questions because they were not required to answer them, depending on what the questions are and the respondent’s circumstances. For example, those under the age of 16 are asked to skip subsequent questions about their employment activity. When guidance to skip subsequent questions is not followed (or misinterpreted), the potential for these answers to be inconsistent with earlier answers on the questionnaire exists.  This creates issues in data processing, particularly for the Edit and Imputation (E&I) function - a process where missing and/or contradictory responses are resolved.

During census processing, there are a number of pre-determined rules which are implemented to assist in addressing this issue.  They cover two overall purposes - firstly, to resolve inconsistencies where the answer can reasonably be assumed, and secondly to reinforce routing by turning appropriately skipped responses from “missing” to “no code required”.  Marking these values as such allow a more accurate measurement of item (question) level non-response. 

It is important to note that the Filter Rules process primarily applies to responses collected via paper questionnaires.  Inconsistent answers or incorrect routing is a much smaller issue for responses made through the census questionnaire online.  As a feature of the online questionnaire, respondents are simply brought to the next question, based on the answer(s) given in preceding questions.  This prevents the respondent from seeing unnecessary questions, which reduces respondent burden and minimises the ability to provide inconsistent answers.  On paper however, a respondent is able to see all of the questions at once, providing the opportunity to create these inconsistencies.

 

2022 Method

As a process, Filter Rules is required as a precursor to Edit and Imputation in order to ensure consistency in the application of questionnaire routing, relieve some of the burden from Edit and Imputation processing, and maximise the E&I donor pool.

Thus, the objectives of Filter Rules in 2022 were to:

1. Ensure the application of questionnaire routing in responses

Primarily, this applied to paper responses as online responses should skip questions that do not apply, and automatically assign the skipped questions with “no code required” (-5) rather than “missing” (-9). This distinguished appropriate non-response at the item (question) level from those questions which were skipped but should have been answered and signalled to Edit and Imputation processes that the remaining “missing” items require imputation.

2. Resolve (certain, limited) inconsistencies

Filter Rules also resolved certain inconsistencies, where we were able to reasonably assume the response. This was done through creating pre-determined rules which follow the questionnaire routing and take into account the intended meaning of the question.

For example, where a person has answered that they own their home, they were then asked to skip the landlord question. If the landlord question was answered, creating an inconsistent response, it was changed to “no code required” as we prioritise the ownership question - thus making answering the landlord question unnecessary.

It is important to note that while these specified changes are a necessary part of data preparation, in cases where ambiguity exists between responses and we cannot reasonably assume a response, changes were not made. These types of conflicts were evaluated through the Edit and Imputation methodology , which is more appropriate to resolve such contradictions. Filter Rules only make certain pre-determined corrections, based on the intended routing of the questionnaire. In those cases, the specification should read a “no action”, to indicate that we have considered the scenario better suited to be resolved by Edit and Imputation’s donor method (rather than the deterministic method employed by the Filter Rules).

3. Create a flag dataset

As part of Filter Rules, a flag dataset at the itemised step of each filter rule was created, where, if a particular variable was changed, a flag was set to indicate it had done so. This allowed the ability to trace back not only to the variable which was changed, but the step in the rule which initiated the change. This helped ensure that the changes made were correctly applied, and assist in tracking the scale at which Filter Rules made these changes.

2022 Filter Rules

Household Rules

A. Filter Rule A - Tenure and Landlord

Person Rules

B. Filter Rule B - Marital Status for Under 16s
C. Filter Rule C - Full Time Education for 0 - 3 Year Olds
D. Filter Rule D – Student and Term Time Address
E. Filter Rule E – Country of Birth and Year of Arrival
F. Filter Rule F – 0 Year Olds and Address One Year Ago
G. Filter Rule G – Carers Aged 0 - 2
H. Filter Rule H – Language Variables for 0 - 2 Year Olds
I. Filter Rule I – Long Term Health
J. Filter Rule J – Under 16s and Economic Activity
K. Filter Rule K - Economic Activity
L. Filter Rule L - Travel to Work or Study
M. Filter Rule M - Method of Travel
N. Filter Rule N - Travel or Work from Home

Notes:
-5 means “No Code Required”
-7 means “Invalid”
-9 means “Missing”

Filter Rule A – Tenure (H12) and Landlord (H13)

 

Filter Rule A is the only filter rule that is included in the Household section of the census questionnaire.

 

 

Item

Variable A

 

Variable B

Action (Then…)

Notes

(If tenure is…)

 

(If landlord is…)

1

Ticked, either:

1 (Owns outright), or

2 (Mortgage/Loan)

And

Ticked:

1 (Council/LA), or

2 (Private/Agency)

No Action

This is (essentially) the same as 2011.

2

 

Ticked, either:

1 (Owns outright), or

2 (Mortgage/Loan)

 

And

Ticked:

3 (Other)

Set:

landlord to -5 (NCR)

This is (essentially) the same as 2011.

3

Ticked, either:

1 (Owns outright), or

2 (Mortgage/Loan)

 

And

-7 (Invalid), or

-9 (Missing)

Set:

landlord to -5 (NCR)

This is (essentially) the same as 2011.

4

Ticked, either:

3 (Owns w/ Shared Equity), or 

5 (Part Owns/Part Rents)

And

Ticked:

1 (Council/LA), or

2 (Private/Agency)

No Action

In 2011, we did not route around the Landlord question for partial ownership answers to the Tenure question.  However, we have elected to do so for 2022, so these three sub-rules reflect this change.

5

Ticked, either:

3 (Owns w/ Shared Equity), or 

5 (Part Owns/Part Rents)

And

Ticked:

3 (Other)

Set:

landlord to -5 (NCR)

6

Ticked, either:

3 (Owns w/ Shared Equity), or 

5 (Part Owns/Part Rents)

 

-7 (Invalid), or

-9 (Missing)

Set:

landlord to -5 (NCR)

7

Ticked, either:

4 (Rents), or 

6 (Lives Here Rent Free), or

-7 (Invalid)

-9 (Missing)

And

Any Value

(ticked or unticked)

No Action

This is (essentially) the same as 2011.

 

Filter Rule B – Marital Status (5) for Under 16s

 

This is the first of several new age-based routing questions, and one where the guidance on paper questionnaires does not match the online routing.  As such, a deterministic edit (in Filter Rules) will only occur where responses match routing appropriately - meaning any contradiction will be resolved in E&I.

 

To qualify as “Filled”, two or more questions in the Dependent Fields must be answered.  See below for a list of the variables in the Dependent Fields group (generally questions that apply to 16+).

 

 

Item

Variable A

 

Variable B

 

Dependent Fields

Action (Then…)

Notes

(If age is…)

 

(If mar_stat is…)

 

1

0 - 15

&

Ticked:

1 (Never)

&

Filled

No Action

Response cannot be reasonably assumed - E&I to resolve

2

0 - 15

&

Ticked:

1 (Never)

&

Not Filled

Set:

mar_stat to -5

(No Code Required)

Generally expected behaviour; since there is no routing guidance for this question on paper, a response of “never” should be sufficient to route

3

0 - 15

&

Ticked:

ANY except

1 (Never)

&

Filled

No Action

Response cannot be reasonably assumed - E&I to resolve

4

0 - 15

&

Ticked:

ANY except

1 (Never))

&

Not Filled

No Action

Response cannot be reasonably assumed - E&I to resolve

5

0 - 15

&

-7 (Invalid), or

-9 (Missing)

&

Filled

No Action

Response cannot be reasonably assumed - E&I to resolve

6

0 - 15

&

-7 (Invalid), or

-9 (Missing)

&

Not Filled

Set:

mar_stat to -5

(No Code Required)

Expected behaviour

7

16+

&

Ticked:  ANY

&

Filled

No Action

Expected behaviour

8

16+

&

Ticked:  ANY

&

Not Filled

No Action

Response cannot be reasonably assumed - E&I to resolve

9

16+

&

-7 (Invalid), or

-9 (Missing)

&

Filled

No Action

Response cannot be reasonably assumed - E&I to resolve

10

16+

&

-7 (Invalid), or

-9 (Missing)

&

Not Filled

No Action

Response cannot be reasonably assumed - E&I to resolve

11

-7 (Invalid), or

-9 (Missing)

&

Ticked:  ANY

&

Filled

No Action

Response cannot be reasonably assumed - E&I to resolve

12

-7 (Invalid), or

-9 (Missing)

&

Ticked:  ANY

&

Not Filled

No Action

Response cannot be reasonably assumed - E&I to resolve

13

-7 (Invalid), or

-9 (Missing)

&

-7 (Invalid), or

-9 (Missing)

&

Filled

No Action

Response cannot be reasonably assumed - E&I to resolve

14

-7 (Invalid), or

-9 (Missing)

&

-7 (Invalid), or

-9 (Missing)

&

Not Filled

No Action

Response cannot be reasonably assumed - E&I to resolve

 

Dependent Fields

trans_stat

sex_orient

quals_school

quals_apprentice

quals_further

quals_higher

ex_service

emplyd_last_week

other_act

look_work

avail_work

wait_work

ever_worked

emp_stat

employer

occupation

industry

supervisor

hours_worked

 

 

Filter Rule C – Full Time Education (6) and 0 - 3 Year Olds (2)

 

This is another (new) age-based filter rule where the guidance on paper questionnaires does not match the online routing.  As such, a deterministic edit (in Filter Rules) will only occur where responses match routing appropriately - meaning any contradiction will be resolved in E&I.

 

To qualify as “Filled”, two or more questions in the Dependent Fields must be answered.  See below for a list of the variables in the Dependent Fields group (generally questions that apply to 16+).

 

 

 

Item

Variable A

 

Variable B

 

Dependent Fields

Action (Then…)

Notes

(If age is…)

 

(If student is…)

 

1

0 - 3

&

Ticked:

1 (Yes), or

2 (No)

&

Filled

No Action

Response cannot be reasonably assumed - E&I to resolve

2

0 - 3

&

Ticked:

1 (Yes)

&

Not Filled

No Action

Response cannot be reasonably assumed - E&I to resolve

3

0 - 3

&

Ticked:

2 (No)

&

Not Filled

Set:

student to -5

(No Code Required)

Expected behaviour without guidance; in line with routing

4

0 - 3

&

-7 (Invalid), or

-9 (Missing)

&

Not Filled

Set:

student to -5

(No Code Required)

Response cannot be reasonably assumed - E&I to resolve

5

0 - 3

&

-7 (Invalid), or

-9 (Missing)

&

Filled

No Action

Expected behaviour in line with routing

6

4+

&

Ticked:

1 (Yes), or

2 (No)

&

Filled

No Action

Expected behaviour

7

4+

&

Ticked:

1 (Yes), or

2 (No)

&

Not Filled

No Action

Response cannot be reasonably assumed - E&I to resolve

8

4+

&

-7 (Invalid), or

-9 (Missing)

&

Filled

No Action

Response cannot be reasonably assumed - E&I to resolve

9

4+

&

-7 (Invalid), or

-9 (Missing)

&

Not Filled

No Action

Response cannot be reasonably assumed - E&I to resolve

10

-7 (Invalid), or

-9 (Missing)

&

Ticked:

1 (Yes), or

2 (No)

&

Filled

No Action

Response cannot be reasonably assumed - E&I to resolve

11

-7 (Invalid), or

-9 (Missing)

&

Ticked:

1 (Yes), or

2 (No)

&

Not Filled

No Action

Response cannot be reasonably assumed - E&I to resolve

12

-7 (Invalid), or

-9 (Missing)

&

-7 (Invalid), or

-9 (Missing)

&

Filled

No Action

Response cannot be reasonably assumed - E&I to resolve

13

-7 (Invalid), or

-9 (Missing)

&

-7 (Invalid), or

-9 (Missing)

&

Not Filled

No Action

Response cannot be reasonably assumed - E&I to resolve

Dependent Fields

trans_stat

sex_orient

quals_school

quals_apprentice

quals_further

quals_higher

ex_service

emplyd_last_week

other_act

look_work

avail_work

wait_work

ever_worked

emp_stat

employer

occupation

industry

supervisor

hours_worked

 

 

Filter Rule D – Student (6) and Term Time Address (7)

 

For Variable Groups A and B:  Variable Groups are made up of several different questions (variables), noted below.  In order to be considered “Filled”, there needs to be a valid response to at least three or more questions in Group A.  “Not Filled” is a response to two or fewer questions.

   

Item

Variable A

 

Variable B

 

Variable Group

Action (Then…)

Notes

(If ft_student is…)

 

(If term_time is…)

 

A

1

Ticked:

1 (Yes)

&

Ticked:

1 (Address on Front)

&

Filled

No Action

As in 2011 (expected outcome)

2

Ticked:

1 (Yes)

&

Ticked:

1 (Address on Front)

&

Not Filled

No Action

As in 2011 (contradictory response; resolved by E&I)

3

Ticked:

1 (Yes)

&

Ticked:

2 (Another Address)

&

Filled

No Action

As in 2011 (contradictory response; resolved by E&I)

4

Ticked:

1 (Yes)

&

Ticked:

2 (Another Address)

&

Not Filled

Set:

Group B to -5

(No Code Required)

As in 2011 (expected outcome)

5

Ticked:

1 (Yes)

&

-7 (Invalid), or

-9 (Missing)

&

Filled

Set:

term_time to -5

(No Code Required)

As in 2011 (behaviour shows respondent simply missed term_time question)

6

Ticked:

1 (Yes)

&

-7 (Invalid), or

-9 (Missing)

&

Not Filled

Set:

term_time to -5 (NCR)

Group B to -5 (NCR)

As in 2011 (behaviour shows it was likely the respondent missed ticking a response for term_time)

7

Ticked:

2 (No)

&

Ticked:

1 (Address on Front)

&

Filled

Set:

term_time to -5

(No Code Required)

As in 2011 (respondent did not have to answer term_time)

8

Ticked:

2 (No)

&

Ticked:

1 (Address on Front)

&

Not Filled

Set:

term_time to -5

(No Code Required)

As in 2011 (respondent did not have to answer term_time; dependent fields will be resolved by E&I)

9

Ticked:

2 (No)

&

Ticked:

2 (Another Address)

&

Filled

Set:

term_time to -5

(No Code Required)

As in 2011 (respondent did not have to answer term_time)

10

Ticked:

2 (No)

&

Ticked:

2 (Another Address)

&

Not Filled

No Action

As in 2011 (contradictory response; resolved by E&I)

11

Ticked:

2 (No)

&

-7 (Invalid), or

-9 (Missing)

&

Filled

Set:

term_time to -5

(No Code Required)

Expected outcome.  This incorporates the 2011 pre-processing rule which was later added during live.

12

Ticked:

2 (No)

&

-7 (Invalid), or

-9 (Missing)

&

Not Filled

Set:

term_time to -5

(No Code Required)

This incorporates the 2011 pre-processing rule which was later added during live.  The dependent fields are left and should be resolved by E&I.

13

-7 (Invalid), or

-9 (Missing)

&

Ticked:

1 (Address on Front)

&

Filled

No Action

As in 2011 (unable to determine; ft_student was missed; resolved by E&I)

14

-7 (Invalid), or

-9 (Missing)

&

Ticked:

1 (Address on Front)

&

Not Filled

No Action

As in 2011 (contradictory response; resolved by E&I)

15

-7 (Invalid), or

-9 (Missing)

&

Ticked:

2 (Another Address)

&

Filled

No Action

As in 2011 (contradictory response; resolved by E&I)

16

Ticked:

2 (No)

&

-7 (Invalid), or

-9 (Missing)

&

Not Filled

No Action

As in 2011 (contradictory response; resolved by E&I)

17

Ticked:

2 (No)

&

-7 (Invalid), or

-9 (Missing)

&

Not Filled

No Action

As in 2011 (contradictory response; resolved by E&I)

18

Ticked:

2 (No)

&

-7 (Invalid), or

-9 (Missing)

&

Not Filled

No Action

As in 2011 (contradictory response; resolved by E&I)

 

 

The Dependent Variable Group A is the group of variables that is checked (if the respondent has filled three or more of) as evidence that the questions were to be answered.  Group B is all the variables that are affected by the Filter Rule/routing (which will be changed to -5, No Code Required where appropriate).

 

In 2011, we did not check voluntary questions (of which there was only religion). 

Dependent Group A

Dependent Group B

 

quals_higher

sex_orient

quals_higher

cob

ex_service

cob

ex_service

 

emplyd_last_week

mig_arr_month

emplyd_last_week

 

emplyd_last_week_grp2

mig_arr_year

emplyd_last_week_grp2

add_1year_ago

 

add_1year_ago

other_act

 

other_act

add_1year_ago_pc

look_work

 

look_work

add_1year_ago_cty

avail_work

carer

avail_work

carer

wait_work

lang_eng_understand

wait_work

lang_eng_understand

ever_worked

lang_eng_speak

emp_stat

lang_eng_speak

emp_stat

lang_eng_read

ever_worked

lang_eng_read

employer

lang_eng_write

occupation

lang_eng_write

occupation

lang_gael

industry

lang_gael

industry

lang_scots

supervisor

lang_scots

supervisor

lang_bsl

hours_worked

lang_bsl

hours_worked

main_lang

 

main_lang

work_study_address

health

 

health

work_study_pc

lt_cond

 

lt_cond

work_study_cty

 

work_study_address

lt_cond_other

work_study

disability

method_travel

disability

method_travel

passport

 

passport

 

 

 

passport_other_1_of_2

 

religion

 

passport_other_2_of_2

 

 

 

religion

 

nat_ident_uk

 

nat_ident_uk

 

 

 

nat_ident_other

 

 

 

ethnic_major

 

 

 

ethnic_1_of_2

 

 

 

ethnic_2_of_2

 

quals_school

 

quals_school

 

quals_apprentice

 

quals_apprentice

 

quals_further

 

quals_further

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Filter Rule E – Country of Birth (9) and Year of Arrival (10)

 

In 2011, where a respondent was born in a UK country - ie Scotland, England, Wales or Northern Ireland (cob), but filled in year of arrival (then known as yrarr) AND the year of arrival was after year of birth (then dob), this rule changed the country you were born to No Code Required.

 

However, for 2022 this Filter Rule will prioritise the Country of Birth variable, meaning the mig_arr_month and ­mig_arr_year variables will be turned to -5 (No Code Required) if a country within the UK is selected.

 

This is because it was found that the level of accuracy for the Country of Birth question was much higher than the Migration question (ONS 2011 Census Quality Survey), likely due to the ability to interpret the migration question differently than intended - for example, if a respondent thought it meant a return from a holiday outside the UK. 

 

Item

Variable A

 

Variable B

Action (Then…)

Notes

(if cob is…)

 

(if mig_arr is…)

1

Ticked, either:

923 (Scotland), or

921 (England), or

922 (Northern Ireland), or

924 (Wales)

&

mig_arr_month and

mig_arr_year are valid

Set:

mig_arr_month and

mig_arr_year to

 

-5 (No Code Required)

This is slightly different to 2011; then, we would have also checked date of birth before setting the mig_arr variables to no code required. For 2022 we have not checked dob (as explained above).

2

 

Ticked, either:

923 (Scotland), or

921 (England), or

922 (Northern Ireland), or

924 (Wales)

 

&

mig_arr_month and

mig_arr_year are invalid

 

-7 (Invalid), or

-9 (Missing)

Set:

mig_arr_month and

mig_arr_year to

 

-5 (No Code Required)

As in 2011

3

Any Country Code OTHER THAN

UK Countries
(923, 921, 922, 924)

&

mig_arr_month and

mig_arr_year are valid

No Action

As in 2011 (appropriate response, no action needed)

4

 

Any Country Code OTHER THAN

UK Countries
(923, 921, 922, 924)

 

&

mig_arr_month and

mig_arr_year are invalid

 

-7 (Invalid), or

-9 (Missing)

No Action

As in 2011 (potentially missed response; E&I to resolve)

5

-7 (Invalid), or

-9 (Missing)

&

mig_arr_month and

mig_arr_year are valid

No Action

As in 2011 (E&I to resolve)

6

-7 (Invalid), or

-9 (Missing)

&

mig_arr_month and

mig_arr_year are invalid

 

-7 (Invalid), or

-9 (Missing)

No Action

As in 2011 (E&I to resolve)

Filter Rule F – 0 Year Olds (2) and Address One Year Ago (11)

 

For Variable Groups A and B:  Variable Groups are made up of several different questions (variables), noted below.  In order to be considered “Filled”, there needs to be a valid response to at least three or more questions in Group A.  “Not Filled” is a response to two or fewer questions.

 

 

Item

Variable A

 

Address 1 Year Ago

Action (Then…)

Notes

(if age is…)

 

1

0

&

Filled - response to one or more variable:

 

add_1year_ago

add_1year_ago_pc

add_1year_ago_cty

 

Set:

All Address 1 Year Ago Fields to -5 (No Code Required)

 

As in 2011 (respondent does not have to answer the Address 1 Year Ago question if under the age of 1)

2

0

&

Not Filled

 

-7 Invalid, or

-9 Missing

Set:

All Address 1 Year Ago Fields to -5 (No Code Required)

As in 2011 (appropriate response; incorporates the pre-processing rule implemented during 2011 live)

3

1+

&

Filled - response to one or more variable:

 

add_1year_ago

add_1year_ago_pc

add_1year_ago_cty

No Action

As in 2011 (appropriate response)

4

1 +

&

Not Filled

 

-7 Invalid, or

-9 Missing

No Action

As in 2011 (resolved by E&I)

5

-7 Invalid, or

-9 Missing

&

Filled - response to one or more variable:

 

add_1year_ago

add_1year_ago_pc

add_1year_ago_cty

No Action

As in 2011 (resolved by E&I)

6

-7 Invalid, or

-9 Missing

&

Not Filled

 

-7 Invalid, or

-9 Missing

No Action

As in 2011 (resolved by E&I)

 

 

Address One Year Ago Fields

add_1year_ago

add_1year_ago_pc

add_1year_ago_cty

 

 

 

 

 

Filter Rule G – Carers (12) Aged 0 - 2 (2)


This is another (new) age-based filter rule where the guidance on paper questionnaires does not match the online routing.  As such, a deterministic edit (in Filter Rules) will only occur where responses match routing appropriately - meaning any contradiction will be resolved in E&I.

 

To qualify as “Filled”, two or more questions in the Dependent Fields must be answered.  See below for a list of the variables in the Dependent Fields group (generally questions that apply to 16+).

 

 

Item

Variable A

 

Variable B

 

Dependent Fields

Action (Then…)

Notes

(If age is…)

 

(If carer is…)

 

1

0 - 2

&

Ticked:

1 (No)

&

Filled

No Action

Response cannot be reasonably assumed - E&I to resolve

2

0 - 2

&

Ticked:

1 (No)

&

Not Filled

Set:

carer to -5

(No Code Required)

Generally expected behaviour; since there is no routing guidance for this question on paper, a response of “no” should be sufficient to route

3

0 - 2

&

Ticked:

Any other than

1 (No)

&

Filled

No Action

Response cannot be reasonably assumed - E&I to resolve

4

0 - 2

&

Ticked:

Any other than

1 (No)

&

Not Filled

No Action

Response cannot be reasonably assumed - E&I to resolve

5

0 - 2

&

-7 (Invalid), or

-9 (Missing)

&

Filled

No Action

Response cannot be reasonably assumed - E&I to resolve

6

0 - 2

&

-7 (Invalid), or

-9 (Missing)

&

Not Filled

Set:

carer to -5

(No Code Required)

Expected behaviour

7

3+

&

Ticked:

1 (Yes), or

2 (No)

&

Filled

No Action

Expected behaviour

8

3+

&

Ticked:

1 (Yes), or

2 (No)

&

Not Filled

No Action

Response cannot be reasonably assumed - E&I to resolve

9

3+

&

-7 (Invalid), or

-9 (Missing)

&

Filled

No Action

Response cannot be reasonably assumed - E&I to resolve

10

3+

&

-7 (Invalid), or

-9 (Missing)

&

Not Filled

No Action

Response cannot be reasonably assumed - E&I to resolve

11

-7 (Invalid), or

-9 (Missing)

&

Ticked:

1 (Yes), or

2 (No)

&

Filled

No Action

Response cannot be reasonably assumed - E&I to resolve

12

-7 (Invalid), or

-9 (Missing)

&

Ticked:

1 (Yes), or

2 (No)

&

Not Filled

No Action

Response cannot be reasonably assumed - E&I to resolve

9

-7 (Invalid), or

-9 (Missing)

&

-7 (Invalid), or

-9 (Missing)

&

Filled

No Action

Response cannot be reasonably assumed - E&I to resolve

10

-7 (Invalid), or

-9 (Missing)

&

-7 (Invalid), or

-9 (Missing)

&

Not Filled

No Action

Response cannot be reasonably assumed - E&I to resolve

 

 

Dependent Fields

trans_stat

sex_orient

quals_school

quals_apprentice

quals_further

quals_higher

ex_service

emplyd_last_week

other_act

look_work

avail_work

wait_work

ever_worked

emp_stat

employer

occupation

industry

supervisor

hours_worked

 

 

 

 

 

 

 

 

 

 

 

 

 

Filter Rule H – Language Variables (13 - 16) for 0 - 2 Year Olds (2)


This is another (new) age-based filter rule where the guidance on paper questionnaires does not match the online routing.  As such, a deterministic edit (in Filter Rules) will only occur where responses match routing appropriately - meaning any contradiction will be resolved in E&I.

 

For the language variable group to qualify as “filled”, one or more questions must be answered.  See below for a list of variables in the language variable group.

 

For the Dependent Fields to qualify as “Filled”, two or more questions must be answered.  See below for a list of the variables in the Dependent Fields group (generally questions that apply to 16+).

 

 

Item

Variable A

 

Variable(s) B

 

Dependent Fields

Action (Then…)

Notes

(If age is…)

 

(If language is…)

 

1

0 - 2

&

Filled

&

Filled

No Action

Response cannot be reasonably assumed - E&I to resolve.

2

0 - 2

&

Filled

&

Not Filled

No Action

Response cannot be reasonably assumed - E&I to resolve.

3

0 - 2

&

Filled:

all language variables are “no skills” and main_lang is

-7 (Invalid), or

-9 (Missing)

&

Not Filled

Set:

language group to

-5 (No Code Required)

Expected behaviour without guidance; in line with routing

4

0 - 2

&

Not Filled, ie

-7 (Invalid), or

-9 (Missing)

&

Filled

No Action

Response cannot be reasonably assumed - E&I to resolve.

5

0 - 2

&

Not Filled, ie

-7 (Invalid), or

-9 (Missing)

&

Not Filled

Set:

language group to

-5 (No Code Required)

Expected behaviour.

6

3+

&

Filled

&

Filled

No Action

Expected behaviour.

7

3+

&

Filled

&

Not Filled

No Action

Response cannot be reasonably assumed - E&I to resolve.

8

3+

&

Not Filled, ie

-7 (Invalid), or

-9 (Missing)

&

Filled

No Action

Response cannot be reasonably assumed - E&I to resolve.

9

3+

&

Not Filled, ie

-7 (Invalid), or

-9 (Missing)

&

Not Filled

No Action

Response cannot be reasonably assumed - E&I to resolve.

10

-7 (Invalid), or

-9 (Missing)

&

Filled

&

Filled

No Action

Response cannot be reasonably assumed - E&I to resolve.

11

-7 (Invalid), or

-9 (Missing)

&

Filled

&

Not Filled

No Action

Response cannot be reasonably assumed - E&I to resolve.

12

-7 (Invalid), or

-9 (Missing)

&

Not Filled, ie

-7 (Invalid), or

-9 (Missing)

&

Filled

No Action

Response cannot be reasonably assumed - E&I to resolve.

13

-7 (Invalid), or

-9 (Missing)

&

Not Filled, ie

-7 (Invalid), or

-9 (Missing)

&

Not Filled

No Action

Response cannot be reasonably assumed - E&I to resolve.

                   

 

Language Variable Group

Dependent Fields

lang_eng_understand

trans_stat

lang_eng_speak

sex_orient

lang_eng_read

quals_school

lang_eng_write

quals_apprentice

lang_gael

quals_further

lang_scots

quals_higher

lang_bsl

ex_service

main_lang

emplyd_last_week

 

other_act

 

look_work

 

avail_work

 

wait_work

 

ever_worked

 

emp_stat

 

employer

 

occupation

 

industry

 

supervisor

 

hours_worked

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Filter Rule I – Long Term Health (18)

 

In 2011, this rule was put into CANCEIS pre-processing during live because it was found a large proportion of people did not respond to the question - presumably because those in good health may have missed the “no conditions” option at the bottom of the list.  For 2022, this may not be as big a problem as online responses (which make up a greater proportion of submissions) should assist with data quality, but for paper, the list layout is nearly the same in 2022 as in 2011 - so this rule is incorporated here to maximise the donor pool.

 

 

Item

Variable A

Variable B

Variable C

Action (Then…

)

Notes

(If lt_cond_other is…)

(If health is…)

(If disability is…)

1

Filled (Any)

Any

Any

 

No Action

 

As 2011.

2

Not Filled, either:

-7 (Invalid)

-9 (Missing)

Ticked:

1 Very Good

2 Good

Ticked:

3 No

 

Set:

lt_cond to 0000000001

 

As 2011.

Filter Rule J – Under 16s (2) and Economic Activity (25 - 40)

 

Note that while this rule primarily deals with the Economic Activity questions, for 2022 it will also route the voluntary under 16 questions (question 4, trans_stat and question 8, sex_orient).  Additionally, unlike 2011, the qualifications variables will also be routed here (in 2011 this was dealt with in a pre-processing step).

 

For the Dependent Fields to qualify as “Filled”, two or more questions must be answered.  See below for a list of the variables in the Dependent Fields group (generally questions that apply to 16+).

 

 

Item

Variable A

 

Variable B

 

Dependent Fields

Action (Then…)

Notes

(If age is…)

 

(If emplyd_last_week_grp2 is…)

 

1

0 - 15

&

1 (Working)

&

Filled

No Action

As 2011.  Response cannot be reasonably assumed - E&I to resolve.

2

0 - 15

&

1 (Working)

&

Not Filled

Set:

Dependent Fields to

-5 (NCR)

As 2011

3

0 - 15

&

2 (Not Working)

&

Filled

Set:

Dependent Fields to

-5 (NCR)

As 2011 - conceivable behaviour IF under 16s answer these questions.

4

0 - 15

&

2 (Not Working)

&

Not Filled

Set:

Dependent Fields to

-5 (NCR)

As 2011.  Generally in line with expected behaviour; just answered a question they were not required to.

5

0 - 15

&

-7 (Invalid), or

-9 (Missing)

&

Filled

No Action

As 2011.  Response cannot be reasonably assumed - E&I to resolve

6

0 - 15

&

-7 (Invalid), or

-9 (Missing)

&

Not Filled

Set:

Dependent Fields to

-5 (NCR)

As 2011 - expected behaviour.

7

16+

&

1 (Working)

&

Filled

No Action

As 2011 - expected behaviour.

8

16+

&

1 (Working)

&

Not Filled

No Action

As 2011 - E&I to resolve.

9

16+

&

2 (Not Working)

&

Filled

No Action

As 2011.

10

16+

&

2 (Not Working)

&

Not Filled

No Action

As 2011 - E&I to resolve.

11

16+

&

-7 (Invalid), or

-9 (Missing)

&

Filled

No Action

As 2011 - E&I to resolve.

12

16+

&

-7 (Invalid), or

-9 (Missing)

&

Not Filled

No Action

As 2011 - E&I to resolve.

 

Dependent Fields

trans_stat

ex_service

wait_work

supervisor

sex_orient

emplyd_last_week

ever_worked

hours_worked

quals_school

emplyd_last_week_grp2

emp_stat

 

quals_apprentice

other_act

employer

 

quals_further

look_work

occupation

 

quals_higher

avail_work

industry

 

 

 

Filter Rule K – Economic Activity (Working Last Week)

 

In 2011, the question, “Have you ever done any paid work?” was covered by two variables, ever worked and last year worked.  Ever worked covered the tick response while a space was provided to write in a year for the last year worked variable.  If the year “2011” was filled in the last year worked variable, the assumption generally was that the respondent was referring to current employment. 

 

For 2022, the options for the ever worked variable have changed.  There is no longer a last year worked variable and to account for this, the options have been split into “Yes, in the last 12 months” and “Yes but not in the last 12 months”.  This has added some uncertainty to changing the ever_worked variable in the 2022 filter rule; as the census is issued in March of 2022, the equivalent to writing “2011” in last year worked is not the same as “Yes, in the last 12 months”.  Thus, many changes that require choice between the two ever worked options have been left for E&I to resolve where we may originally (in 2011) have made changes. 

 

For the Dependent Fields to qualify as “Filled”, 2 or more questions must be answered. See below for a list of the variables in the Dependent Fields group (generally questions that apply to 16+).

 

 

Item

Variable A

Variable B

Dependent Fields

Action (Then…)

Notes

(emplyd_last_week_grp2)

(ever_worked)

A

B

1

1 (Working)

1 (Last 12 Months)

Filled

Filled

No Action

Unable to tell if answers pertain to current or past employment.  E&I to resolve.  This is similar to what was done in 2011.

2

1 (Working)

1 (Last 12 Months)

Filled

Not Filled

No Action

Unable to tell if answers pertain to current or past employment, and respondent should have filled Fields B according to routing question ever_worked.  E&I to resolve.  This is a different to what was done in 2011, but due to question change.

3

1 (Working)

1 (Last 12 Months)

Not Filled

Filled

Set:

ever_worked to -5

Fields A to -5

A respondent currently working who followed routing, except they also answered ever_worked.. 

 

This is the same as 2011 (except there is no last year worked to change).

4

1 (Working)

1 (Last 12 Months)

Not Filled

Not Filled

Set:

ever_worked to -5

Fields A to -5

A respondent who did not fill Fields A or B, so followed at least part of the routing. They should have filled B (but not A) so leave B as missing and enforce routing by setting Fields A to -5. 

 

This is the same as 2011 (except for lastyrwrk, as it does not exist in 2022).

5

1 (Working)

2 (Not Last 12 Months)

Filled

Filled

Set:

emplyd_last_week to 000001 and

emplyd_last_week_grp2 to

2 Not Working

A respondent who said they are both currently working but haven’t worked in the last 12 months; however, routing is followed except for the Working Last Week question.  This is similar to the rule in 2011.

6

1 (Working)

2 (Not Last 12 Months)

Filled

Not Filled

Set:

emplyd_last_week to 000001 and

emplyd_last_week_grp2 to

2 Not Working

A respondent who said they are both currently working but haven’t worked in the last 12 months, however, routing is followed except for the Working Last Week question.  They should have filled B, so leave these as missing.  This is similar to the rule in 2011.

7

1 (Working)

2 (Not Last 12 Months)

Not Filled

Filled

No Action

A respondent who said they are both currently working but haven’t worked in the last 12 months.  Unable to follow routing/determine which answers pertain to, so no action.

8

1 (Working)

2 (Not Last 12 Months)

Not Filled

Not Filled

No Action

As above.

9

1 (Working)

3 (Never Worked)

Filled

Filled

No Action

A contradictory response where the respondent answers they are both working and have never worked.   However, they provided all Fields A and Fields B, so there is an indication that they have at least worked in the past.

 

In 2011, we changed the emplyd_last_week variables (to not working) and lastyrwrkd to missing (to allow E&I to resolve).  This question isn’t the same for 2022 so we will allow E&I to resolve.  In a way, this is similar to Item 1 (2022).

10

1 (Working)

3 (Never Worked)

Filled

Not Filled

Set:

emplyd_last_week to 000001

emplyd_last_week_grp2 to

2 Not Working

 

and

 

Fields B to

-5 No Code Required

A contradictory response where the respondent answers that they are working and have never worked.  Since they filled Fields A but not B, there is heavier indication that they have never worked (routing followed except for emplyd variables).  As such, we can resolve the inconsistency by changing the emplyd_last_week variables (and setting B to NCR).  This is similar to what we did in 2011.

11

1 (Working)

3 (Never Worked)

Not Filled

Filled

No Action

A contradictory response where the respondent answers they are both working and have never worked.  We don't have evidence whether the respondent is currently working or has not worked in the last 12 months so leave this for E&I to resolve (this is also what we did in 2011).

12

1 (Working)

3 (Never Worked)

Not Filled

Not Filled

No Action

A contradictory response where the respondent answers they are working and have never worked.  Can’t tell if routing was followed so leave for E&I to resolve.

13

1 (Working)

-7 (Invalid), or

-9 (Missing)

Filled

Filled

Set:

ever_worked to -5

Fields A to -5

Similar to 2011

14

1 (Working)

-7 (Invalid), or

-9 (Missing)

Filled

Not Filled

No Action

In 2011, we changed the emplyd variables to Not Working and ever_worked to missing to allow E&I to resolve.  This isn’t quite the same in 2022, so No Action to allow E&I to resolve.

15

1 (Working)

-7 (Invalid), or

-9 (Missing)

Not Filled

Filled

Set:

ever_worked to -5

Fields A to -5

Appropriate response - enforce routing.

16

1 (Working)

-7 (Invalid), or

-9 (Missing)

Not Filled

Not Filled

No Action

Not enough information - E&I to resolve.

17

2 (Not Working)

1 (Last 12 Months)

Filled

Filled

No Action

Appropriate response, no actions needed.

18

2 (Not Working)

1 (Last 12 Months)

Filled

Not Filled

No Action

Respondent should have filled B - No action, allow E&I to resolve.

19

2 (Not Working)

1 (Last 12 Months)

Not Filled

Filled

No Action

Respondent should have filled A - No action, allow E&I to resolve.

20

2 (Not Working)

1 (Last 12 Months)

Not Filled

Not Filled

No Action

Respondent should have filled A and B - No Action, allow E&I to resolve.

21

2 (Not Working)

2 (Not Last 12 Months)

Filled

Filled

No Action

Appropriate response, no actions needed.

22

2 (Not Working)

2 (Not Last 12 Months)

Filled

Not Filled

No Action

Respondent should have filled B - No action, allow E&I to resolve.

23

2 (Not Working)

2 (Not Last 12 Months)

Not Filled

Filled

No Action

Respondent should have filled A - No action, allow E&I to resolve.

24

2 (Not Working)

2 (Not Last 12 Months)

Not Filled

Not Filled

No Action

Respondent should have filled A and B - No Action, allow E&I to resolve.

25

2 (Not Working)

3 (Never Worked)

Filled

Filled

No Action

Respondent has ticked they have never worked, but by filling B there is a strong indication they have done some work previously.  In 2011, we would have changed the ever_worked variable to No, but this isn’t the same in 2022 so leave for E&I to resolve.

26

2 (Not Working)

3 (Never Worked)

Filled

Not Filled

No Action

Appropriate response - no action needed.

27

2 (Not Working)

3 (Never Worked)

Not Filled

Filled

No Action

In 2011, we changed ever_worked to missing to allow E&I to resolve.  This isn’t quite the same in 2022, so No Action to allow E&I to resolve.

28

2 (Not Working)

3 (Never Worked)

Not Filled

Not Filled

Set:

Fields B to

-5 No Code Required

Respondent has largely followed routing, except they should have filled Fields A.  Leaving these unchanged will be resolved in E&I, but routing can be enforced here by setting B to NCR.  This is similar to what was done in 2011.

29

2 (Not Working)

-7 (Invalid), or

-9 (Missing)

Filled

Filled

 No Action

In 2011, we changed ever_worked to Yes and allowed E&I to resolve lastyrwrkd.  However, this isn’t quite the same in 2022, so No Action allows E&I to resolve.

30

2 (Not Working)

-7 (Invalid), or

-9 (Missing)

Filled

Not Filled

No Action

E&I to resolve - this is the same as 2011.

31

2 (Not Working)

-7 (Invalid), or

-9 (Missing)

Not Filled

Filled

No Action

In 2011, we changed ever_worked to Yes and allowed E&I to resolve lastyrwrkd.  However, this isn’t quite the same in 2022, so No Action allows E&I to resolve.

32

2 (Not Working)

-7 (Invalid), or

-9 (Missing)

Not Filled

Not Filled

No Action

Not enough information - E&I to resolve.

33

-7 (Invalid), or

-9 (Missing)

1 (Last 12 Months)

Filled

Filled

Set:

emplyd_last_week to 000001

emplyd_last_week_grp2 to

2 Not Working

Aligns with routing; similar to 2011.

34

-7 (Invalid), or

-9 (Missing)

1 (Last 12 Months)

Filled

Not Filled

Set:

emplyd_last_week to 000001

emplyd_last_week_grp2 to

2 Not Working

Aligns with routing, although respondent should have filled Fields B - E&I to resolve this. 

35

-7 (Invalid), or

-9 (Missing)

1 (Last 12 Months)

Not Filled

Filled

No Action

E&I to resolve - this is also what we did in 2011.

36

-7 (Invalid), or

-9 (Missing)

1 (Last 12 Months)

Not Filled

Not Filled

No Action

Not enough information - E&I to resolve.

37

-7 (Invalid), or

-9 (Missing)

2 (Not Last 12 Months)

Filled

Filled

Set:

emplyd_last_week to 000001

emplyd_last_week_grp2 to

2 Not Working

As 2011.

38

-7 (Invalid), or

-9 (Missing)

2 (Not Last 12 Months)

Filled

Not Filled

Set:

emplyd_last_week to 000001

emplyd_last_week_grp2 to

2 Not Working

Aligns with routing, although respondent should have filled Fields B - E&I to resolve this. 

39

-7 (Invalid), or

-9 (Missing)

2 (Not Last 12 Months)

Not Filled

Filled

No Action

E&I to resolve.  This is what was done in 2011.

40

-7 (Invalid), or

-9 (Missing)

2 (Not Last 12 Months)

Not Filled

Not Filled

No Action

Not enough information - E&I to resolve.

41

-7 (Invalid), or

-9 (Missing)

3 (Never Worked)

Filled

Filled

No Action

Contradictory information - respondent indicates they have never worked, but filled in Fields B (which indicates they have worked).  E&I to resolve.

42

-7 (Invalid), or

-9 (Missing)

3 (Never Worked;)

Filled

Not Filled

Set:

emplyd_last_week to 000001

emplyd_last_week_grp2 to

2 Not Working

and

Fields B to

-5 No Code Required

Generally aligns with appropriate routing.

43

-7 (Invalid), or

-9 (Missing)

3 (Never Worked)

Not Filled

Filled

No Action

Contradictory information - respondent indicates they have never worked, but filled in Fields B (which indicates they have worked).  E&I to resolve.

44

-7 (Invalid), or

-9 (Missing)

3 (Never Worked)

Not Filled

Not Filled

Set:3

 

emplyd_last_week to 000001

emplyd_last_week_grp2 to

2 Not Working

and

Fields B to

-5 No Code Required

Generally aligns with appropriate routing, though E&I will resolve Fields A.

45

-7 (Invalid), or

-9 (Missing)

-7 (Invalid), or

-9 (Missing)

Filled

Filled

Set:

emplyd_last_week to 000001

emplyd_lastflag_week_grp2 to

2 Not Working

Filling Fields B indicates that there is a history of work; however, leaving the ever_worked variable for E&I to resolve.

46

-7 (Invalid), or

-9 (Missing)

-7 (Invalid), or

-9 (Missing)

Filled

Not Filled

No Action

Not enough information - E&I to resolve.

47

-7 (Invalid), or

-9 (Missing)

-7 (Invalid), or

-9 (Missing)

Not Filled

Filled

No Action

E&I to resolve.

48

-7 (Invalid), or

-9 (Missing)

-7 (Invalid), or

-9 (Missing)

Not Filled

Not Filled

No Action

Not enough information - E&I to resolve.

 

Dependent Fields A

Dependent Fields B

other_act

emp_stat

look_work

employer

avail_work

occupation

wait_work

industry

 

supervisor

 

hours_worked

Filter Rule L  - Work or Study More

 

The online questionnaire has a series of sub-questions around working and studying which determines which activity the respondent does more if they have indicated they both work and study.  This is because the “Travel to Work” address response should apply to the place where the respondent spends the most time.  However, there is no such indicator on a paper questionnaire.  For the most part, the Filter Rules can’t make this determination, except in a couple of straightforward cases, detailed here.

 

Item

Variable A

Variable B

Variable C

Action

(Then…)

Notes

(If ft_student is…)

(If other_act is…)

emplyd_last_week_grp2

1

1 (Yes)

2 (Studying)

1 (Working)

 

No Action

 

Unable to determine if respondent works or studies more.

2

1 (Yes)

2 (Studying)

2 (Not Working)

Set:

work_study to 2 (Study)

 

3

1 (Yes)

Anything other than 2 (Studying)[1]

1 (Working)

 

No Action

 

Unable to determine if respondent works or studies more.

4

1 (Yes)

Anything other than 2 (Studying)

2 (Not Working)

No Action

Unable to determine if respondent works or studies more.

5

2 (No)

2 (Studying)

1 (Working)

No Action

Unable to determine if respondent works or studies more.

6

2 (No)

2 (Studying)

2 (Not Working)

No Action

Unable to determine if respondent works or studies more.

7

2 (No)

Anything other than 2 (Studying)

1 (Working)

Set:

work_study to 1 (Work)

 

8

2 (No)

Anything other than 2 (Studying)

2 (Not Working)

No Action

Not enough information



[1] Including Invalid or Missing

Filter Rule M  - Method of Travel

 

This rule is essentially the same as 2011, with the exception that travel options for “outside of the UK” are now set to “no code required”.

 

Item

Variable A

Variable B

Action

(Then…)

Notes

(If work_study_address is…)

(If method_travel is…)

1

1 Work from Home

Any Value

Set:

method_travel  to

-5 No Code Required

As in 2011

2

2 Equivalent

Any Value

Set:

method_travel  to

-5 No Code Required

This is similar to 2011, but we did not have the work_study_address option (2 Equivalent)

3

3 No Fixed Place

Any Value

 

No Action

 

As in 2011

4

4 Offshore

Any Value

 

No Action

 

As in 2011

5

5 Another Address

Any Value

 

No Action

 

As in 2011

6

6 Outside UK

Any Value

Set:

method_travel  to

-5 No Code Required

This is new for 2022 and will mirror the routing on OCI

7

-7 Invalid, or

-9 Missing

Any Value

No Action

As in 2011

Filter Rule N  - Travel or Work from Home (Transportation Question Recode Option)

 

There is one situation where the Filter Rules will create a new code for an (inherent) answer option in order to prevent an inconsistency.  The “Travel to Work or Study” question (42) gives options for work or studying from home, however, the “Method of Travel” question (43) does not give a tick option for this.  In these circumstances, rather than setting a –5 (no code required), the related filter rule (N) will create the code “11” (Work or Study from home).  This was done in 2011 as well for E&I purposes. 

 

Item

Variable A

Variable B

Action

(Then…)

Notes

(If work_study_address is…)

(If method_travel is…)

1

1 Work from Home

Any Value

Set:

method_travel  to

11 (Work or Study from Home)

As in 2011

2

2 Equivalent

Any Value

Set:

method_travel  to

11 (Work or Study from Home)

This is similar to 2011, but we did not have the work_study_address option (2 Equivalent)

3

3 Any other option

Any Value

 

No Action

 

As in 2011

4

-7 Invalid, or

-9 Missing

Any Value

No Action

As in 2011

3. Conclusion

 

Filter Rules were successfully completed as part of Scotland’s Census 2022.

As a feature of the online questionnaire, respondents are simply brought to the next question, based on the answer(s) given in preceding questions.  This prevents the respondent from seeing unnecessary questions, which reduces respondent burden and minimises the ability to provide inconsistent answers.  On paper however, a respondent is able to see all of the questions at once, providing the opportunity to create these inconsistencies. To minimise the potential bias between the two modes of completion, all responses (both online and paper) will be processed through the filter rules.

 

Filter Rules is required as a precursor to Edit and Imputation. This is to ensure consistency in the application of questionnaire routing and resolve inconsistencies in responses, which in turn relieve some of the burden from Edit and Imputation processing, and maximise the E&I donor pool.


Contents