What this lesson teaches
I can justify a sample size, how to select it, when to collect data and which methods to use.
Syllabus 9173, Fieldwork, 1. Pre-fieldwork stage. Students should be able to develop a plan that: establishes the primary and secondary data needed to examine the question/hypothesis posed; identifies appropriate methods to determine sample size, select sample and collect data.
I can tell accuracy, reliability and validity apart and say how to improve each one.
Syllabus 9173, Fieldwork, 1. Pre-fieldwork stage. Students should be able to develop a plan that: ensures accuracy and reliability of data collected.
Make a guess
In a survey, 80% of residents tick "agree" that a route is easy to walk. What do you actually know?
- That those asked agreed with that wording.
- Nothing, because opinions are useless as evidence.
- That residents are about 80% satisfied with it.
- That the route is easy for every resident.
Show the answer
That those asked agreed with that wording.
Likert answers are ranks shaped by wording and by who was asked. Check the statement, the sample and a second kind of evidence before concluding.
Rating scales turn opinions into numbers, so check their wording and test any conclusion with a second, different kind of evidence.
A Likert scale asks people how far they agree with a statement. For example: "I can walk to the polyclinic without difficulty." Strongly disagree, disagree, neutral, agree, strongly agree. Each answer gets a score from 1 to 5, so many opinions can be counted, compared and graphed. The scores are ranks, not true measurements. The gap between "agree" and "strongly agree" is not the same for everyone. So the median, or the percentage who agree, is safer to report than a precise mean.
Likert scales have known weaknesses. Some people tick the middle option to avoid choosing. Others tend to agree with whatever is written, especially if a student is standing beside them. A leading statement, such as "The new linkway has made walking much easier", pushes people towards agreeing. A double-barrelled statement asks two things at once: "The route is safe and shaded." Fix these by writing neutral, single-idea statements, mixing positive and negative wording, and adding a "don't know" option.
A bipolar scale places two opposite words at the ends of a line. Examples are noisy 1 2 3 4 5 quiet, and unsafe 1 2 3 4 5 safe. Fieldworkers often use it to score a place's environmental quality. It is quick, but it is subjective: two students may score the same street differently. Reliability improves if you describe what each score means, train everyone on the same example, and average the scores of several observers.
A recce, short for reconnaissance, is a short visit before the real fieldwork. It follows a desk study of maps and satellite images. On the recce you check that sites are safe and open to the public, and find good sampling points. You also time each task and note when places are busy. Then run a pilot, trying the questionnaire with a few people, and revise unclear questions before the main data collection.
Put these in order
Put the planning stages of a fieldwork investigation in order.
- Revise the questions and sampling plan
- Desk study of maps and satellite images
- Main data collection
- Pilot the questionnaire with a few people
- Recce visit to check safety, access and timing
Show the answer
- Desk study of maps and satellite images The desk study shows likely sites, so you know where to visit on the recce.
- Recce visit to check safety, access and timing Once sites and timings are known, you can try the questions on real people.
- Pilot the questionnaire with a few people The pilot shows which questions confuse people, so you can fix them.
- Revise the questions and sampling plan With a tested method, the main data collection gives usable results.
- Main data collection
Triangulation means checking a conclusion with two or more different kinds of evidence. In an older residents' study, Likert ratings, a route audit and short interviews can each show whether a route is hard to use. If all three agree, the conclusion is more valid. If they disagree, for example high ratings but a route with many steps, that is a finding to explain, not an error to hide. Perhaps only residents who can manage the steps were asked.
- Likert scale
- Agreement with a statement, scored 1 to 5; the scores are ranks, not measurements.
- Bipolar scale
- A score between two opposite words, such as noisy and quiet.
- Recce
- A short visit before fieldwork to check safety, access, sites and timing.
- Triangulation
- Testing one conclusion with two or more different kinds of evidence.
Step by step
Write neutral items
One idea per statement, no loaded words, and a don't know option.
Recce and pilot
Check sites and timing, then try the questions on a few people.
Triangulate
Compare ratings with an audit, observation or interviews.
Worked example: Repair a Likert item
Weak: "The wonderful new park connector is safe and pleasant." It is leading ("wonderful") and double-barrelled ("safe and pleasant"). Better: two statements, "I feel safe using the park connector after 7 pm" and "I enjoy walking along the park connector", each with strongly disagree to strongly agree and a don't know option.
Watch out for this
A mean Likert score of 3.6 proves that residents are satisfied.
Likert scores are ranks, and a mean can hide a split between very happy and very unhappy people. Report the share who agree, show the spread, and check it against other evidence.
Check your understanding
Ratings say a route is easy to walk, but the audit finds 42 steps and no lift. What is the best next step?
- Delete the audit, since the residents know their own route best.
- Check who was asked; people who avoid the steps may be missing.
- Average the audit and the ratings into one combined score.
Show the answer
Check who was asked; people who avoid the steps may be missing.
Yes. Disagreement between methods is a finding; triangulation asks why the evidence differs.