Investigating the Potential of Crowdsourcing Validation (COR-V) for a New Qualitative Data Collection Method

Researchers are continuously developing new or modified data collection methodologies to work with users with diverse needs and capabilities. However, external validation of these new methods is often missing because it can be a time-consuming and arduous process that does not fit into a research project’s timeline, nor is it the purpose of that research. As a result, other researchers may not be confident using these new methods, and the evolution of research methodologies falters. This thesis explores the opportunities to crowd-source the validation process for new User Centred Needs Elicitation Methods (UCNEMs) that are often created when working with older adults because existing methods can be insufficient. Toward this goal, a framework for the most common validation techniques used in UCNEMs has been devised. The framework for validation techniques is based on existing literature and content analysis. The most common techniques identified are Triangulation, Replication/Repetition, Case Study, Member Check, Peer Review, and Statistical Validation. The framework includes which techniques are used for different data collection methods. In a user study created to evaluate user preferences and opportunities for using these validation techniques by researchers, developers, and designers for a newly created UCNEM, named Show-and-Tell, Triangulation is found to be the preferred validation method. In addition, participants are willing to contribute different types of research documents such as summary data and publications that they have generated based on their use of Show-and-Tell in their research towards the validation of the new Show-and-Tell, hence a crowdsourced validation process.