Unexpected Research Findings - Reasons and Consequences |
|
| Coordinator 1 | Dr Uta Landrock (LIfBi – Leibniz Institute for Educational Trajectories) |
| Coordinator 2 | Dr Detlev Lück (Federal Institute for Population Research (BiB)) |
In social science research, we do not always find what we expect, but sometimes surprising and disappointing results. Instead of the theoretically predicted effects, we may find unpredicted ones or no effect at all. Sometimes results even completely contradict our hypotheses. Given that such outcomes are unpleasant and difficult to publish, we assume that they occur much more often than we hear about them.
We do not want to complain about this problem (not only!), but also discuss possible reasons as well as possible ways how to deal with this issue. Reasons may include limited data quality, unavailable information or insufficient analytical potential of our data. For example, we may not be able to consider multilevel structures due to low case numbers or because statistical requirements are not met. Problems of data availability may include indicators not included in the questionnaire or unobserved third variables. Unexpected research results may also occur when fieldwork procedures or the research design are documented insufficiently.
Regarding the issue of how to deal with unexpected findings, one concern is to give them more visibility. This is the precondition for improving future research instead of repeating failures that others have secretly made before. A second concern is to avoid common reasons for failures. Perhaps the most important advice is to carefully consider what information we need to apply a particular theoretical framework or to address a particular research question before collecting the data. In the face of limited data availability or analytical potential, new and alternative data sources may offer opportunities to fill these gaps.
We invite researchers to participate in this discussion and to learn from each other in order to avoid disappointments.