Study Quality and Risk of Bias | ||||||||||
---|---|---|---|---|---|---|---|---|---|---|
Risk of Bias: High/ Low/ Unclear | Imprecision | Inconsistency | Indirectness | Publication Bias | Quality of Evidence | |||||
S/No | Study | Selection | Detection | Attrition | Reporting | Serious? | Serious? | Serious? | Serious? | High, Medium, Low, Very Low |
1 | Khodarahimi & Fathi, 2017 | “using a systematic random sampling method” (p. 125) → low risk Allocation concealment: n/a because only questionnaire used. → low risk | n/a, no experiment → low risk | “Ten participants dropped for incomplete and vague responses” (p. 125). No mention which groups of social media users they belonged → high risk | Results for all research questions published. However, depression not (barely) treated in the results section. Unclear. → middle risk | No. of participants vary widely in use of different Apps. E.g., WhatsApp users are 10x more than other Apps. Secondly, users were few for many Apps, E.g., only 14 for Instagram. → Serious | Not serious | Instagram is only a subcategory, while depression is barely mentioned in the conclusion. → Serious | Low | Medium |
2 | Weinstein, 2017 | Random sequence generation. Allocation bias: “I embedded a randomizer within the survey flow to assign participants randomly to one of three groups (between- subjects setting).” (p. 11) → low risk | N/A | “No significant differences in number of students in each of the three groups who provided partial versus full responses. Only one exception for percent of 9th graders, t(588) = −2.760, p = 0.006. Baseline characteristics did not jointly predict attrition.”(p. 11) → low risk | Low | Not serious | pre-test ran (p.13 f.) → Not serious | research questions answered well → Not serious | Low | Medium |
3 | Li et al., 2018 | “We contacted local secondary schools to recruit participants. The sampling criteria of participants were: (1) female students in secondary schools in Singapore, (2) aged 12–18 years, and (3) Instagram users who have posted selfies.” (p. 5) → low risk | no experiment → low risk | 7 cases were excluded for incompleteness → low risk | “High mean for the importance of peer feedback shows that teenage girls take a serious perspective on online peer feedback.” (p. 8) “M = 2.53 SD = 0.93” (p. 7) → middle risk | CI of 90% was employed for the structural model, error- rate, therefore, higher than usual, only some coping strategies were examined → not serious | I2 was not measured, chi-square was significant heterogenous, overlapping CIs not found → not serious | no evidence of indirectness → not serious | Low | Low |
4 | Sherlock & Wagstaff, 2018 | no evidence of excluded cases → low risk | OAs were privy to the allocation of the participants; researchers did not need to intervene or interact with participants → low risk | no evidence of excluded cases → low risk | no detectable bias → low risk | Not serious | Many measures were taken to test the (inter) reliability of items. → Not serious | Outcomes focused on effect of females of all ages and males but does explicitly claim one. → Not serious | Low | Medium |
5 | Jeri-Yabar et al., 2019 | Sample size using three different campuses, a stratified randomized sample with proportional fixation, only students → Unclear | N/A | The exclusion criterion was being clinically diagnosed with depression, younger than 18 and no answer. (p.15). of 397 students only 212 students left for final sample size, after 13 losses → low risk | Addiction to Internet Test was adapted, but it is not stated how exactly. Internet Addiction is different from social media addiction: no differentiation was stated clearly between social networking sites → high risk | no effect sizes besides correlation, little to none model fit analysis besides Shapiro-Wilk, which is an inconsistent measure → high risk | → Not serious | → Not serious | Evidence is weak. → Not serious | Medium |
6 | Frison & Eggermont, 2017 | n/a → unclear | participants knew the study investigated their social media use and emotions –→ high risk | “The full information maximum likelihood procedure was used to estimate missing data” (p. 605). → medium risk | “We decided to include all participants, also those that did not complete the survey at both time points, (1) to increase statistical power and (2) to reduce the risk of problems encountered in SEM when sample sizes are small” (p. 605). → unclear | → Serious | → Serious | participants dropped out but still went into data analysis (p. 605). → Not serious | → Serious | Medium |
7 | Stapleton, Luiz & Chatwin, 2017 | chain sampling & omnibus survey → high risk | n/a → Unclear | clearly stated (p. 143) → low risk | very detailed statements of reporting → low risk | high number of participants → Serious | → Serious | → Not serious | → Not serious | Low |
8 | Lup, Trub & Rosenthal, 2015 | recruitment was posted on the first author’s Facebook page, then shared by other users → high risk | n/a → unclear | only one participant was removed because of being a “multivariate outlier” (p. 248). → low risk | detailed statements (p. 248 f.) → low risk | small number of a very homogenous number of participants → not serious | → Serious | → Serious | → Serious | Low |
9 | Yang, 2016 | “Research information was announced through e-mail” (p.704), no randomization mentioned → middle risk | n/a, no experiment → low risk | not mentioned. “all of them filled out an online self- report” (p.705) → middle risk | no detectable bias → low risk | a relatively high number of Instagram users & two established scales→ serious | → serious | → serious | n/a, one author? → unclear | Low |