Skip to main content

Table 5 Study Quality and Risk of Bias

From: Behavioural symptoms of mental health disorder such as depression among young people using Instagram: a systematic review

Study Quality and Risk of Bias

  

Risk of Bias: High/ Low/ Unclear

Imprecision

Inconsistency

Indirectness

Publication Bias

Quality of Evidence

S/No

Study

Selection

Detection

Attrition

Reporting

Serious?

Serious?

Serious?

Serious?

High, Medium, Low, Very Low

1

Khodarahimi & Fathi, 2017

“using a systematic random sampling method” (p. 125)

→ low risk Allocation concealment: n/a because only questionnaire used.

→ low risk

n/a, no experiment

→ low risk

“Ten participants dropped for incomplete and vague responses” (p. 125). No mention which groups of social media users they belonged

→ high risk

Results for all research questions published. However, depression not (barely) treated in the results section. Unclear.

→ middle risk

No. of participants vary widely in use of different Apps. E.g., WhatsApp users are 10x more than other Apps. Secondly, users were few for many Apps, E.g., only 14 for Instagram.

→ Serious

Not serious

Instagram is only a subcategory, while depression is barely mentioned in the conclusion.

→ Serious

Low

Medium

2

Weinstein, 2017

Random sequence generation.

Allocation bias: “I embedded a randomizer within the survey flow to assign participants randomly to one of three groups (between- subjects setting).” (p. 11)

→ low risk

N/A

“No significant differences in number of students in each of the three groups who provided partial versus full responses. Only one exception for percent of 9th graders, t(588) = −2.760, p = 0.006. Baseline characteristics did not jointly predict attrition.”(p. 11)

→ low risk

Low

Not serious

pre-test ran (p.13 f.)

→ Not serious

research questions answered well

→ Not serious

Low

Medium

3

Li et al., 2018

“We contacted local secondary schools to recruit participants. The sampling criteria of participants were: (1) female students in secondary schools in Singapore, (2) aged 12–18 years, and (3) Instagram users who have posted selfies.” (p. 5) → low risk

no experiment → low risk

7 cases were excluded for incompleteness → low risk

“High mean for the importance of peer feedback shows that teenage girls take a serious perspective on online peer feedback.” (p. 8) “M = 2.53 SD = 0.93” (p. 7)

→ middle risk

CI of 90% was employed for the structural model, error- rate, therefore, higher than usual, only some coping strategies were examined → not serious

I2 was not measured, chi-square was significant heterogenous, overlapping CIs not found → not serious

no evidence of indirectness → not serious

Low

Low

4

Sherlock & Wagstaff, 2018

no evidence of excluded cases

→ low risk

OAs were privy to the allocation of the participants; researchers did not need to intervene or interact with participants → low risk

no evidence of excluded cases

→ low risk

no detectable bias → low risk

Not serious

Many measures were taken to test the (inter) reliability of items. → Not serious

Outcomes focused on effect of females of all ages and males but does explicitly claim one. → Not serious

Low

Medium

5

Jeri-Yabar et al., 2019

Sample size using three different campuses, a stratified randomized sample with proportional fixation, only students → Unclear

N/A

The exclusion criterion was being clinically diagnosed with depression, younger than 18 and no answer. (p.15). of 397 students only 212 students left for final sample size, after 13 losses → low risk

Addiction to Internet Test was adapted, but it is not stated how exactly. Internet Addiction is different from social media addiction: no differentiation was stated clearly between social networking sites → high risk

no effect sizes besides correlation, little to none model fit analysis besides Shapiro-Wilk, which is an inconsistent measure

→ high risk

→ Not serious

→ Not serious

Evidence is weak.

→ Not serious

Medium

6

Frison & Eggermont, 2017

n/a → unclear

participants knew the study investigated their social media use and emotions –→ high risk

“The full information maximum likelihood procedure was used to estimate missing data” (p. 605). → medium risk

“We decided to include all participants, also those that did not complete the survey at both time points, (1) to increase statistical power and (2) to reduce the risk of problems encountered in SEM when sample sizes are small” (p. 605). → unclear

→ Serious

→ Serious

participants dropped out but still went into data analysis (p. 605). → Not serious

→ Serious

Medium

7

Stapleton, Luiz & Chatwin, 2017

chain sampling & omnibus survey

→ high risk

n/a → Unclear

clearly stated (p. 143) → low risk

very detailed statements of reporting → low risk

high number of participants → Serious

→ Serious

→ Not serious

→ Not serious

Low

8

Lup, Trub & Rosenthal, 2015

recruitment was posted on the first author’s Facebook page, then shared by other users → high risk

n/a → unclear

only one participant was removed because of being a “multivariate outlier” (p. 248).

→ low risk

detailed statements (p. 248 f.) → low risk

small number of a very homogenous number of participants → not serious

→ Serious

→ Serious

→ Serious

Low

9

Yang, 2016

“Research information was announced through e-mail” (p.704), no randomization mentioned → middle risk

n/a, no experiment

→ low risk

not mentioned. “all of them filled out an online self- report” (p.705) → middle risk

no detectable bias

→ low risk

a relatively high number of Instagram users & two established scales→ serious

→ serious

→ serious

n/a, one author?

→ unclear

Low