Full Text:

INTRODUCTION

Over the last decade, many efforts have been exerted towards defining and measuring the concept of pathological involvement with computer or video games. Although playing video games is not considered intrinsically pathologic or problematic, gaming can become pathological for some players when the activity becomes dysfunctional, harming an individual’s social, occupational, familial, academic, and psychological functioning (1). In general, “pathological gaming” can be described as persistent, recurrent, and excessive involvement with computer or video games that cannot be controlled, despite associated problems (2,3). Most recent studies on “game addiction” or similar constructs have adapted the definition and six criteria for “pathological gambling” from the fourth edition of the Diagnostic and Statistical Manual of Mental Disorders (DSM-IV) (4), and many have therefore applied the term “pathological gaming” to this type of behavior (5-12). These six criteria (preoccupation, tolerance, withdrawal, persistence, escape, and problems) can also be found among the proposed criteria for IGD in the latest (fifth) edition of the DSM (DSM-5). In addition to these six criteria, the DSM-5 includes three other criteria that are diagnostic indicators of dysfunction: deception (1,13), displacement (14,15), and conflict (3). After careful consideration by a multidisciplinary expert workgroup, the APA decided on the tentative term “Internet gaming disorder” in the DSM (16).

The DSM-5 (17) included Internet gaming disorder (IGD) as a condition that needs further research before being fully recognized and accepted as an independent disorder in subsequent publications of the DSM (16). Although the disorder is labeled “Internet” gaming disorder, the DSM-5 states that “Internet gaming disorder most often involves specific Internet games, but it could involve non-Internet computerized games as well, although these have been less researched” (17). Of the nine criteria, seven are identical to those of gambling disorder and five to substance use disorder (18), and refer to preoccupation with Internet games, withdrawal symptoms, tolerance, unsuccessful attempts to control participation in Internet games, loss of interest in previous hobbies, continued excessive use of Internet games, deceiving family members, the use of Internet games to escape, and loss of a significant relationship, job or education, or career opportunity. To be diagnosed as a disordered gamer, five (or more) out of these criteria need to be endorsed over a period of 12 months (17).

Although IGD is not yet recognized as an official clinical entity (16), several new promising psychometric tools covering the nine IGD criteria have been developed following the formal recognition of this condition by the APA in May 2013 (12-21).

The study of Lemmens et al. (12) was among the first to begin an empirical examination of the new definition of IGD from the DSM-5, in which they developed new scale titled the Internet Gaming Disorder Scale (IGDS). Since the nine criteria for IGD are described in the DSM-5 in very broad terms (17), several researchers have suggested that to distinguish specific aspects of DSM criteria (e.g., relationship, job, education), items can be broken into discrete components (18). Thus, to provide information about which specific aspect of a broadly defined criterion matches the concept of disordered gaming, Lemmens et al. (12) measured each of the nine DSM-5 definitions with three items, either through separating core aspects of a criterion into different items or by applying slight modifications in phrasing or synonyms. Lemmens et al. (12), in their online survey study, aimed to statistically determine whether sum scores of affirmative answers on a dichotomous scale and mean scores on a polytomous measurement can both provide valid and reliable measurements in order to obtain a valid and reliable instrument for IGD suited for diagnostic and research purposes. To facilitate incorporation of both 27-item IGD scales into space-limited surveys, they further aimed to investigate whether a model with fewer items would provide an equal or even better description of the data. To create short versions of the scales, the highest loading item from each criterion was selected to create nine-item versions of the scales that encompassed all criteria. Thus, they evaluated the reliability and validity of 4 survey instruments to measure IGD on the basis of the 9 criteria from the DSM-5: a long (27-item) and short (nine-item) polytomous scale and a long (27-item) and short (nine-item) dichotomous scale.

Although the short Turkish version (seven-item) (22) of the 21-item (long version) Digital Game Addiction Scale (3) was validated only among a small number of adolescents (n=95), this was an earlier scale that did not correspond to DSM-5. The aim of the present study has been to evaluate the psychometric properties of the 27-item and nine-item IGDS in Turkish university students and online game players.

METHOD

Participants and Procedure

The psychometric properties of the IGDS were tested through a cross-sectional online self-report survey among volunteer university students in Ankara and active Internet gamers as well as individuals who are in the e-mail database of an Istanbul-based company which organizes e-sports tournaments (ESL Turkey Amateur e-sport players). A website was prepared for online participation. Approval was taken from the Ethical Committee of the Cankaya University. The students were asked to fill out the form on the website anonymously. Informed consent was approved by students online before continuing with further questions. Exclusion criteria were incomplete forms.

A total of 457 participants were included in the study. Among these 172 were male (37.6%) and 285 were female (62.4%), 6 (1.3%) were elementary school graduates, 24 (5.3%) were highschool graduates, 298 (65.2%) were university students and 129 (28.2%) were university graduates.

Among the participants, 7 (1.5%) are e-sports professional gamers (regularly receive a monthly salary), 14 (3.1%) are e-sports amateur gamers (have a team and participate in tournaments and make money in tournaments), 138 (30.2%) play games for their own pleasure and/or follow e-sports, 111 (24.3%) are university students who play games on the Internet and 187 (40.9%) are university students who do not play games on the Internet. The average number of hours spent playing online games (not for work / not academical) per day over the past year was less than 1 hour for 280 (61.3%) more than 1 hour, less than 3 hours for 93 (20.4%), more than 3 hours, less than 6 hours for 52 (11.4%), more than 6 hours, less than 9 hours 26 (5.7%) and more than 9 hours for 6 (1.3%).

Measures

The Internet Gaming Disorder Scale: Lemmens et al. (12) created three items for each of the nine previously identified criteria—preoccupation, tolerance, withdrawal, persistence, escape, problems, deception, displacement, and conflict— resulting in a total of 27 items (12). The dichotomous and polytomous IGDSs both consisted of the same items, differing only in their response options. Both the dichotomous and polytomous IGD scales were valid and reliable. But since we conducted the present study for polytomous IGD scales, we only shared the psychometric characteristics of these scales.

According to the DSM-5, IGD is present when a person meets five (or more) of the nine criteria during a period of 12 months (17). In accordance with this temporal rule, every item on both IGD scales was preceded by this statement: “During the last 12 months...” Respondents who were administered with the polytomous IGDS rated all items on a six-point ordinal-frequency scale: (0) never, (1) one to four times in the last year, (2) five to 11 times in the last year, (3) about once to three times a month, (4) once or more a week, and (5) every day or almost every day (12).

The 27-item polytomous IGDS showed good reliability with a Cronbach’s alpha of 0.94. The unconstrained model for nine-item polytomous IGDS yielded an acceptable model fit, χ2= 112.240, p=lt;0.001, CFI=0.988, WRMR=0.810, RMSEA=0.080. The short, nine-item polytomous IGDS was strongly correlated with the 27-item polytomous IGDS (r=0.98, plt;0.001) and showed good reliability with a Cronbach’s alpha of 0.95.

Young’s Internet Addiction Test - Short Form (YIAT-SF): YIAT-SF, developed by Young (23) and transformed into a short form by Pawlikowski et al. (24) is composed of 12 items and is a type of five-point Likert scale (1=Never, 5=Very frequent). Confirmatory factor analysis revealed that YIAT-SF resulted in an acceptable model fit (χ2=173.58, sd=53, CFI=0.95, SRMR=0.064 and RMSEA=0.079). The internal consistency reliability coefficient of the scale was 0.85. The results show the validity and reliability of the YIAT-SF. High scores on the scale indicate that the level of Internet dependency is high. The Turkish version of the YIAT-SF is a reliable and valid scale for both university students and adolescents (25). Cronbach’s alpha in the present study was 0.90.

Translation

Two experts in psychiatry independently translated the original Internet Gaming Disorder Scale (IGDS) from English into Turkish after receiving permission from the developers of the scale to translate it into Turkish and to conduct a validity and reliability study. These experts reached a consensus on a common draft. The final translation was then presented to 10 students who play online games to determine whether the language was clear. This Turkish version was translated back into English by an independent translator and sent to the developer of the scale who confirmed its accuracy.

Statistical Analysis

The following strategies were used to investigate the psychometric properties of both the 27-item and nine-item IGDS: (a) its factorial structure was examined using a exploratory factor analysis (EFA) then confirmatory factor analysis (CFA); (b) Test-retest for both the long and short versions were conducted and significance of the baseline and retest scores of IGDS were evaluated by using a Paired- Samples t test; (c) criterion-related validity was evaluated by calculating a Pearson product– moment correlation between the IGDS and IAS and time spent playing games; (d) internal consistency reliability was assessed using Cronbach’s alpha.

RESULTS

Factorial Structure

To explore the unidimensional construct validity of both the 27-item and the nine-item IGDS, EFA then CFA for the nine-item IGDS were conducted, involving all participants (n=457).

Prior to any further analysis, the adequacy of sample size was verified using Bartlett’s test of sphericity and the Keiser-Meyer-Olkin (KMO) measurement of sampling adequacy. Bartlett’s test of sphericity was significant (χ2=13976.521, df=351, plt;0.001) for the 27-item IGDS, and the KMO measure of sampling adequacy was acceptable at 0.957. The single component on the 27-item IGDS reached the criterion of an Eigenvalue greater than one (16.255), and the variance accounted for by this component was 60.20% (Table 1).

As was conducted in the original study, to facilitate the incorporation of the 27-item IGDS into space-limited survey, we wanted to investigate whether a model with fewer items would provide an equal or even better description of the data. As with the original study conducted to create short version of the scale, the highest loading item from each criterion was selected to create nine-item versions of the scale that encompassed all criteria (Table 2). This nine-item short version of the scale was then tested for unidimensionality. Bartlett’s test of sphericity was significant (χ2=2976.725, df=36, plt;0.001) for the nine-item IGDS, and the KMO measure of sampling adequacy was acceptable at 0.926. A single component on the nine-item IGDS reached the criterion of an Eigenvalue greater than one (5.926), and the variance accounted for by this component was 65.85% (Table 2).

The unidimensionality of the nine-item scale was then assessed with CFA. Estimation of the model produced a good fit (χ2/df=57.45/20=2.87; root mean square error of approximation [RMSEA]=0.064, goodness of fit index [GFI]=0.973, adjusted GFI=0.940, parsimony GFI=0.433, normed fit index [NFI]=0.981, comparative fit index [CFI]=0.987 and incremental fit index [IFI]=0.987). As generally accepted, we took as our criteria: Chi-Square/df≤5, gt;0.90 for GFI, CFI, NFI and IFI, and for RMSEAlt;0.05 being perfect when evaluating the fit index (26,27). As seen in Table 2, all item-component loadings were in the “good” to “excellent” range. Thus, results from the PCA and the CFA suggest that the nine-item IGDS assesses a unidimensional construct.

Criterion-related Validity and Internal

Consistency Reliability

Test-retest correlation for the 27-item IGDS and the nine-item IGDS were high (n=261; r=0.759, plt;0.001; r=0.756, plt;0.001 respectively). Mean scores of baseline (15.61±22.74) and retest (16.51±22.50) of the 27-item IGDS did not differ significantly (n=261; t=-0.925, 0=0.356). Mean scores of baseline (5.39±8.06) and retest (5.55±7.69) of the nine-item IGDS also did not differ significantly (n=261; t=-0.461, 0=0.645) (not shown).

Correlation coefficient between the 27-item IGDS and the nine-item IGDS was very high (n=457; r=0.988, plt;0.001) as was the correlation of retests of the 27-item IGDS and the nine-item IGDS (n=261; r=0.988, plt;0.001) (Table 3).

The Pearson product–moment correlation between the YIAT-SF and the 27-item IGDS (n=457; r=0.403, plt;0.001) and the nine-item IGDS (n=457; r=0.390, plt;0.001) were mild. Moderate correlations were found between time spent daily on the Internet and both the 27-item IGDS (n=457; r=0.564, plt;0.001) and the nine-item IGDS (n=457; r=0.556, plt;0.001) (Table 3).

Internal consistency for the 27-item IGDS (coefficient α=0.973) and nine-item IGDS (coefficient α=0.931) examined by Cronbach’s alpha, were high (Table 1 and 2).

Item-total correlations for the 27-item IGDS ranged between 0.576 (Item 27- Conflict 3) and 0.852 (Item 4- Tolerance 1) (Table 1). Inter-item correlations for the nine-item IGDS ranged between 0.506 (between 12 [Persistence 3] and 14 [Escape 2]) and 0.797 (between 2 [Preoccupation 2] and 4 [Tolerance 1]) (not shown). Item-total correlations for the nine-item IGDS ranged between 0.663 (Item 19- Deception 1) and 0.826 (Item 4- Tolerance 1) (Table 2).

In the present study, using nine-item IGDS and accepting the answers of “(4) once or more a week, and (5) every day or almost every day” to meet the criteria of each item, prevalence of IGD was 4.2% (n=19), since according to DSM-5, individials with the presence of at least 5 items among 9 items is considered as IGD. Among these 7 were e-sports amateur gamers (50.0%), 9 were those who play games for his/her own pleasure and/or follow e-sports (6.5%), and 3 were university students who play games on the Internet (2.7%). 27-item IGDS, nine-item IGDS and Internet scores were higher among those with probable IGD (Table 4). Also accepting the answers of “(3) about once to three times a month, (4) once or more a week, and (5) every day or almost every day” to meet the criteria of each item, the prevalence was 9.2% (n=42).

DISCUSSION

Because Turkish measurement tools are needed for research and diagnostic purposes among young adults regarding IGD that correspond to DSM-5, the main aim of the current study was to test the reliability and validity of both the 27-item and nine item IGDS with polytomous response option (12). This was tested using a cross-sectional online self-report survey among volunteer university students in Ankara and online game players, as well as individuals who are in the e-mail database of an Istanbul-based company which organizes e-sports tournaments. As in the original study, the most suited item from each of the nine criteria was selected for the short, nine-item versions of the scale on the basis of the factor loadings. A single component on the nine-item IGDS attained the criterion of an eigenvalue greater than one (5.926), and the variance accounted for by this component was 65.85%. Also, the use of CFA showed good model fits, providing further support for the unidimensional structures of the scale. These suggest that the scale has good psychometric properties and solid structural validity. The criterion-related validity of both the long and short versions of the scale was indicated by the significant correlations with time spent on games and IAS score. In general, higher mean scores on the scales indicated more time spent on games and a higher severity of Internet addiction. Thus, the short nine-item version of the scale provided a valid and reliable measure of IGD with good diagnostic accuracy that can be used for research and diagnostic purposes among male and female gamers of young adults.

In the original study, both the 27-item (Cronbach’s alpha of 0.94) and nine-item (Cronbach’s alpha of 0.95) polytomous IGDS showed good reliability with the long and short versions strongly correlating with each other (r=0.98, plt;0.001). Consistent with these Cronbach’s alpha for the 27-item IGDS was 0.97, Cronbach’s alpha for nine-item IGDS was 0.93, and these scales were strongly correlated with each other (r=0.99, plt;0.001). Also these scales were mildly correlated with YIAT-SF and moderately correlated with time spent daily playing games. Thus, the results suggested that IGDS has strong psychometric properties, including high internal consistency and criterion-related validity.

The prevalence of IGD found in the United States (8.5%; 6), Germany (11.9%; 28), Taiwan (7.5%; 29) and Singapore (8.7%; 30) ranged between 7.5% and 11.9%. Using the sum scores of dichotomous answers on the nine-item scale to assess the prevalence of IGD when the DSM-5 cutoff point of five or more (out of nine) was administered, Lemmens et al. (12) found the prevalence of IGD among the Dutch sample (ages 13-40 years) as 5.4%. In the present study, using nine-item IGDS and accepting the answers of “(4) once or more a week, and (5) every day or almost every day” in order to meet the criteria of each item, prevalence was 4.2% (n=19), which was consistent with Dutch study, whereas accepting the answers of “(3) about once to three times a month, (4) once or more a week, and (5) every day or almost every day” to meet the criteria of each item, prevalence was 9.2% (n=42) and was consistent with other studies. However, despite the high reliability and validity of the short version of IGDS, this version is derived from specific aspects of the broad DSM criteria. Therefore, prevalence rates based on these scales should be interpreted with caution until their items are validated in a clinical or diagnostic setting. Also, while Lemmens et al. (12) used the nine-item IGDS with dichotomous answers, our nine-item IGDS had 5 different items as compared to the original scale (since we used the higher loadings for each criteria found in the present study) and we used the polytomous IGD scale, which was done to provide more information on the severity of the addiction than dichotomous answers alone can provide. Some suggested that the raising the threshold for diagnosis to six out of nine may be appropriate so as to avoid overdiagnosing disordered gamers (12). Using this threshold the prevalence was found to be 6.6% (n=30) in the present study. Regarding the appropriateness of the nine IGD criteria, consistent with the original study, gamers in general are much less likely to experience conflict (2.6%) than escape (14.2%), indicating that not all criteria are equally prevalent and that some items may provide better discriminative power when diagnosing this disorder than others. Future studies conducted in Turkey may examine how increasing the threshold for diagnosing disordered gamers to six or more criteria will affect prevalence estimates and relations with criterion-related variables.

Several shortcomings of our study should be addressed. First, the use of an online survey excludes people who do not have access to the Internet. Nevertheless, in the present study participants were reached by e-mail, which may suggest that the whole sample may have access to the Internet. Another shortcoming of the study is that we did not examine the convergent validity of our IGD measure (i.e., we did not empirically examine the relationship between our IGD measures and similar measures). Including multiple measures for IGD, game addiction, or problematic gaming in a survey would allow for comparison of the psychometric properties of these instruments and their underlying criteria. Some of the items selected for the short scale covered a specific aspect of a criterion, as suggested by Petry and O’Brien (16), thereby excluding other aspects. Although the specific items selected for the nine-item scale in the original study may have adequately reflected the whole sample of 13-40-year-old men and women, as the authors suggested, the most suitable item from each criterion may differ between genders and age groups. Thus, we evaluated which specific aspect of a criterion may be more suitable for diagnosing IGD among our sample. Future studies may indicate whether certain aspects are more suitable for diagnosing IGD among specific demographic groups. Finally, in the original study, although both dichotomous and polytomous versions of the scale were found to be valid and reliable, particularly the dichotomous nine-item IGD scale was suggested to have solid psychometric properties and was deemed as the most practical scale for diagnostic purposes. In an effort to increase the variance, and thereby the predictive power of the measurement in survey research, we used polytomous responses through Likert-type response options (6,31). Thus, future studies may indicate whether dichotomous or polytomous Turkish versions are more suitable for diagnosing IGD.

Despite these limitations, the results of the validity and reliability testing of the Turkish polytomous versions of the both 27-item and nine-item IGDS were found to be similar to the findings of the original scale. These findings support the Turkish version of the IGDS, particularly the nine-item version, which measures a unidimensional construct as being a valid and reliable IGD screening instrument in determining problematic IGD among young adults, for purposes of early diagnosis and for use in other relevant research.

Conflict of Interest: Authors declared no conflict of interest.

Financial Disclosure: Authors declared no financial support.

REFERENCES

1.Gentile DA, Choo H, Liau A, Sim T, Li D, Fung D, Khoo A. Pathological video game use among youth: a two-year longitudinal study. Pediatrics 2011; 127:e319-e329. [CrossRef]

2.Griffiths M. A “components” model of addiction within a biopsychosocial framework. J Subst Use 2005; 10:191-197. [CrossRef]

3.Lemmens JS, Valkenburg PM, Peter J. Development and validation of a game addiction scale for adolescents. Media Psychol 2009; 12:77-95. [CrossRef]

4.American Psychiatric Association. Diagnostic and statistical manual of mental disorders (4th ed., text rev.). Washington, DC: Author, 2000.

5.Chiou WB, Wan CS. Using cognitive dissonance to induce adolescents’ escaping from the claw of online gaming: the roles of personal responsibility and justification of cost. Cyberpsychol Behav 2007; 10:663-670. [CrossRef]

6.Gentile D. Pathological video-game use among youth ages 8 to 18: a national study. Psychol Sci 2009; 20:594-602. [CrossRef]

7.Johansson A, Gotestam KG. Problems with computer games without monetary reward: similarity to pathological gambling. Psychol Rep 2004; 95:641-650. [CrossRef]

8.Keepers GA. Pathological preoccupation with video games. J Am Acad Child Adolesc Psychiatry 1990; 29:49-50. [CrossRef]

9.Lemmens JS, Valkenburg PM, Peter J. The effects of pathological gaming on aggressive behavior. J Youth Adolesc 2011; 40:38-47. [CrossRef]

10.Lemmens JS, Valkenburg PM, Peter J. Psychosocial causes and consequences of pathological gaming. Comput Human Behav 2011; 27:144-152. [CrossRef]

11.King DL, Haagsma MC, Delfabbro PH, Gradisar M, Griffiths MD. Toward a consensus definition of pathological video-gaming: a systematic review of psychometric assessment tools. Clin Psychol Rev 2013; 33:331-342. [CrossRef]

12.Lemmens JS, Valkenburg PM, Gentile DA. The Internet Gaming Disorder Scale. Psychol Assess 2015; 27:567-582. [CrossRef]

13.Demetrovics Z, Urbán R, Nagygyörgy K, Farkas J, Griffiths MD, Pápay O, Kökönyei G, Felvinczi K, Oláh A. The development of the Problematic Online Gaming Questionnaire (POGQ). PLoS One 2012; 7:e36417. [CrossRef]

14.Huang Z, Wang M, Qian M, Zhong J, Tao R. Chinese internet addiction inventory: developing a measure of problematic internet use for Chinese college students. Cyberpsychol Behav 2007; 10:805-811. [CrossRef]

15.Rehbein F, Kleimann M, Mössle T. Prevalence and risk factors of video game dependency in adolescence: results of a German nationwide survey. Cyberpsychol Behav Soc Netw 2010; 13:269-277. [CrossRef]

16.Petry NM, O’Brien CP. Internet gaming disorder and the DSM-5. Addiction 2013; 108:1186-1187. [CrossRef]

17.American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders (5th ed.). Arlington, VA: American Psychiatric Association; 2013.

18.Petry NM, Rehbein F, Gentile DA, Lemmens JS, Rumpf HJ, Mößle T, Bischof G, Tao R, Fung DS, Borges G, Auriacombe M, González Ibánez A, Tam P, O’Brien CP. An international consensus for assessing internet gaming disorder using the new DSM-5 approach. Addiction 2014; 109:1399-1406. [CrossRef]

19.Király O, Sleczka P, Pontes HM, Urbán R, Griffiths MD, Demetrovics Z. Validation of the Ten-Item Internet Gaming Disorder Test (IGDT-10) and evaluation of the nine DSM-5 Internet Gaming Disorder criteria. Addict Behav 2017; 64:253-260. [CrossRef]

20.Pontes HM, Király O, Demetrovics Z, Griffiths MD. The conceptualisation and measurement of DSM-5 internet gaming disorder: the development of the IGD-20 test. PLoS One 2014; 9:e110137. [CrossRef]

21.Pontes HM, Griffiths MD. Measuring DSM-5 Internet gaming disorder: development and validation of a short psychometric scale. Comput Human Behav 2015; 45:137-143. [CrossRef]

22.Yalcin-Irmak A, Erdogan S. Validity and reliability of the Turkish version of the Digital Game Addiction Scale. Anatolian Journal of Psychiatry 2015; 16(Suppl.1):10-18. [CrossRef]

23.Young KS. Caught in the net: how to recognize the signs of internet addiction and a winning strategy for recovery. New York: John Wiley and Sons, 1998.

24.Pawlikowski M, Altstötter-Gleich C, Brand M. Validation and psychometric properties of a short version of Young’s Internet Addiction Test. Comput Human Behav 2013; 29:1212-1223. [CrossRef]

25.Kutlu M, Savci M, Demir Y, Aysan F. Turkish adaptation of Young’s Internet Addiction Test-Short Form: a reliability and validity study on university students and adolescents. Anatolian Journal of Psychiatry 2015; 17(Suppl.1):69-76. [CrossRef]

26.Byrne BM. Structural equation modeling with AMOS: Basic concepts, Applications and programming. Second ed., Taylor and Francis, New York, NY: 2010.

27.Hair JF, Anderson RE, Tatham RL, Black WC. Multivariate Data Analysis. Seventh ed., Prentice-Hall, Upper Saddle River, New Jersey, 2010.

28.Grüsser SM, Thalemann R, Griffiths MD. Excessive computer game playing: evidence for addiction and aggression Cyberpsychol Behav 2007; 10:290-292. [CrossRef]

29.Ko CH, Yen JY, Yen CF, Lin HC, Yang MJ. Factors predictive for incidence and remission of internet addiction in young adolescents: a prospective study. Cyberpsychol Behav 2007; 10:545-551. [CrossRef]

30.Choo H, Gentile DA, Sim T, Li D, Khoo A, Liau AK. Pathological video-gaming among Singaporean youth. Ann Acad Med Singapore 2010; 39:822-829.

31.Jap T, Tiatri S, Jaya ES, Suteja MS. The development of Indonesian online game addiction questionnaire. PloS One 2013; 8:e61098. [CrossRef]