Novel computerized psychometric tests as primary screening tools for the diagnos

Ming Luo,Rui Mu,Jian-Fang Liu,Fei-Hu Bai

Ming Luo,Rui Mu,Jian-Fang Liu,Fei-Hu Bai,Department of Gastroenterology,People’s Hospital of Ningxia Hui Autonomous Region,Yinchuan 750021,Ningxia Hui Autonomous Region,China

Abstract

Key words:Minimal hepatic encephalopathy;Diagnosis;Psychometric test;Inhibitory control test;EncephalApp Stroop App;Critical flicker frequency

INTRODUCTION

Hepatic encephalopathy (HE) is a serious neuropsychiatric syndrome in patients with decompensated liver diseases and/or portosystemic shunting[1].It involves impairments in both executive and cognitive function,ranging from mild psychometric abnormalities to a coma.Based on the severity of HE symptoms,the syndrome is categorized into overt HE (OHE) and covert HE.Covert HE includes minimal HE (MHE) and WestHaven grade I HE[1].Patients with OHE present with neuropsychiatric symptoms including asterixis,dyspraxia,and even a coma.In contrast,patients with MHE present with a spectrum of trivial cognitive deficits in attention span,psychomotor speed,and working memory without obvious neurological symptoms[2].The prevalence of MHE has been reported in up to 50% of cirrhotic patients,and MHE impairs daily function,affects health-related quality of life,and induces a high risk of progression to OHE[3-6].

Although the early diagnosis of MHE has been shown to be beneficial for further treatment and follow-up,diagnosing MHE is hampered by the lack of standard and reliable diagnostic tests in clinical practice.The American Association for the Study of Liver Diseases updated guidelines in 2014 recommends the most commonly used paper-and-pencil psychometric test,psychometric hepatic encephalopathy score(PHES),for the diagnosis of MHE[1].Although the PHES have been validated in clinical studies,it requires physicians for administration and interpretation;thus,it cannot be rapidly and conveniently performed in a clinic.A survey by the American Association for the Study of Liver Diseases showed that most physicians have realized the necessity of MHE testing in high-risk cirrhotic patients;however,only a minority of cirrhotic patients are routinely tested for MHE[7].Therefore,simple and rapid tests that can be performed by clinic assistants would increase the rates of MHE testing in the initial clinical assessment and follow-up.

Psychometric tests can be computerized for simplification and performed in a clinic within just a few minutes.Recent studies showed that novel computerized psychometric tests have been used to screen MHE in cirrhotic patients,including the inhibitory control test (ICT),EncephalApp Stroop App (EncephalApp),and critical flicker frequency (CFF)[6,8-17].These tests have been verified to be rapid,simple,and reliable methods for screening MHE and for predicting the development of OHE compared with paper-and-pencil psychometric tests[6,8-17](Table 1).High-risk cirrhotic patients are initially screened for MHE using rapid,convenient,and highly sensitive computerized psychometric tests,and then the findings are validated by more detailed neuropsychometric tests (Figure 1).Here,we review the clinical evidence and advances of novel computerized psychometric tests for screening MHE.

INHIBITORY CONTROL TEST

Origin and operation

Inhibitory control is also widely known as response inhibition.It is an executive function,and self-control is one of the considerable aspects of inhibitory control.Executive functions of the prefrontal cortex,caudate nucleus,and subthalamic nucleus are found to be associated with inhibitory control cognition,and inhibitory control is impaired in addiction,schizophrenia,and traumatic brain injury[18-21].ICT is a neuropsychological test that evaluates subjects’ ability to override their natural and habitual behavioral responses to stimulus in order to select more appropriate behaviors to achieve their goals[22].

Table 1 Studies of novel computerized psychometric tests for screening minimal hepatic encephalopathy

The ICT can be performed by medical assistants after the training session using laptop computers with automatic analyzed systems,which greatly improves the applicability and convenience of this test in the clinical setting.A running stream of letters within which the letters X and Y are distributed is randomly running across a computer screen at 500 ms intervals.The subject is instructed to only respond upon seeing alternating X and Y (called targets) and to stay quiet as seeing nonalternating X and Y (called lures) (Figure 2)[6].Six test sessions including a total of 40 lures,212 targets,and 1728 random letters are generated in the training in approximately 15 min.It takes about 15 min to complete the ICT assessment.Response rates and reactiontimes of lures and targets are automatically calculated upon completion of the test.The results are reported in the form of a number and percentage respectively of incorrect lure responses,correct lure inhibitions,correct target responses,and incorrect target misses[11].ICT lures measure inhibition ability,and target accuracy reflects attention ability[11].A good psychometric performance will be indicated by a lower lure response,higher target response,and shorter reaction times of lures and targets.Therefore,the ICT evaluates response inhibition,and assesses a subject’s ability to inhibit lure responses and correctly respond to targets.A significant correlation was found between lures and targets with psychometric impairment in PHES components,which provides discrete measures yet assisting perspectives into impairment in MHE[8].

Figure 1 Flow diagram of screening and diagnosing minimal hepatic encephalopathy in high-risk cirrhotic patients.High-risk cirrhotic patients are screened by the inhibitory control test,EncephalApp Stroop App,or critical flicker frequency and are subsequently validated using the psychometric hepatic encephalopathy score (PHES).MHE:Minimal hepatic encephalopathy.

Figure 2 Ideograph of inhibitory control test.A continuous stream of letters is randomly displayed on the computer screen.The subject is instructed to respond if an X is preceded by a Y or a Y is preceded by an X (targets) and is inhibited to respond if an X is followed by an X or a Y followed by a Y (lures).

Diagnostic efficiency for MHE

The ICT has been used to screen MHE in the United States,Germany,Italy,and India(Table 1)[6,8-11,23,24].In the United States,with a cut-off value of ≥ 5 lures/person,Bajajet al[6]found that the ICT had a 90% sensitivity and specificity for screening MHE,and the area under the receiver operator characteristic (AUROC) was 0.902 when using the standard paper-and-pencil psychometric tests (SPT) as the reference standard.Although hepatitis C has been found to induce cognitive deficits,this study showed no significant distinction in ICT results between cirrhotic patients with hepatitis C and those with other etiologies[6,25,26].Moreover,there was no significant difference in ICT performance between patients with alcoholic cirrhosis and those with nonalcoholic cirrhosis[8].These findings extend the clinical application of the ICT across different etiologies of liver cirrhosis.The PHES norms depend on the age bracket;however,ICT results,including lures and target accuracy,were found to be independent of age[8,11].This is a crucial difference between the PHES and ICT,which may potentially simplify and generalize the ICT application for screening MHE.Additionally,compared with the learning effects of PHES,lure and target responses of ICT are closely associated between the initial and following testing sessions,which indicates that the ICT has good test-retest reliability[8,9,11].Because of the high sensitivity and convenience of performance,the ICT is an effective diagnostic test for MHE in clinics,and it may help physicians to decide whether to send a patient for further evaluation or to administer treatment.

In an Indian population,the ICT showed a sensitivity of 92.6% and a specificity of 78.5% for screening MHE with a cut-off value of≥14 lures when using the PHES as a reference standard[9].The cut-off value of Indian patients in this study was higher than that reported earlier in the United States population[6,8].Moreover,studies in India and Italy showed that the ICT had lower sensitivity,specificity,and AUROC for screening MHE compared to the results reported by Bajajet al[6,8,13,14].Therefore,Amodioet al[11]established a novel measurement method of ICT results in Italian patients,i.e.lures weighted by target accuracy,and it was shown to effectively identify MHE patients from the controls.When cirrhotic patients in India were given a questionnaire regarding which tests they would like to repeat during follow-up,only 12% of patients preferred the ICT over the PHES[27].An explanation for this difference may be that most cirrhotic patients included in these studies were from an underdeveloped area and had lower educational levels.Furthermore,these patients were less familiar with computer games than United States controls,and unfamiliarity with a computer may lower the preference for ICT repetition and would be a critical influencing factor for the ICT results[9,10,27].These results indicated that the ICT,like most psychometric tests,was influenced by demographic variables,and use of the ICT should be adjusted when it is performed in different populations.Most studies excluded cirrhotic patients receiving treatment with psychoactive medications such as benzodiazepines and antipsychotics.However,these psychoactive medications are used for treatment of 50% of patients with liver cirrhosis in real life,and this factor may disturb ICT performance and the sensitivity/specificity of unaffected patients observed[28].

Follow-up for MHE

The class of Child-Turcotte-Pugh (CTP) and the score of model for end-stage liver disease (MELD) are effective measurements for predicting survival in cirrhotic patients,and ICT results are closely related to both the CTP class and MELD score[9,29,30].The ability of the ICT for predicting the development of OHE and the cumulative survival is found to be equivalent to the SPT and PHES[6,9].Moreover,although the sample sizes for the transvenous intrahepatic portosystemic shunting(TIPS) and yogurt trial were limited,ICT results worsened after application of TIPS,and improved after yogurt treatment simultaneously with the SPT[8].In contrast,in the study by Tanejaet al[10]in India,there is no external validity of the ICT for diagnosing MHE due to the fact that there is no possible prediction of either survival or episodes of OHE during follow-up.The explanation to this finding could be attributed to the fact that there is no correlation of the ICT with the CTP class and MELD scores,which were adopted as the predictors of survival in this study.Additionally,several studies included patients with OHE history,and these studies combined the ICT results of cirrhotic patients with and without prior OHE[6,9-11].Cirrhotic patients with OHE history have a high risk for recurrent OHE;therefore,further studies have focused on the ability of the ICT to OHE development prediction in a larger sample space with cirrhotic patients without prior OHE[31,32].

Because of impaired driving skills,cirrhotic patients with MHE have a higherreported occurrence of motor vehicle accidents and traffic violations[33].Compared with the SPT,the ICT is significantly correlated with a higher rate of motor vehicle crashes in the last year and prospective follow-up for 1 year[34].Navigation skill is an important aspect of driving difficulties induced by MHE,which is also associated with impairment of ICT results[35].Therefore,the ICT may be superior to the SPT for screening MHE patients who are at risk of vehicle crashes during follow-up.Use of the ICT for screening MHE may reduce the risk of motor vehicle crashes.

ENCEPHALAPP STROOP APP

Origin and operation

EncephalApp originated from the Stroop test,whose reputation is attributed to its assessment of the Stroop effect.The Stroop test assesses psychomotor speed and inhibitory control.There comes into conflict between word reading and color naming in the Stroop test.The individual has to suppress the brain’s word reading activity in order to conduct the color naming activity[36].The Stroop test was previously performed using paper-and-pencil assessment methods,and it has been used to evaluate cognitive deficits due to brain damage,Alzheimer’s disease,and Parkinson's disease[37-39].Based on the Stroop test,Bajajet al[14]developed a computerized test,EncephalApp,for diagnosing MHE in 2013.The EncephalApp is operated by the iOS and Android system on the smartphone or computer,and the application and operational instructions of the EncephalApp can be easily downloaded from www.encephalapp.com.

The EncephalApp includes an easy “off” state and a difficult “on” state.In the “off”state,colored signs ### are presented on the screen,and subjects are required to identify the corresponding color (Figure 3A and 3B).In the “on” state,the app presents a discordant colored word;for example,the word “blue” will be presented in green color,and the correct answer is green (Figure 3C and 3D).The EncephalApp has practice runs before the test runs,and the order of tests are as follows:(1) Two practice“off” runs;(2) Five test “off” runs;(3) Two practice “on” runs;and (4) Five test “on”runs[14].It takes about 15 min to complete all subtests of EncephalApp.Whether the EncephalApp is in the “on” or “off” state,it requires subjects to concentrate on the visual stimulus and choose the appropriate color by pressing on the screen,and the response time is calculated using “on-time” and “off-time”.Recent studies have found that “on-time” and “off-time” were highly associated with a wide spectrum of cognitive aspects,including visuomotor coordination,set-shifting,and response inhibition,indicating that the PHES and EncephalApp evaluate several similar cognitive domains in patients with MHE[13,14].

Diagnostic efficiency for MHE

Clinical application of the EncephalApp for screening MHE was first reported by Bajajet al[14]in 2013.This study showed that the EncephalApp had a sensitivity of 78% and specificity of 90% for diagnosing MHE when using the PHES as the reference standard.In a subsequent study also conducted by Bajajet al[13],the EncephalApp identified cirrhotic patients with MHE with an AUROC value of 0.91,and the AUROC value was 0.88 for screening MHE in those without OHE.A multicenter study reported that the EncephalApp had a sensitivity of 80% based on the PHES and sensitivity of 70% based on the ICT for screening MHE when using adjusted United States population norms[12].The PHES and EncephalApp,whether alone or in combination,were found to be equivalent for screening MHE,which indicates that single testing with the EncephalApp is sufficient for screening MHE in clinical practice[40].The EncephalApp is a convenient,simple,and highly sensitive test for screening MHE,and application of the EncephalApp may increase the testing likelihood for MHE and subsequent therapy rates.

Previous research found that hepatitis C could induce cognitive impairment;however,the two above-mentioned studies found no differences in EncephalApp results in cirrhotic patients with hepatitis C compared to other etiologies of cirrhosis[14,41,42].The reason for this finding may be that cognitive impairment due to hepatitis C is overshadowed by the impairment caused by liver cirrhosis,which expands the range of EncephalApp application and increases the rates of subsequent therapy for cirrhosis resulting from viral hepatitis.These studies have also established an instrumental comparison between the iPad version and iPod/iPhone version.Compared with the iPad version,the iPod/iPhone version was considered to be an easier approach;however,there was no significant distinction in EncephalApp resultsregardless of the device differences and familiarity with a smartphone,which might be attributed to the practice runs before each tests[13].

Figure 3 Ideograph of EncephalApp Stroop App.A,B:The subject is instructed to choose the color corresponding to the color of the presented symbols(“off” runs);C,D:The subject is instructed to choose the color corresponding to the color of the presented words (“on” runs).

Although the effect of alcohol abuse on cognitive function was found to be confined to psychomotor speed,patients with alcoholic cirrhosis showed more serious cognitive impairment in the EncephalApp results despite similar severity of cirrhosis[14,43].It was recognized that cirrhotic patients with red-green color blindness were excluded from studies because that color discrimination is indispensable for the EncephalApp test.As far as the present practices are concerned,the EncephalApp is not appropriate for screening MHE in cirrhotic patients with red-green color blindness and alcoholic cirrhotic patients without abstinence.As previously mentioned,patients with alcoholic cirrhosis should be tested for MHE using the ICT.In the studies we reviewed,patients receiving treatment with psychoactive medications and/or patients diagnosed with common comorbidities were excluded[12-14],which resulted in limiting generalizability and implementation of the EncephalApp for screening MHE.Therefore,it requires further study by increasing the number of clinically representative patients with alcoholic liver cirrhosis,psychotropic medications,and extrahepatic comorbidities.

Age was found to be a significant influence factor of EncephalApp results[12-14].Although current studies excluded cirrhotic patients with Alzheimer’s disease,Parkinson’s disease,and cerebrovascular disease,these age-related cognitive deficits may influence the reliability of the EncephalApp in the elderly population.Therefore,age-adjusted norms and cut-offs should be developed to reduce the related bias.Although education levels were found to be independent of EncephalApp results,cirrhotic patients included in studies conducted by Bajajet al[13,14]had higher levels of education.On the contrary,epidemiological surveys in different countries showed that a fairly large number of cirrhotic patients had a lower educational background[3-5].The relatively high educational levels of patients in the current studies may affect the potential influence of education on these results and limit the generalizability of the EncephalApp.Additionally,the EncephalApp has been translated into several languages,including English,German,and Chinese;thus,linguistic differences and ethnic origin may have certain effects that should be considered[44].Therefore,further validation of the use of the EncephalApp for screening MHE should control for possible influence factors,including educational background,age,and cultural differences.

Follow-up for MHE

Current studies have demonstrated that the EncephalApp has stable test-retest reliability and external validity for screening MHE,indicating that the EncephalApp is potentially appropriate for repeated tests during follow-up[12-14].A multivariable analysis showed that EncephalApp abnormalities and the MELD score were significant predictors of the initial episode of OHE in cirrhotic patients in a 10-mo follow-up,independent of age and alcoholic etiology[12].The PHES or EncephalApp alone,or a combination of the PHES and EncephalApp are all acceptable strategies to predict initial OHE development[40].Therefore,whether based on reference standards or based on population norms,EncephalApp results independently predict the development of an OHE episode.

TIPS and hyponatremia are considered to be the common etiology of MHE,and studies have shown that EncephalApp results worsened after TIPS placement and improved after hyponatremia correction[13,14].These results showed that the EncephalApp can evaluate cognitive impairment in decompensated cirrhosis and extend the realm of its performance across the range of cognitive deficits in liver cirrhosis.The EncephalApp specifically was associated with basic domains involved in psychomotor speed,visuomotor coordination,and reaction time.Similar to the ICT,EncephalApp results were associated with illegal turns and crashes in driving simulation tests,which highlights its face validity[13].

CRITICAL FLICKER FREQUENCY

Origin and operation

The CFF test is a neurophysiological assessment method that evaluates the ability of the cerebral cortex to identify flickering lights.It was originally designed to examine optic nerve lesions in ophthalmology clinics[45].In the CFF assessment that utilizes the portable Hepatonorm Analyzer (R&R Medical Business Freiburg GmbH,Freiburg,Germany),light pulses decreasing in frequency from 60 to 25 Hz are presented to a subject,and the subject has to press a button as soon as the impression of fused light shifts to that of flickering light.After a training session,flicker frequencies are measured eight to ten times,and the mean value of these runs is calculated to determine the result of the CFF test.The test takes about ten minutes to complete.CFF is well tolerated and easily understood by cirrhotic patients,and it does not require the supervision of specialized personnel.However,CFF assessment requires good eyesight,the absence of red-green blindness,and specialized equipment.Moreover,CFF results are mainly influenced by the luminance of the stimuli,the visual angle,and the distance between eyes and the light source during assessment[16].

Diagnostic efficiency for MHE

CFF was recommended for screening MHE in 2002,and it has been performed in Germany,Spain,Turkey,Egypt,and India[15,17,46-49](Table 1).Using a cut-off of 39 Hz,CFF showed a sensitivity of 55% and a specificity of 100% for screening MHE in German cirrhotic patients[46].Similarly,in Indian cirrhotic patients,a CFF threshold of 38 Hz identified MHE patients with a sensitivity of 72.4% and a specificity of 77.2%using the PHES as the reference standard[15].Moreover,in comparison with PHES,CFF also had a sensitivity of 91.1%,specificity of 92.7%,and area under the curve of 0.937 for screening MHE in Egypt[17].A meta-analysis of nine studies indicated that CFF has a high summary specificity (79%) and moderate sensitivity (61%) for screening MHE[50].These results showed that CFF is a promising method for screening MHE,and its results are independent of educational background,verbal fluency,language,and numeracy[15,17,49].

Although there was a significant correlation between CFF and PHES test results,CFF had a lower sensitivity of 39% for detecting MHE in Turkish cirrhotic patients with a cut-off value of <39 Hz[48].Similarly,Kircheiset al[51]reported that the diagnostic agreement value between CFF and conventional PHES in MHE patients was only 54%.The explanation for this discrepancy of CFF sensitivity is that CFF has advantages in the evaluation of attention abnormalities and is not specialized in assessing motor function,suggesting that CFF could only detect certain features of MHE.Furthermore,several studies have found that CFF values decreased by 0.6–0.7 Hz during every decade of life,and that patients with alcoholic cirrhosis had significantly lower CFF than those with liver cirrhosis due to other etiologies[23,46,47].Therefore,a fixed cut-off may induce biased CFF results in elderly patients and in patients with alcoholic cirrhosis,and age-matched cut-off values should accordingly be established and applied.All things considered,CFF is a simple and easy-to-perform test for screening MHE in cirrhotic patients,which nevertheless should be used as an adjunct to,and not a replacement of,conventional PHES.

Follow-up for MHE

CFF was found to have no learning effects,which indicates that it is appropriate for follow-up studies[27].The study by Sharmaet al[49]showed that CFF is a reliable and accurate test not only for MHE screening but also for the assessment of recovery of MHE patients treated with lactulose for 1 mo.In Spain,a prospective study of 117 consecutive cirrhotic patients showed that CFF predicted the first appearance of OHE during the 5-year follow-up period[52].Moreover,CFF <39 Hz,not the PHES,wasassociated with lower survival,irrespective of MELD scores[52].Similarly,there was a correlation between CFF results and CTP scores but not with MELD scores,and Cox regression showed that CFF together with CTP scores was independently associated with a risk for the development of OHE[15].TIPS is a common etiology of MHE,and it can increase the incidence of OHE.Compared to the PHES,CFF has a good negative predictive value (91%) for the risk of recurrent OHE after TIPS,which suggests that CFF may help select cirrhotic patients for TIPS and decrease the incidence of OHE after TIPS[53].These results indicated that CFF can predict the survival of and the occurrence of OHE in cirrhotic patients with MHE,which is crucial for early prevention and subsequent treatment of MHE.

OTHER MODALITIES

StroopEffect-2 and electronic number connection test

The Stroop test has been validated for screening MHE in adults.In the study by Sureshet al[54],the Tamil language version of the computerized Stroop test,called StroopEffect-2,had a sensitivity of 89.6% and specificity of 100% for diagnosing MHE in Indian children with extrahepatic portal vein obstruction with a cut-off of >180.4 s and an AUROC of 0.97.This study indicated that the computerized Stroop test is suitable for MHE screening in children.Moreover,Wuenschet al[55],who was inspired by the paper-pencil number connection test version A,developed a novel electronic number connection test.The electronic number connection test exhibited high sensitivity (>82%) and high specificity (>85%) in MHE detection with excellent test–retest reliability.

Cognitive drug research assessment battery

The cognitive drug research (CDR) assessment battery is a computerized cognitive testing tool developed for the assessment of human cognitive function impairment.The stimuli of the task are displayed on a laptop,and both accuracy and reaction time of participants’ responses are recorded with the “yes” and “no” buttons on a twobutton response box[56].Mardiniet al[57]reported that the CDR assessment battery has a sensitivity of 86.4% and a specificity of 81.0% for screening MHE in cirrhotic patients compared with the PHES as the reference standard.Furthermore,the CDR system has shown appreciable test-retest reliability,which may help evaluate cognitive improvement when cirrhotic patients with MHE are retested after treatment[56,58].However,the CDR assessment battery requires a practice session before administration,which induces the learning effect that may influence the reliability of the battery for screening MHE.

Central nervous system vital signs

The central nervous system vital signs (CNSVS) is a computerized neurocognitive test battery that was developed into a routine clinical screening instrument.It includes seven tests:verbal and visual memory,finger tapping,symbol digit coding,the Stroop test,a test of shifting attention,and the continuous performance test[59].The CNSVS is able to evaluate verbal memory,processing speed,executive function,reaction time,cognitive flexibility,and complex attention.Prakashet al[60]prospectively enrolled 100 cirrhotic patients and 110 controls,and a high correlation between the CNSVS and PHES was observed.In this study,the CNSVS identified MHE with 85% sensitivity,64% specificity,and AUROC value of 0.74[60].It seems promising,but it still requires validation with larger populations in multicenter studies.

Immediate postconcussion assessment and cognitive test

The immediate postconcussion assessment and cognitive test (ImPACT) is a computerbased program for assessing neurocognitive function and concussion symptoms.The ImPACT executes measurement of attention span,working memory,processing speed,and reaction time and yields four composite scores:verbal memory,visual memory,visual motor speed,and reaction time[61].Tsushimaet al[62]found the correlation between the ImPACT scores and results of paper-and-pencil testing and health-related quality of life.In comparison with paper-and-pencil testing,the ImPACT showed 62.5% sensitivity and 94.7% specificity for screening MHE[62].The ImPACT offers web-based neuropsychological evaluation of MHE in a brief,userfriendly,and effective approach.However,it has not been validated in multicenter studies and is innovated for subjects of college enrollment;thus,the usage of which formost patients with liver cirrhosis is not suitable.

CONCLUSION

MHE lacks a universal and standard method for diagnosis.Novel computerized psychometric tests are rapid,effective,and convenient methods for screening MHE in clinical practice and for identifying high-risk cirrhotic patients for further validation by the PHES test.Computerized psychometric tests may help facilitate early diagnosis and treatment of MHE,which may improve patients’ quality of life,reduce the risk of progression to OHE,and consequently reduce the risk of motor vehicular accidents.However,diagnostic accuracy of computerized psychometric tests is influenced by educational background,age,and cultural differences.Therefore,calibration and validation of these tests are required to reach more convincing conclusions before their widespread clinical application in cirrhotic patients at risk for MHE.