Comparison of neural network and logistic regression for dementia prediction: results from the PREADViSE trial| JOURNAL OF GERONTOLOGY AND GERIATRICS

Abstract

Objective. Two systematic reviews suggest that current parametric predictive models are not recommended for use in population demen- tia diagnostic screening. This study was to compare predictive perfor- mance between logistic regression (conventional method) and neural network (non-conventional method).
Method. Neural network analysis was performed through the R pack- age “Neuralnet” by using the same covariates as the logistic regression model. Results. Results show that neural network had a slightly ap- parently better predictive performance (area under curve (AUC): 0.732 neural network vs. 0.725 logistic regression). Neural network performed similarly as logistic regression. Furthermore, logistic regression con- firmed that the interaction effect among covariates, which elucidated from neural network.
Conclusions. Neural network performed slightly apparently better than logistic regression, and it is able to elucidate complicated relationships among covariates."

INTRODUCTION

Rising prevalence of dementia has become a major public health concern as disability associated with dementia, especially at the late stage, leads to high personal, social and economic costs. Early identification of individuals with high risk for dementia is of great importance for dementia prevention, especially when interventions are developed. To identify these high-risk individuals as early as possible, effective predictive or prognostic models with risk factors become a research priority. So far, many studies have attempted to find a useful prediction model.

Conventional prediction models have been predominantly developed from logistic regression ^1-4 or proportional hazards regression analysis ^5-10. For non- conventional models, the classification tree is the most often used method ^11,12. Alternative approaches, also include non-conventional statistical learning methods such as random forest ¹³ and neural network analyses ¹⁴. Covariates used in the majority of predictive modeling studies include demographic variables, such as age, education, body mass index (BMI), medical comorbidity (e.g., history of cardiovascular disease) or cognitive tests. Recently, studies have incorporated genetic risk factors and imaging data into predictive models ^7,15. However, some have argued that genetic risk factors and neuroimaging variables have not significantly increased discriminative accuracy ¹⁵. Further, these data are often difficult and expensive to obtain ¹⁶, while evidence shows that a third of Alzheimer’s disease (AD) cases worldwide may be due to modifiable risk factors ¹⁷.

Tang et al. conducted an updated systematic review based on their 2010 review to evaluate latest development on methods for prediction of dementia risk ^18,19. The updated systematic review concluded that despite the significant increase in the number of risk modeling studies, the predictive accuracy of these parametric models has not changed to a significant degree after 5 years (range 0.49-0.91 in 2010 review, and 0.49-0.89 in the 2015 review), and none of the methods are recommended for dementia risk prediction in the population setting due to insufficient consideration of sample selection, model diagnostics, and model validation ^18,19. Most of risk models included in the reviews were developed from conventional methods including logistic regression or COX proportional hazard model, and none of risk models was created by using neural network. In this study, we sought to compare the predictive performance between neural network and logistic regression using mainly mental status and self-reported data from the Prevention of Alzheimer’s Disease with Vitamin E and Selenium (PREADViSE) trial with a known AD genetic risk (APOE genotype) and clinical diagnosis of dementia to compare predictive models.

METHODS

STUDY SAMPLE AND DATA SOURCES

The PREADViSE trial was an ancillary study to the Selenium and Vitamin E Cancer Prevention Trial (SELECT) (a large prostate cancer prevention randomized controlled trial (RCT)) ²⁰ and was designed to evaluate the effectiveness of antioxidant supplements vitamin E and selenium in preventing incident AD and other forms of dementia. During the recruiting period (2002 to 2009) PREADViSE enrolled 7,547 non-demented male participants age 62 years and older (60+ if African American) from 128 participating SELECT clinical sites in the US, Canada, and Puerto Rico. The eligibility criteria for participating in PREADViSE included active SELECT enrollment at a participating site, and absence of dementia and other active neurologic conditions that affect cognition such as major psychiatric disorder, including depression.

The SELECT study supplements were discontinued by its Data Safety Monitoring Committee in 2008 following a futility analysis on its primary endpoint of prostate cancer incidence ²¹, and then participants in PREADViSE and SELECT were invited to continue as participants in observational cohort studies, and 4,271 of 7,547 original PREADViSE volunteers consented to continue participation. In order to maximize the consistency and completeness of follow-up, only participants who were screened in both the RCT and exposure phases of PREADViSE are included in the current study (n = 3784). PREADViSE was approved by the University of Kentucky Institutional Review Board (IRB) as well as the IRBs at each SELECT study site. Each participant provided written informed consent.

MENTAL STATUS SCREENING

The Memory Impairment Screen (MIS) ²² was used as the primary screening instrument for memory impairment in both the RCT and observational portions of PREADViSE. The MIS was given annually. If participants failed the MIS (that is, the participant scored 5 or less out of 8 on either the immediate or delayed recall portion of the MIS), a second tier screen was administered. An expanded Consortium to Establish a Registry in AD battery (CERAD-e) ²³ was used during the RCT period and the modified Telephone Interview for Cognitive Status (TICS-m) ²⁴, was used during the observational study. Both the CERAD-e and the TICS-m assess participants’ global cognitive function. Failure on the secondary screen (T score ≤ 35 on CERAD-e battery or total score ≤ 35 on TICS-m) would lead to a recommendation for a clinic visit with their local physician. Records from the clinic visit were reviewed by 3-5 expert clinicians, including two neurologists and at least one neuropsychologist, for a consensus diagnosis. In cases where the neurologists disagreed in their diagnoses, the study PI made the final determination. Annual screenings were completed in May 2014, and a small number of participants were followed for medical records through August 2015.

COVARIATES

APOE genotype was obtained for 3,681 participants (ε2/2: 26 (0.71%); ε2/3: 459 (12.47%); ε2/4: 86 (2.34%); ε3/3: 2,240 (60.85%); ε3/4: 808(21.95%); ε4/4: 62(1.68%)). These genotypes were converted to a dummy indicator for at least one ε4 allele, where the presence of at least one ε4 allele was considered a carrier. For 103 subjects without APOE information, SAS 9.4^® procedure PROC MI was used to impute missing values for the indicator variable based on family history of dementia. Four imputed data sets were generated; participants with two or more positive imputations for APOE ε4 were coded as APOE ε4 positive. APOE ε4 positivity is a major risk factor for AD-type dementia ²⁵. Other data collected included age at baseline, race, BMI, years of education, as well as self-reported indicators of cardiovascular disease (i.e., diabetes, hypertension, and smoking), coronary artery bypass graft (CABG), congestive heart failure, hypertensive medication, and memory change at the baseline. These data were obtained at enrollment and annually thereafter as recognized risk factors for dementia ²⁶. History of significant cognitive or motor impairment due to stroke was an exclusion criterion, thus stroke was not considered in the models.

CASE ASCERTAINMENT

To create a predictive model, we used clinical dementia status (dementia vs. non- dementia) at end of follow-up as the outcome in this study. Dementia cases were identified through two methods. First, as described above, a medical records-based consensus diagnosis was used. Date of diagnosis was assigned as the date of the failed screen. Second, because many participants were reluctant to obtain medical workups for their memory, additional longitudinal measures including the AD8 Dementia Screening Interview ²⁷, self-reported medical history, self-reported diagnosis of dementia, use of memory enhancing prescription drug, and cognitive scores including the MIS, CERAD-e Score, NYU Paragraph Delayed Recall, and TICS-m were used to identify cases. The diagnostic criteria for the second method were AD8 total of ≥ 1 (at any time during follow-up) to indicate functional impairment ²⁷ plus one of the following: a self-reported diagnosis of dementia, use of a memory enhancing prescription drug (donepezil, rivastigmine, galantamine, or memantine), or cognitive score below cutoffs for intact cognition on any test (for example: 1.5 SDs below expected performance based on age and education normative data ²⁸. The date of diagnosis was assigned to the earliest event.

DATA ANALYSIS

Chi-square and t-test statistics were used to examine differences in categorical and continuous variables between dementia groups except for congestive heart disease (Fisher’s exact test was used). Univariate logistic regressions were performed first, and only those variables significantly associated with probability of dementia at univariate analysis were included in the multivariable logistic regression. Covariates included in the initial multivariable logistic regression model were age, education, smoking, APOE-ε4 allele status (any vs. none), history of hypertension, diabetes, coronary artery bypass graft (CABG), antihypertensive medication use, and memory change. In the model, age and education were used as continuous variables, and the rest were binary variables (yes vs. no). Logistic regression with backward elimination method was performed to compare with the neural network. Covariates age, education, APOE-ε4, and self-reported memory change were left in the final logistic regression model without interaction terms. Then neural network elucidated an interaction effect among years of education, age at baseline, status of memory change, and APOE-ε4 allele status.

Logistic regression was conducted again to confirm the interaction effects.

NEURAL NETWORK

As an extension of generalized linear models (GLM), artificial neural network (ANN) was applied to explore the complex relationship between covariates and response ²⁹. Unlike GLM, ANN does not need to specify the form of the relationship between covariates and response variables. In this study, ANN was performed in R package “Neuralnet” under R (version 3.1.2), details about Neuralnet can be found in Gunther’s paper ³⁰. Multilayer perceptron (MLP) ^29,31 is the main model for neural network, which consists of vertices and directed edges called neurons and synapses respectively. Neurons are organized as layers and connected by synapses. Our ANN model had three neuron layers: input, hidden, output (See Figure 1). The input layer included all covariates in separate neurons, and the output layer consisted of the response variable (output). The layers between input and output layers are referred as hidden layers because they are not observed. For each synapse, a weight is assigned to indicate the effect of the corresponding neuron. Based on Fristsch, then “all data will pass through the neural network as signals, and these incoming signals will be first processed by the integration function, and then by activation function to transform as output of the neuron” ²⁹. According to Hornik³², one hidden layer is sufficient to fit any real-valued continuous function ^29,32.

Then supervised learning is applied in which desired output (or given output) is defined and is compared to the predicted output. Weights are also chosen at this stage ^29,33. The starting weights are usually assigned randomly from the standard normal distribution. To fit the neural network, the following steps are repeated ²⁹:

1. neural network calculates an output for given inputs and starting weights;
2. an error function such as sum of squared errors (SSE) or the cross-entropy will be applied to measure the difference between the actual output and predicted output;
3. then all weights are adapted based on the rule of a learning algorithm;
4. the process will stop if the pre-specified criterion (rule of a learning algorithm) is reached, for example, all absolute partial derivatives of the error function with respect to the weights are smaller than a given threshold or a specified maximum step is reached ²⁹. The resilient backpropagation algorithm (rprop+) is the most commonly used learning algorithm ³⁴. Weights are modified by searching in the opposite direction of the partial derivatives until a local minimum is found ²⁹. Additional technical details about ANN can be found in Gunther’s technical report and Quintana’s paper ^29,35.

Our ANN input layer included four covariates including age, education, APOE-ε4, and self-reported memory change, in order to be directly comparable to the logistic regression model. We decided to have 10 hidden units based on the consideration of the sample size. The output layer had one neuron, which was dementia status at end of follow-up. Logistic regression and identity function were used as the activation function and integration function, respectively. The “rprop+” algorithm was used to determine the weights. AUC was calculated to compare the performance between logistic regression and ANN on classification of dementia status.

Descriptive analysis and logistic regression were conducted by using SAS 9.4^® (SAS Institute, Inc., Cary, NC). Statistical significance was set at p = 0.05.

RESULTS

Table I presents the general characteristics of participants in both RCT and central follow up. Of 3,784 subjects, 227 had been diagnosed with dementia at the end of follow-up. Compared to subjects who did not develop dementia, subjects who developed dementia were older at baseline, less educated, were more often smokers, more likely carried the APOE-ε4 allele, used antihypertensive medication, and reported experiencing a memory change at baseline (Tab. I).

Based on preliminary analysis (data not shown), the prediction error in the neural network did not change dramatically as the threshold of the partial derivatives of the error function changed; we chose 0.1 as the threshold. Fig. 1 depicts the neural network structure for the current study and shows the final weights of the corresponding synapses. These weights were used to calculate the estimated probability of incidence of dementia. To interepret the association found in the neural network, the estimated probabilities of having dementia for 36 hypothetical subjects are presented in Table II. The measure of association for having dementia given a certain covariate in the neural network depends on the covariate and other covariates in the model.

Keeping other covariates in the model constant, as age increased, the estimated probability of having dementia is increased (Tab. II). For example, subjects 1, 2, and 3, who represent persons who are without an APOE-ε4 allele, absence of memory change, lower education(13.2 years), are aged at 62.2 years, at 67.2 years, and at 72.2 years, respectively. The estimated probabilities of developing dementia for subjects 1, 2, 3 are 0.029, 0.053, and 0.068, respectively. Similarly, we can conclude that the presence of an APOE-ε4 allele is associated with increased estimated probability of having dementia (subject 3 vs subject 12 in Table II).

As illustrated in Table 2, the effect of education on risk of dementia depended on age, APOE-ε4 allele status, and perception of memory change. Higher education was associated with lower risk of dementia only in younger subjects (62.2 years) with the exception of younger subjects with APOE-ε4 and self-reported memory change. For example, in the younger age group, the estimated probability of having dementia for subject 7 (= 0.003) with higher education (17.8 years) is much lower than subject 1 (= 0.029) with lower education (13.2 years). Similar comparisons can be made for subjects 10 (= 0.054) and 16 (= 0.002), but not the comparison between subject 28 (= 0.070) and subject 34 (= 0.098), who represent subjects with presence of both APOE-ε4 allele and memory change. Education did not show a protective effect on risk of dementia in older subjects (72.2 years), especially for older subjects with presence of both APOE-ε4 allele and memory change. No matter the education level, older subjects who were APOE ε4 carriers and had memory changes had the highest risk of dementia, such as subjects 30 (= 0.354), 33 (= 0.364), and 36 (= 0.372). In contrast, well-educated younger subjects who did not have either APOE-ε4 or memory changes had the lowest risk of dementia, such as subject 7 (= 0.003), subject 16 (= 0.002), and subject 25 (= 0.001).

According to the results in neural network in which the effect of education was modified by age, status of APOE-ε4, and status of memory change, logistic regression were performed to confirm the interaction effects. Table III shows parameter estimates and p values for one 3-way interaction regression model from logistic regression final model, and the four-way interaction term and other three-way interaction terms were removed from final model due to the insignificance. The 3-way interaction among education in years, APOE-ε4 allele carrier, and self- reported memory changes is significant (p = 0.01). To demonstrate the effect modification identified in the logistic regression model, same 36 hypothetical subjects are presented in table 4. Subjects with presence of both APOE-ε4 allele and memory change had highest estimated probability of having dementia, such as subject 30 (p̃ = 0.664), and subject 33 (p̃ = 0.594). Age modified the effect of education; however, comparing subjects 28, 29, 30, or subje cts 19, 20, 21 from Table IV, the interaction effect among age, education, APOE-ε4 and memory change is not significant.

Comparison of overall performance between logistic regression and ANN for predicting incident dementia was recorded in Table V. For calculating sensitivity, etc., a model-estimated probability greater than 0.05 was regarded as a positive classification. ANN had similar predictive accuracy as logistic regression (AUC in neural network = 0.732 vs AUC in logistic regression 0.725). Overall, neural network has similar performance in sensitivity (83.2%) and negative predictive value (98.0%) than sensitivity (83.3%) and negative predictive value (98.0%) in logistic regression, but worse in the positive predictive value (9.9% in neural network vs 9.8% in logistic regression).

DISCUSSION

The purpose of this study was to compare predictive accuracy for incident dementia between neural network and logistic regression in the PREADViSE trial. Neural network analysis showed slightly apparently improved predictive accuracy (AUC = 0.732) compared to logistic regression (AUC = 0.725). The model obtained from the neural network had similar performance as the logistic regression model. Similar association between covariates and the dementia outcome were found in neural network and logistic regression, but the model in neural network is more difficult to interpret. Furthermore, neural network may easily elucidate more complex relationships between model variables, such as education, age, APOE, and self-reported memory change, and the outcome. While higher education is usually considered universally protective against dementia ^36,37, the effect of education on dementia in the neural network depended on age, APOE-ε4 allele status, and self-reported memory change.

Stephan et al. ¹⁸ evaluated predictive accuracy of dementia prediction models and found that poor predictive accuracy is associated with single-factor models, long follow-up intervals, and all-cause dementia for outcome ascertainment, which assumes all dementias share risk factors. The Canadian Study of Health and Aging (CSHA) ³⁸ showed lower predictive accuracy (AUC = 0.77 in 10-year follow-up than 5–year follow-up (AUC = 0.83). The predictive accuracy (AUC = 0.732) in our neural network model is slightly lower than the CSHA 10-year study, but is comparable to the Gothenburg H-70 1901-02 birth cohort for 20-years of follow-up (AUC = 0.74) ³⁹. In contrast to the CSHA study, Exalto et al did not find any significantly different results on predictive accuracy in two Cardiovascular Risk Factors, Aging, and Incidence of Dementia (CAIDE) studies based on follow-up time (one is 10 years follow-up and another one is 36 years) ^40,41. Follow-up time in the current study was over 10 years.

Based on covariates used to generate the predictive model, models generated in the previous papers can be summarized into the following categories: 1) demographic only models; 2) cognitive test based models with or without demographic data; 3) comorbidity data models; 4) genetic and biomarker models; and 5) models including demographics, comorbidities, genetics, and biomarkers. Our logistic regression model included age, education, APOE, and memory changes to predict incident dementia and had moderate predictive accuracy (AUC: 0.725, sensitivity 83.3% and specificity 51.3%). A similar study ⁴⁰ in which the model was derived from demographic variables, health risk factors and APOE, obtained slightly better diagnostic accuracy (AUC = 0.78).This study also argued that diagnostic accuracy did not change significantly after removing APOE from the model (AUC: 0.77; sensitivity 77% and specificity 63%). Other models include neuroimaging information and/or neuropsychological tests. Tang et al. argued in their review that genetic information and/or imaging data do not improve diagnostic accuracy significantly ^15,19. Furthermore, predictive models using one or multiple neuropsychological tests as covariates seem to have higher predictive accuracy, but there is not direct comparison for these two approaches due to between-study variation, such as different criteria for outcome measurement ¹⁸. Waite and colleagues argued that refining the subgroups of dementia types may improve diagnostic accuracy, but is unlikely to be cost effective because defining these subgroups of dementia can be expensive ⁴².

On the other hand, from machine learning and classical statistical methods, neural network in several studies has demonstrated superior ability to capture complex relationships in data compared to classical statistical methods ^43,44. Also, neural networks obtained higher predictive accuracy rate than linear discriminant analysis and successfully distinguished Alzheimer’s patients from control aged 80 years and older in the Nun study using neurofibrillary tangles and neurotic plaques counts (AUC was not reported) ⁴⁵. In contrast, Maroco et al. ⁴⁶ suggested that random forest and linear discriminant analysis performed better than other statistical methods, such as neural network, support vector machines, and logistic regression based on the consideration of predictive accuracy, sensitivity, specificity. They also argued that neural networks and logistic regression are inappropriate for unbalanced data, which means small frequency vs. large frequency group in response variable. Furthermore, Song et al. ⁴⁷ compared the machine learning methods with classic statistical methods for two biomedical datasets: one was from patient care records and another was from a population survey, and they did not find big differences in prediction between the two datasets, which indicates that the quality of the data, such as accuracy and completeness may be more important than leaning technique.

Strengths for this study include larger sample size and long follow-up. We were also able to consider most well-established risk factors for dementia, including demographic, genetic, and medical characteristics. This study also had some limitations. Our outcome diagnosis was based on two criteria due to lack of medical records from many participants, our case ascertainment may be less accurate. However, misclassification of diagnosis is independent of exposure measurement, so non-differential misclassification is unlikely. Thus, results are likely biased toward the null.

CONCLUSIONS

Neural network did not significantly improve predict accuracy over logistic regression and also increased difficulty of interpretation of the association between the outcome and covariates. The most important aspect to improve performance of a model, does not depend on statistical methods, or computational techniques, but depends on how much accurate information the dataset contains. In the future, similar studies should focus on refining the definition of outcome diagnosis, improving quality of assessments, and performing validation after generating a risk model. Neural network may not improve prediction, but it has potential to identify interaction effects.

Figures and tables

Figure 1.Neural network model for incidence of dementia in PREADViSE trial. Age, APOE- ε4, Education, Memory Change represent the 4 input neurons on the left side of the diagram. Each input neuron was connected with 10 hidden neurons (second column of empty circles from the left of figure) by 10 corresponding synaptic weights. The 10 hidden units and the output neuron – dementia were connected by the synapses starting from the hidden units and ending at the output layer. The first “1” in the circle from the left of the figure represents the intercepts of each hidden neuron, and the second “1” in the circle stands for the intercept of output neuron. These weights and intercepts were adapted to calculate the estimated probability of dementia. The model was stopped after 203,313 steps, and prediction error is 99.02907.

References

1. Chary E, Amieva H, Pérès K. Short-versus long-term prediction of dementia among subjects with low and high educational levels. Alzheimers Dement. 2013; 9:562-71. DOI
2. Wolfsgruber S, Jessen F, Wiese B. The CERAD neuropsychological assessment battery total score detects and predicts Alzheimer disease dementia with high diagnostic accuracy. Am J Geriatr Psychiatry. 2014; 22:1017-28. DOI
3. Song X, Mitnitski A, Rockwood K. Nontraditional risk factors combine to predict Alzheimer disease and dementia. Neurology. 2011; 77:227-34. DOI
4. Tierney MC, Moineddin R, McDowell I. Prediction of all-cause dementia using neuropsychological tests within 10 and 5 years of diagnosis in a community-based sample. J Alzheimers Dis. 2010; 22:1231-40. DOI
5. Grober E, Sanders AE, Hall C. Free and cued selective reminding identifies very mild dementia in primary care. Alzheimer Dis Assoc Disord. 2010; 24:284-90. DOI
6. Barnes DE, Beiser AS, Lee A. Development and validation of a brief dementia screening indicator for primary care. Alzheimers Dement. 2014; 10:656-65.e1. DOI
7. Verhaaren BF, Vernooij MW, Koudstaal PJ. Alzheimer’s disease genes and cognition in the nondemented general population. Biol Psychiatry. 2013; 73:429-34. DOI
8. Exalto LG, Biessels GJ, Karter AJ. Risk score for prediction of 10 year dementia risk in individuals with type 2 diabetes: a cohort study. Lancet Diabetes Endocrinol. 2013; 1:183-90. DOI
9. Derby CA, Burns LC, Wang C. Screening for predementia AD: time-dependent operating characteristics of episodic memory tests. Neurology. 2013; 80:1307-14. DOI
10. Jessen F, Wiese B, Bickel H. Prediction of dementia in primary care patients. PLoS One. 2011; 6:e16852. DOI
11. Weakley A, Williams JA, Schmitter-Edgecombe M. Neuropsychological test selection for cognitive impairment classification: a machine learning approach. J Clin Exp Neuropsychol. 2015; 37:899-916. DOI
12. Haghighi M, Smith A, Morgan D. Identifying cost-effective predictive rules of amyloid-beta level by integrating neuropsychological tests and plasma-based markers. J Alzheimers Dis. 2015; 43:1261-70. DOI
13. Mattsson N, Insel PS, Donohue M. Predicting reduction of cerebrospinal fluid beta-amyloid 42 in cognitively healthy controls. JAMA Neurol. 2015; 72:554-60. DOI
14. Li L, Huang J, Sun S. Selecting pre-screening items for early intervention trials of dementia – a case study. Stat Med. 2004; 23:271-83. DOI
15. Seshadri S, Fitzpatrick AL, Ikram MA. Genome-wide analysis of genetic loci associated with Alzheimer disease. Jama. 2010; 303:1832-40. DOI
16. Stephan BC, Tzourio C, Auriacombe S. Usefulness of data from magnetic resonance imaging to improve prediction of dementia: population based cohort study. Bmj. 2015; 350:h2863. DOI
17. Norton S, Matthews FE, Barnes DE. Potential for primary prevention of Alzheimer’s disease: an analysis of population-based data. Lancet Neurol. 2014; 13:788-94. DOI
18. Stephan BC, Kurth T, Matthews FE. Dementia risk prediction in the population: are screening models accurate?. Nat Rev Neurol. 2010; 6:318-26. DOI
19. Tang EY, Harrison SL, Errington L. Current developments in dementia risk prediction modelling: an updated systematic review. PLoS One. 2015; 10:e0136181. DOI
20. Lippman SM, Klein EA, Goodman PJ. Effect of selenium and vitamin E on risk of prostate cancer and other cancers: the Selenium and Vitamin E Cancer Prevention Trial (SELECT). Jama. 2009; 301:39-51. DOI
21. Goodman PJ, Hartline JA, Tangen CM. Moving a randomized clinical trial into an observational cohort. Clin Trials. 2013; 10:131-42. DOI
22. Buschke H, Kuslansky G, Katz M. Screening for dementia with the memory impairment screen. Neurology. 1999; 52:231-8. DOI
23. Morris JC, Heyman A, Mohs RC. The Consortium to Establish a Registry for Alzheimer’s Disease (CERAD). Part I. Clinical and neuropsychological assessment of Alzheimer’s disease. Neurology. 1989; 39:1159-65. DOI
24. de Jager CA, Budge MM, Clarke R. Utility of TICS-M for the assessment of cognitive function in older adults. Int J Geriatr Psychiatry. 2003; 18:318-24. DOI
25. Lopez OL. The growing burden of Alzheimer’s disease. Am J Manag Care. 2011; 17:S339-45.
26. Barnes DE, Yaffe K. The projected effect of risk factor reduction on Alzheimer’s disease prevalence. Lancet Neurol. 2011; 10:819-28. DOI
27. Galvin JE, Roe CM, Powlishta KK. The AD8: a brief informant interview to detect dementia. Neurology. 2005; 65:559-64. DOI
28. Petersen RC. Mild cognitive impairment as a diagnostic entity. J Intern Med. 2004; 256:183-94. DOI
29. Fristsch FGS. Neuralnet: training of neural networks. The R Journal. 2010;2.
30. Stefan Fritsch FG. Package’neuralnet’. CRAN. 2012.
31. Bishop CM. Neural networks for patter recognition. Clarendon press: Oxford; 1995.
32. Hornik K. Multiplayer feedforward networks are universal approximators. 1989; 2:7.
33. Rojas RU. Neural networks – a systematic introduction. Springer-Verlag: Berlin; 1996.
34. Riedmiller M. Rprop – description and implementation. Details Technical Rep. 1994.
35. Quintana M, Guàrdia J, Sánchez-Benavides G. Using artificial neural networks in clinical neuropsychology: high performance in mild cognitive impairment and Alzheimer’s disease. J Clin Exp Neuropsychol. 2012; 34:195-208. DOI
36. EClipSE Collaborative Members, Brayne C, Ince PG. Education, the brain and dementia: neuroprotection or compensation?. Brain. 2010; 133:2210-6. DOI
37. Wilson RS, Hebert LE, Scherr PA. Educational attainment and cognitive decline in old age. Neurology. 2009; 72:460-5. DOI
38. Tierney MC, Yao C, Kiss A. Neuropsychological tests accurately predict incident Alzheimer disease after 5 and 10 years. Neurology. 2005; 64:1853-9. DOI
39. Mitnitski A, Skoog I, Song X. A vascular risk factor index in relation to mortality and incident dementia. Eur J Neurol. 2006; 13:514-21. DOI
40. Kivipelto M, Ngandu T, Laatikainen T. Risk score for the prediction of dementia risk in 20 years among middle aged people: a longitudinal, population-based study. Lancet Neurol. 2006; 5:735-41. DOI
41. Exalto LG, Quesenberry CP, Barnes D. Midlife risk score for the prediction of dementia four decades later. Alzheimers Dement. 2014; 10:562-70. DOI
42. Waite LM, Broe GA, Grayson DA. Preclinical syndromes predict dementia: the Sydney older persons study. J Neurol Neurosurg Psychiatry. 2001; 71:296-302. DOI
43. Parsons TD, Rizzo AA, Buckwalter JG. Backpropagation and regression: comparative utility for neuropsychologists. J Clin Exp Neuropsychol. 2004; 26:95-104. DOI
44. French BM, Dawson MR, Dobbs AR. Classification and staging of dementia of the Alzheimer type: a comparison between neural networks and linear discriminant analysis. Arch Neurol. 1997; 54:1001-9.
45. Grossi E, Buscema MP, Snowdon D. Neuropathological findings processed by artificial neural networks (ANNs) can perfectly distinguish Alzheimer’s patients from controls in the Nun. Study BMC Neurol. 2007; 7:15. DOI
46. Maroco J, Silva D, Rodrigues A. Data mining methods in the prediction of dementia: a real-data comparison of the accuracy, sensitivity and specificity of linear discriminant analysis, logistic regression, neural networks, support vector machines, classification trees and random forests. BMC Res Notes. 2011; 4:299. DOI
47. Song X, Mitnitski A, Cox J. Comparison of machine learning techniques with classical statistical models in predicting health outcomes. Stud Health Technol Inform. 2004; 107:736-40.