Dementia Risk Assessment Models Have Limited Ability in Predicting Disease

Dementia risk assessment models are found to be prone to high error rates, limiting their ability to predict disease. These are the findings of a study published in the Journal of the American Medical Association.

With the projected incidence of dementia estimated to double by 2050, multifactorial risk prediction models have emerged, aiming at preventative care. For the study, researchers investigated the clinical utility of several of these models in estimating 10-year dementia risk.

In the prospective, population-based study, the researchers assessed various dementia risk scores at baseline, and then perceived incident dementia over the following 10 years. To conduct the study, the researchers used 2 cohorts: the UK Biobank Study, which was used to evaluate 10-year dementia risk, and the British Whitehall II study, which analyzed the estimated 20-year dementia projection in a younger cohort. Participants who did not have dementia at baseline, and had complete data on at least 1 of the dementia risk scores that were evaluated, were included in the study.

A total of 4 prediction models, which distinguished those at highest risk for dementia, were used in the study. The models included the Cardiovascular Risk Factors, Aging, and Dementia (CAIDE) clinical score, CAIDE-APOE (apolipoprotein E)-supplemented, the Brief Dementia Screening Indicator (BDSI), and the Australian National University Alzheimer Disease Risk Index (ANU-ADRI).

The primary outcomes of the study were the detection rate, false positive rate, as well as C statistic (a measure of accuracy for predicting 10-year risk for dementia), for each of the models.

From a cohort of 465,929 eligible UK Biobank participants, 3,421 (0.7%) were diagnosed with dementia (7.5 per 10,000 person years).Affiliated C statistics for all-cause dementia were 0.59 (95% CI, 0.58-0.60) for ANU-ADRI and 0.73 (95% CI, 0.72-0.73) for CAIDE–APOE-supplemented. Notably, the highest C statistic was found for vascular dementia, and the lowest for frontotemporal dementia.

Adjusting the test-positive threshold to detect more than 50% of incident dementia found a false-positive rate of 32%, with a true to false positive ratio of 1 to 88 for CAIDE clinical, and a false positive rate of 24% and a true to false positive ratio of 1 to 66 for CAIDE-APOE-supplemented. Similar false-positive rates were noted for BDSI and ANU-ADRI (34% and 43%, respectively).

When the researchers maintained a test-positive threshold that maintained a false-positive rate at or below 5%, incident dementia was found to be missed in approximately 8.4 to 9.1 per 10 participants, with lower error rates when analyzing risk based on age alone.

Compared with the UK Biobank study, the 20-year dementia predictions using the Whitehall II study were not vastly different. However, 1 exception was the improvement of true to false positives ratio, as a larger proportion of participants were diagnosed with incident dementia. Additionally, in order to detect half of future dementia cases using CAIDE, BDSI, and ANU-ADRI scores, it was found that each correct prediction of dementia was associated with 66 to 116 false-positive predictions. Also, in order to achieve a low false-positive rate of ≤5%, all scores missed 84% to 91% of incident dementia.

“Further research is needed to develop better risk prediction algorithms for dementia,” the researchers wrote. “Ideally, risk markers used in algorithms would be surrogate markers responsive to change in risk (unlike age, sex, and APOE genotype), as such markers could inform clinical decisions on individualized preventive strategies, a goal increasingly adopted in modern biomarker-based risk prediction tools for chronic conditions.”

The limitations of the study included a lack of generalizability and over/underestimation of predictive capacity, as participation from the UK Biobank cohort was low (5.5%).