Nicholas Gray

Clinical decision support systems (CDSS) employing data-driven technology such as artificial intelligence, machine- and statistical-learning are increasingly deployed in healthcare settings. These systems often provide clinicians with diagnostic, prognostic, or risk scores modelled from curated patient-level data and frequently involve iterative and non-deterministic optimisation of flexible, parameterised models. All of these data and algorithms have uncertainties associated with them that should be taken into account when used to support clinical decisions at the patient level. This scoping review aims to describe the literature on how deployed data-driven CDSSs present information about uncertainty to their intended users. We describe common clinical applications of CDSSs, characterise the decisions that are being supported, and examine how the CDSS provides outputs to end users, including uncertainty at the individual patient level, as well as indirect measures such as CDSS performance metrics.

Abstract available via OSF preprint.

Algorithms have no idea of the significance of the calculations that they are performing. They just mindlessly output the results of complex mathematical operations, often requiring untenable assumptions to be made, irrespective of the risk posed by even simple errors, frequently in inhumane ways. This thesis argues that careful consideration of the uncertainties within systems and environments can aid ethical human-AI decision-making (especially under epistemic uncertainty) in high-risk scenarios. It is multidisciplinary in nature, spanning AI ethics, engineering, statistics and risk science.

An uncertainty compiler is a tool that automatically translates original computer source code lacking explicit uncertainty quantification into code containing appropriate uncertainty representations and uncertainty propagation algorithms. It handles the specifications of input uncertainties, and inserts calls to intrusive uncertainty quantification algorithms. In theory, one could create an uncertainty compiler for any scientific programming language. The uncertainty compiler can apply intrusive uncertainty propagation methods to codes or parts of codes and, therefore, more comprehensively and flexibly address epistemic and aleatory uncertainties.

Vertical farming (VF) is a method of indoor agricultural production, involving stacked layers of crops, utilising technologies to increase yields per unit area. However, this emerging sector has struggled with profitability and a high failure rate. Practitioners and academics call for a comprehensive economic analysis of vertical farming, but efforts have been stifled by a lack of valid and available data as existing studies are unable to address risks and uncertainty that may support risk-empowered business planning. An adaptable economic analysis is necessary that considers imprecise variables and risks. The financial risk analysis presented uses a first-hitting-time model with probability bounds to evaluate quasi-insolvency for two unique vertical farms. The UK farm results show that capital injection, robust data collection, frequent cleaning, efficient distribution and cheaper packaging are pathways to profitability and have a safer risk profile. For the Japanese farm, diversification of revenue streams like tours or education reduce financial risk associated with yield and sales. This is the first instance of applying risk and uncertainty quantification for VF business models and it can support wider agricultural projects.

Probability bounds analysis (PBA) is a collection of mathematical methods generalising interval analysis and probability theory. PBA can be utilised for uncertainty quantification for both aleatory and epistemic uncertainty across a wide range of scientific fields. PBA is most useful when information about variables is only partially known and can be used without requiring untenable assumptions to be made about parameter values, distribution shapes or dependence between variables. This paper introduces a PBA library for the Python programming language.

Confidence intervals are an established means of portraying uncertainty about an inferred parameter and can be generated through the use of confidence distributions. For a confidence distribution to be valid, it must maintain frequentist coverage of the true parameter. This can be represented for a precise distribution by adherence to a cumulative unit uniform distribution, a comparison that is referred to here as a Singh plot. This article extends this to imprecise confidence procedures with bounds around the uniform distribution, and describes how deviations convey information regarding the characteristics of confidence procedures designed for inference and prediction. An example of how Singh plots can highlight deficiencies in proposed methods is given with the ProUCL Chebyshev upper confidence limit estimator for the mean of an unknown distribution.

An uncertainty compiler is a tool that automatically translates original computer source code lacking explicit uncertainty analysis into code containing appropriate uncertainty representations and uncertainty propagation algorithms. We have developed a prototype uncertainty compiler along with an associated object-oriented uncertainty language in the form of a stand-alone Python library. It handles the specifications of input uncertainties and inserts calls to intrusive uncertainty quantification algorithms in the library. The uncertainty compiler can apply intrusive uncertainty propagation methods to codes or parts of codes and therefore more comprehensively and flexibly address both epistemic and aleatory uncertainties.

Logistic regression is an important statistical tool for assessing the probability of an outcome based upon some predictive variables. Standard methods can only deal with precisely known data, however many datasets have uncertainties which traditional methods either reduce to a single point or completely disregard. In this paper we show that it is possible to include these uncertainties by considering an imprecise logistic regression model using the set of possible models that can be obtained from values from within the intervals. This has the advantage of clearly expressing the epistemic uncertainty removed by traditional methods.

Testing is viewed as a critical aspect of any strategy to tackle epidemics. Much of the dialogue around testing has concentrated on how countries can scale up capacity, but the uncertainty in testing has not received nearly as much attention beyond asking if a test is accurate enough to be used. Even for highly accurate tests, false positives and false negatives will accumulate as mass testing strategies are employed under pressure, and these misdiagnoses could have major implications on the ability of governments to suppress the virus. The present analysis uses a modified SIR model to understand the implication and magnitude of misdiagnosis in the context of ending lockdown measures. The results indicate that increased testing capacity alone will not provide a solution to lockdown measures. The progression of the epidemic and peak infections is shown to depend heavily on test characteristics, test targeting, and prevalence of the infection.