- Title
- Development of novel Bayesian inference algorithms for decision trees with application to clinical decision support tools
- Creator
- Cochrane, Jodie A.
- Relation
- University of Newcastle Research Higher Degree Thesis
- Resource Type
- thesis
- Date
- 2024
- Description
- Research Doctorate - Doctor of Philosophy (PhD)
- Description
- Prediction modelling is ubiquitous in the clinical decision-making literature, where the complexity of the relationship between patient parameters and outcomes often requires a data-driven approach. Machine learning is a recently popularised approach that enables the combination of data and an appropriate model to generate these predictions. One such model that has found widespread application within both the machine learning and clinical communities due to its flexibility and interpretability is the decision tree, which is the focus of this thesis. Uncertainty appears in many aspects of prediction modelling, from the data to the model and, subsequently, the predictions themselves. It is important to consider this to avoid overconfidence in predictions, which can adversely affect future decisions. Quantifying prediction uncertainty can be addressed in a mathematically coherent way by using Bayesian inference. This approach generates a posterior predictive distribution, which can be used to provide estimates for the prediction uncertainty. However, Bayesian inference of the decision tree model is challenging due to the nonlinear model definition and variable dimension parameter space. As such, approximation methods are required for this type of model, with existing attempts using sample-based approximate inference. Markov chain Monte Carlo (MCMC) is a commonly used sample-based method that uses a Markov chain to generate samples from the desired posterior distribution. The efficiency of this method is dependent on the correlation between samples, where high correlation can hinder convergence. Current MCMC methods for decision trees are based on local, random-walk characteristics. This results in high sample correlation, which negatively affects the exploration of the posterior distribution, making the approach computationally inefficient. The Hamiltonian Monte Carlo (HMC) algorithm addresses this issue by using the posterior distribution to propose distant samples, reducing correlation and improving exploration. This thesis investigates ways to incorporate HMC into the sampling scheme to generate samples from the posterior distribution of decision trees. Two novel HMC-based methods for Bayesian inference of decision trees are proposed in this thesis. In each case, HMC is used to conduct local inference within each tree structure, with two frameworks considered for global exploration of the overall space. The first approach uses a random-walk-based MCMC proposal scheme, which is similar to existing methods and is entitled the RJHMC-Tree algorithm. The second approach, DCC-Tree, considers using a utility function similar to that recently proposed in the Divide, Conquer, Combine (DCC) sampling strategy for probabilistic programs [145]. In general, HMC-based methods show better testing performance compared to existing methods. The RJHMC-Tree algorithm demonstrates significantly higher acceptance rates than existing methods. The DCC-Tree algorithm improves further on this method by removing the observed inefficiencies associated with moving between different tree structures. The two proposed Bayesian decision tree methods are applied to a medical dataset to predict patient outcomes after total knee arthroplasty. Outcomes and associated posterior distributions are predicted for post-surgery physical quality of life and overall knee symptomology. Results show the proposed algorithms have better testing performance and find trees of lower root mean squared error than those learnt by the standard decision tree algorithm. The predictive distributions show that the true output is almost always within the support of the posterior, emphasising the importance of uncertainty quantification in clinical prediction modelling.
- Subject
- Bayesian inference algorithms; clinical prediction modelling; decision support tools; decision trees
- Identifier
- http://hdl.handle.net/1959.13/1511460
- Identifier
- uon:56500
- Rights
- Copyright 2024 Jodie A. Cochrane
- Language
- eng
- Full Text
- Hits: 1066
- Visitors: 1104
- Downloads: 44
| Thumbnail | File | Description | Size | Format | |||
|---|---|---|---|---|---|---|---|
| View Details Download | ATTACHMENT01 | Thesis | 4 MB | Adobe Acrobat PDF | View Details Download | ||
| View Details Download | ATTACHMENT02 | Abstract | 193 KB | Adobe Acrobat PDF | View Details Download |