- Title
- On the construction of probabilistic Newton-type algorithms
- Creator
- Wills, Adrian G.; Schön, Thomas B.
- Relation
- 2017 IEEE 56th Annual Conference on Decision and Control (CDC). Proceedings of th 2017 IEEE 56th Annual Conference on Decision and Control, CDC 2017 (Melbourne, Australia 12-15 December, 2017) p. 6499-6504
- Publisher Link
- http://dx.doi.org/10.1109/CDC.2017.8264638
- Publisher
- Institute of Electrical and Electronics Engineers (IEEE)
- Resource Type
- conference paper
- Date
- 2017
- Description
- It has recently been shown that many of the existing quasi-Newton algorithms can be formulated as learning algorithms, capable of learning local models of the cost functions. Importantly, this understanding allows us to safely start assembling probabilistic Newton-type algorithms, applicable in situations where we only have access to noisy observations of the cost function and its derivatives. This is where our interest lies. We make contributions to the use of the non-parametric and probabilistic Gaussian process models in solving these stochastic optimisation problems. Specifically, we present a new algorithm that unites these approximations together with recent probabilistic line search routines to deliver a probabilistic quasi-Newton approach. We also show that the probabilistic optimisation algorithms deliver promising results on challenging nonlinear system identification problems where the very nature of the problem is such that we can only access the cost function and its derivative via noisy observations, since there are no closed-form expressions available.
- Subject
- cost function; probabilistic logic; noise measurement; Gaussian processes; mathematical model; approximation algorithms
- Identifier
- http://hdl.handle.net/1959.13/1448157
- Identifier
- uon:43328
- Identifier
- ISBN:9781509028733
- Language
- eng
- Reviewed
- Hits: 482
- Visitors: 479
- Downloads: 0