BoE Survey: Machine Learning Affected by COVID
Posted by Colin Lambert. Last updated: February 25, 2021
The Bank of England this week published results of a survey of UK banks intended to ascertain the impact of the COVID-19 pandemic on machine learning (ML) and data science (DS).
The survey found that the pandemic is likely to increase the importance of ML and DS, especially in helping deal with a dispersed workforce, however there were also signs that their benefits were diluted by the unprecedented nature of the global event.
Half of the banks surveyed reported an increase in the importance of ML and DS as a result of the pandemic and none of those surveyed believe that Covid will reduce the importance of ML and DS for them. Around 35% of banks reported a ‘positive’ impact from Covid on the ML and DS technologies that support remote working among employees. That said, only a third said there was an increase in the number of planned or existing ML or DS projects.
The survey also found, however, that existing risks may be amplified or new risks may emerge from the use of ML and DS in financial services. For example, ML models may perform poorly when applied to a situation they have not encountered before in the training data. This is particularly relevant in the context of the Covid pandemic when the underlying data may have changed or the statistical properties of the data may have changed.
Around 35% of banks reported a negative impact on ML model performance as a result of Covid. This is likely because the pandemic has created a major downturn that could not have been forecasted on the basis of economic data alone or historical predictors.
While few will be surprised that machine learning has grown in importance, anecdotally it has played a major role in helping compliance teams monitor activity during the pandemic, it also has to be noted that the survey also highlights one of the bigger challenges of ML – a lack of precedent. The sense is that people think ML will work perfectly well as long as conditions are within certain parameters and that sufficient data is available to power the models. With market events that are unprecedented, including perhaps those from the pre-electronic trading era where data is less comprehensive, there is clearly evidence that machine learning can be found wanting.
This is an extension in some ways of the flash crash phenomena where computers, faced with a lack of data, merely shut down, thus exacerbating liquidity conditions even further. It also highlights one of the truisms of markets – things work perfectly well…until they don’t.
The Bank of England says it will continue to “monitor developments closely”, along with other regulators like the Financial Conduct Authority, and take necessary steps to support the safe adoption of ML and DS in financial services.