Step Aside ChatGPT: Here is BloombergGPT
Posted by Colin Lambert. Last updated: March 31, 2023
With ChatGPT filling the pages of social media and more, Bloomberg has released a research paper detailing the development of its own AI and machine learning model, BloombergGPT.
The firm says the large language model (LLM) has been specifically trained on a wide range of financial data to support a diverse set of natural language processing (NLP) tasks within the financial industry.
It adds that recent advances in Artificial Intelligence (AI) based on LLMs have already demonstrated new applications for many domains, however, the complexity and unique terminology of the financial domain warrant a domain-specific model. “BloombergGPT represents the first step in the development and application of this new technology for the financial industry,” it states.
The model will assist Bloomberg in improving existing financial NLP tasks, such as sentiment analysis, named entity recognition, news classification, and question answering, among others. The firm says it will also unlock new opportunities for marshalling the vast quantities of data available on the Bloomberg Terminal to better help the firm’s customers, while bringing the full potential of AI to the financial domain.
Bloomberg’s ML product and research group collaborated with its AI engineering team to construct what it says is one of the largest domain-specific datasets yet, drawing on the company’s existing data creation, collection, and curation resources. The team pulled from the firm’s archive of financial data to create a 363 billion token dataset consisting of English financial documents.
This data was augmented with a 345 billion token public dataset to create a large training corpus with over 700 billion tokens. Using a portion of this training corpus, the team trained a 50-billion parameter decoder-only causal language model. The resulting model was validated on existing finance-specific NLP benchmarks, Bloomberg says, as well as on a suite of Bloomberg internal benchmarks, and broad categories of general-purpose NLP tasks from popular benchmarks. Notably, the firm claims, the BloombergGPT model outperforms existing open models of a similar size on financial tasks by large margins, while still performing on par or better on general NLP benchmarks.
“For all the reasons generative LLMs are attractive – few-shot learning, text generation, conversational systems, etc – we see tremendous value in having developed the first LLM focused on the financial domain,” says Shawn Edwards, Bloomberg’s chief technology officer. “BloombergGPT will enable us to tackle many new types of applications, while it delivers much higher performance out-of-the-box than custom models for each application, at a faster time-to-market.”
Gideon Mann, head of Bloomberg’s ML product and research team, adds, “The quality of machine learning and NLP models comes down to the data you put into them. Thanks to the collection of financial documents Bloomberg has curated over four decades, we were able to carefully create a large and clean, domain-specific dataset to train a LLM that is best suited for financial use cases.”