Putting Data at the Heart of Energy Trading

In a digital world, data is King, and while energy and commodities have been going through a period of digitalizing for the last several years, there can be no doubt that coming up with data management strategies has also played an important, if not critical, role in those strategies as well. Indeed, this was strongly emphasized in Commodity Technology Advisory’s disruptive technologies research over the last few years, where more than half of all of those questioned in the industry were planning on further investment in data management and data mining. This was also where other new technologies like AI, for example, were thought likely to be applied for data mining purposes along with the use of enhanced visualization techniques.

Since that report was issued, much has already shifted due to industry events and drivers. Increased price volatility, particularly in energy, because of the Russia-Ukraine war and the current stage in the ongoing energy transition, along with digitalization and an increase in speed and data volumes around automated trading, has only re-affirmed many of its findings. Supreme among those would be the need for reliable and up to the minute data to support all trading and associated activities. Despite that, many in the industry still struggle with data – either in terms of being able to extract and utilize it from systems like ETRM or CTRM or, being able to access it in the right time frame and at the right quality.

Modern technologies have also meant that more data is available and collectible than ever before. All forms of data, both structured and unstructured, can be usefully used in trading and risk management if it is of sufficiently good quality and provided in a timely manner. A lot of effort is spent identifying, collecting, and processing all types of data from a variety of sources by most trading or commodity-related firms. While some data like weather and pricing, for example, can be provided by data aggregating services, other data sources are web content often obtained via screen scraping, documents, images and so on, that need proprietary methods to obtain, cleanse, and utilize them. This is why there has been, and continues to be, significant investment in data and data management, whether that is in the form of data lakes or sophisticated data management applications.