Historically, financial service professionals have used terminal-based products for research and analytics to value companies, manage portfolios, and make data-driven decisions. Today, financial content consumers from data scientists to traders to compliance require flexible, scalable, end-to-end workflow solutions for their data—whether they take the form of bulk feeds, tick history, or real-time streaming data for trading strategies, reporting best execution, or delivering content directly into proprietary applications on demand via API. Implementing the right solution can be filled with options as well as numerous challenges, including the time required to get a customer up and running with the content they require.
In addition, the COVID-19 pandemic accelerated the need for people to take a fresh look at their technological footprint, looking for ways to power systems with next-generation tools, infrastructure, and data delivery. Working from home for an extended period of time also presented new challenges related to accessing and analyzing data; both increased security and flexibility were required to access corporate systems remotely. The growth of cloud-based data sharing promises increased efficiency for data transportation and ingestion. It can also unlock the potential for technological innovations and result in a much faster time to market.
The Challenges with Data Feeds
When remote work became a necessity, organizations needed to act fast in delivering data to their end users. For some organizations, requirements included:
- Working with market data procurement teams to validate and update contracting and licensing
- Provisioning new hardware and upgraded network connections via the internal information technology (IT) team
- Updating firewalls and user authentication within the firm’s information security team
- Creating an ETL (extract, transform, and load) process or installing a vendor feed loader as part of the established data management workflow
Fulfilling these undifferentiated requirements takes valuable time away that could otherwise be used to generate value for the firm. With a myriad of database technologies available, and each with its own proprietary method of loading, many data providers are starting to create a custom ETL process for each technology system that clients need them to support. For example, geographical differences in where data is sourced and consumed can create reporting headaches, even within systems that allow easy and relatively open access between a provider and its consumers. In many instances, replicating data or the ETL process regionally may result in more streamlined data access and reporting.
Eliminating Obstacles with Technology Solutions
Datasets, such as tick history across securities, contain hundreds of terabytes of data. Previously, datasets of this size would have been prohibitively expensive for most firms to collect, store, and use. Technological advances have provided cost-effective options to get this data into the hands of customers via both on-demand and batch delivery methods.
In terms of data governance, data vendors are required by the content creators to ensure customers report usage of their data appropriately. Advances in access controls and audit logs allow vendors to provide better reporting and transparency to meet these requirements on the data they are delivering.
Data consumers need increased transparency into when, how, and why data was updated, modified, or enriched. When a customer manages their ETL process, they are left to find a solution—whether that’s maintaining the content in a point-in-time system, or just keeping an archive of changes to reference. Data providers that operated this ETL process for their client can efficiently scale and offer increased visibility to the end consumers.
|
Data Management Challenges
|
Cloud-Based Data Benefits
|
|
Costs (time and money) of implementing a new data feed
|
Ability to turn feeds on/off on demand
|
|
Moving and managing hundreds of terabytes of data across multiple tools and systems
|
Ability to bring data management tools to the data lake
|
|
Ensuring robust data governance and usage reporting
|
Authentication, audit logs, and automated reporting allow for better permissioning
|
|
Growing need for data transparency surrounding updates and enrichment
|
Data providers can manage the ETL on behalf of clients and provide the transparency
|
This article is an excerpt of “How to Gain a Competitive Advantage with Data Access in the Cloud.” View the full report on Finextra.com and read the related FactSet/AWS case study here.
The information contained in this article is not investment advice. FactSet does not endorse or recommend any investments and assumes no liability for any consequence relating directly or indirectly to any action or inaction taken based on the information contained in this article.
Post Comment
