When it comes to data warehousing, the offerings have undergone constant improvements and have become increasingly progressive in recent years. One of OSS Group’s long-standing clients recently implemented the Snowflake solution. In this article, we will explain the reasons for this choice and the results obtained.
Reinventing Old Data Models
It had become increasingly clear that the client’s existing data models were unable to meet the emerging needs of the business. Too many data silos, a lack of trusted data, poor data availability, and an on-prem database operating close to capacity: the time had come to implement changes.
Keeping the existing data warehouse and improving it was no longer an option. Upgrading and building out the legacy solution would have been too slow and too expensive and still wouldn’t meet the future demands of the business: to deliver near real-time data and analytics to propel the business forward.
As a result, not only did the technology layer need improving, but a new data model was needed to provide more flexibility and agility. The objectives were clear: more scalability, reliability, security and more simplicity.
Modernising Data Using the Snowflake Cloud Platform
OSS Group advocated for Snowflake, an increasingly popular cloud data warehouse solution. The combination of its capabilities, pricing, on-demand scaling and ease of use, soon convinced our client of its significant benefits to its business.
The project was kicked off and we agreed on launching several parallel workstreams to increase our efficiency and speed of delivery.
A primary workstream introduced a data governance framework establishing the policies, standards, and practices necessary to ensure that data quality, data access and security were baked into the data platform. This guided the client in how to redefine roles and responsibilities, as well as developing a data dictionary that enabled them to adopt a common language within the organisation. In doing so, it allowed the client to future proof its data strategy and to further leverage its data assets across the business.
In conjunction with Snowflake, the client implemented Talend, a leading cloud data integration and governance platform. Talend’s drag-and-drop low code environment, provided much needed efficiency gains when it came to building, maintaining and operating their data pipelines. This modern platform allowed the client to enhance their data integration processes and to adopt re-usable patterns, ensure ongoing compliance and refinement.
The core of our approach to assuring the ongoing success of the platform has been the involvement and enablement of the client’s in-house teams that will be responsible for supporting and extending the solution. The objective being for the client to be self-sufficient for the most part but with OSS Group being in a position to be called on to help them flex that capability if and when required.
Delivering a Data Platform for Data-Driven Insights
The key main benefits of this project included:
Security and trust: this solution implemented improved the permissions model and user management processes. Data lineage is more transparent
Ease of use: data silos were replaced with a consolidated data model, providing a more complete picture for data and reporting
Operational efficiency: delivering new data pipelines takes less time and is more cost-efficient as the client pays for actual use
Timeliness: the solution supports near real-time delivery of data
Collaboration: data consumers refer to the data dictionary for authoritative definitions of what data represents.
In a nutshell, the data platform based on Snowflake and Talend, enabled the client to shift towards its goal of generating more comprehensive insights from its data. This flexible and cloud-based solution also provided the assurances that it will support its future growth and aspirations.