4 best practices for a successful Snowflake migration

Article by: Mick Wagner

As adoption of data-intensive applications escalates, many organizations are opting to migrate from their legacy on-premise data platforms to Snowflake’s cloud-based data warehouse solution, and with good reason. Among other benefits, Snowflake separates storage and compute resources, allowing businesses to use and pay for the resources they actually use and to scale each independently, thus optimizing use of their budgetary dollars. Done right, a migration to Snowflake can save money and accelerate progress toward becoming a data-driven organization. But as you might expect, there are some common pitfalls to be avoided and also some best practices that can help ensure businesses get the greatest bang for their budgetary bucks.

In this article, we’ll explore four categories of best practices that we’ve implemented to help our clients ensure a smooth migration and make the best use of the features and resources Snowflake offers.

1. Plan for cost management

If an on-prem platform is the equivalent of having your own car in the garage, Snowflake is more like using Uber: you don’t have to worry about gas or maintenance, but every trip incurs a discrete cost.

When managed well, Snowflake’s subscription model can make it significantly cheaper than an on-premise platform; conversely, poor management can lead to substantial unnecessary costs. To prevent a case of sticker shock when the first bill arrives — or when you find out your Snowflake credits have been prematurely depleted — make sure that cost management is part of your initial migration planning process. Be sure to consider questions such as

Who in the organization should have access to Snowflake, and what privileges will each person have? Snowflake allows admins to restrict account access to specific IP addresses and to set network policies on a per-user basis. Make sure that access is granted only to users who absolutely need it and who understand how the per-query pricing model works.

What are your organization’s typical data workflows and usage scenarios, and what are the storage and compute requirements for each? Once you understand your data workflows, you can do cost modeling for the appropriately sized compute warehouses.

Which data needs to be moved, and which data can remain in its current place? Snowflake has powerful data sharing options that allow data to stay in one spot with explicit permissions for different use cases, whereas in the past you may have had to copy data to multiple environments.

How will your disaster recovery and mitigation planning change? Snowflake’s Time Travel and data protection features may meet your business needs for data continuity so that you can decrease your disaster recovery storage costs.

2. Coordinate complementary tools

Snowflake’s extended ecosystem encompasses a wide range of third-party solutions, and some vendors are “certified” partners that can roll up costs into the same cloud provider that Snowflake is using. Their partner network encompasses tools for:

• Data pipelines / connectors (e.g. Fivetran)

• Data integration / ELT (e.g. Matillion)

• Reporting and analytics (e.g. Microsoft Power BI, Tableau)

• Data governance (e.g. Informatica)

• Data catalogs (e.g. data.gov, Collibra)

As you plan your Snowflake migration, consider which tools you will be using and how to coordinate them for optimal performance. You may also want to consider Snowflake’s certified partners to simplify billing.

3. Build training and communication into the process … and start early

The earlier you consider the training and communication requirements for a successful migration, the better prepared your users will be once the transition is complete — and the sooner you can begin realizing the benefits of the Snowflake platform. Be sure to consider the unique knowledge requirements of each user group:

Data developers

Examples of key considerations:

• The impact of changing data warehouse compute settings when optimizing query performance

• Design patterns to resolve differences in SQL between Snowflake and the original databases; it’s also useful to understand and share anti-patterns that could cause expensive queries.

• How to use the Time Travel feature to restore data objects in case of accidental deletion or modification

Dashboard developers

Examples of key considerations:

• Impact of their dashboard designs

• How to plan a refresh strategy (live connect versus once-daily refreshes) based on underlying changes in the data

• Key differences between connecting with Snowflake and connecting with other sources; a training guide (or even a video walkthrough) should be created for less-technical users.

Data analysts

Examples of key considerations:

• Key differences in using the online Snowflake UI, since they will be accustomed to using other querying tools

• Cost and impact of large queries, especially in cases where sampling would have met their needs

• Benefits of unique features such as data profiling, saving queries, etc.

Data product owners

Example of a key consideration:

• Both the benefits and the costs of high-performance requests, and the impact on the overall design of their product

DBAs

Examples of key considerations:

• What developers will be doing differently

• Data-sharing concepts

• Price modeling for different warehouse sizes

• Documenting and designing workloads

• Appropriate management tasks

4. Have a plan in place for monitoring, management, and measurement

Monitoring

Management

Measurement

Snowflake offers businesses the opportunity to share data seamlessly, eliminate resource contention, easily scale virtual warehouses, and realize a host of other benefits — all while managing costs. The pay-as-you-go pricing model enables organizations to directly link costs with the value extracted from their data, providing greater granularity in value cost management. And with careful planning, mindful execution, and ongoing diligence, migrating to Snowflake can be one of the best moves they ever made.

View the full article with images here: https://www.logic2020.com/insight/snowflake-migration-best-practices

Enabling clarity through business and technology solutions.