Snowflake: The New Cloud-Based Data Warehousing Paradigm

Snowflake

The New Cloud-Based Data Warehousing Paradigm

Analytic Insights

Snowflake is the summary of many years of data-warehousing progress. It really shows how when you redefine an MPP (performance-type database) for the Cloud, with full elasticity and scalability, as well as unlimited concurrency, you can reduce your Cost of Ownership and your overall capital investment. It’s a win-win for everyone.  

By Archetype Team

I fly to Boston regularly for business and often sit in its notorious traffic. The city is old and the roads are narrow with cars trying to zip down streets built for horses and carriages. There’s no grid (like Chicago or New York) so there’s no easy way to get from one end of the city to the other. It’s easy to look around and wonder how city officials will plan for Boston’s future development and optimize for growth in a way that makes sense.

Like the City of Boston, many businesses already have systems and architecture in place thus requiring companies to build “go-arounds” in order to be successful. But unlike Boston, business can choose to completely change their system – start fresh – and disrupt their organizations for the better.

The data warehouse database Snowflake does just that, allowing businesses to completely overhaul their data warehouse systems. To me, Snowflake is the summary of many years of data-warehousing progress. It really shows how when you redefine an MPP (performance-type database) for the Cloud, with full elasticity and scalability, as well as unlimited concurrency, you can reduce your Cost of Ownership and your overall capital investment. It’s a win-win for everyone.  

Here are some more reasons why Snowflake is a best-in-class product.

BUILT FOR THE CLOUD

First and foremost, Snowflake was built for the Cloud. You don’t have finesse the software and do work-arounds. It also keeps the ANSI SQL standards and is extremely feature-rich.

Think about the Boston example. Due to geological constraints, the city doesn’t have the flexibility – or space, frankly – to grow as it needs to. Developers shove new developments into places that fully can’t meet their needs. And Bostonians are the ones who feel the effects – and struggle to find “new ways” around new congestion.

In the same way, IBM, Teradata, Google Big Query, and Amazon RedShift also moved to the Cloud. While their products have evolved for the Cloud, their structures’ flexibility comes with a lot of overhead. It’s kicking the can down the one of Boston’s narrow roads: it’s the same can and all the problems that existed on-prem still exist in the Cloud with those solutions. Not so with Snowflake.

BUILT-IN OPTIMIZATION

When you build a database, you must consider your organization’s big events that will, as a result, run big queries. Year end, month end, you name it — you’ll need all the “horsepower” available to you to run these queries. However, there are 300 “other” days where you aren’t running big queries and you don’t need the same amount of horsepower.

Most databases require a massive investment to have that horsepower available even if you only need it 60 days out of the year. To most companies, it’s a necessary evil to pay for.

Snowflake, on the other hand, gives us the epiphany of “warehouse as a service.” It allows companies to scale up or scale down on-demand. Organizations can scale compute up and down as needed, as well as storage (independently of each other on top of it all). If you’re about to do a big month-end process, you can scale up on demand to account for the higher requirement, get that query running and report in minutes and after, scale back down without making a long-term investment.

How does Snowflake allow this to take place? They have a complete separation of compute and storage. Since you separate the storage, you can scale the “horsepower” up and down on-demand without moving data to that faster compute.

IT ELIMINATES A LARGE CAPITAL AND ADMINISTRATOR INVESTMENT 

Conventional database warehouses are a large capital investment as well as a large administrator investment. Organizations need those skills on staff to ensure success. Think about the cost of scaling up and down. Administrators “stand up the data” and it’s a very manual and time-consuming process. Then, those same administrators have limits on how much they can scale up going forward.

Snowflake eliminates all of that. Because Snowflake has the ability to scale up or down on demand, the traditional constraints of standing up additional data storage is gone. Additionally, organizations don’t need to have those administrators on staff and can save on overhead. For example, you don’t need a DBA (database administrator) to write DDLs (data definition language) within Snowflake.

In the same light, organizations can also save on another important investment: time.

Snowflake allows organizations to do things in a development environment that isn’t available in other databases:

  • Clone data. If a user sees an issue, they can clone the production data to the development environment. When it comes to troubleshooting, that’s huge! The cloning takes place without consuming your storage space.

  • Travel back in time. If something goes awry, employees can go back to see the same schema, table, and row when it existed at any point back in time. For example, employees can see an issue before ETL ran, allowing them to easily spot the errors.

  • Query JSON-document (i.e. semi-structured) data. In Snowflake, documents are skinny and easy to read. In other databases, flattening documents requires programming and a developer needs to massage the data so it’s easy to query. It takes a long time. In Snowflake, it’s more straightforward.

When organizations think about growth, they need something that they can grow with, and not grow against. Don’t let database restrictions – or cost – hold your organization back. Use Snowflake to customize the capacity you need, right now, to ensure success and not waste efforts going forward.

Archetype is proud to partner with Snowflake. To learn more about their product, go here.