Best Practices

Why marketers should care about the data warehouse | Census

Sylvain Giuliani
Sylvain Giuliani December 07, 2022

Syl is the Head of Growth & Operations at Census. He's a revenue leader and mentor with a decade of experience building go-to-market strategies for developer tools. San Francisco, California, United States

Marketers typically don’t care about the technical backend of their software – they care about outcomes. So I know it’s a hard sell to get you to care about the backend of your customer data. 

But there’s a huge and important shift happening in the MarTech space that will change the way marketers do their job, as well as the way marketing technology is built and integrated into the long term.

The data warehouse is emerging as the single source of truth for marketing teams to do their job better and faster with customer data. There's only one problem: They can't access it. 🚫

“I’m blocked by the engineering team” isn’t going to cut it

One of the core challenges for marketing teams has always been data accessibility. Most marketers can relate to the pain of being stuck waiting on data from engineering teams, but this over-reliance on engineering is killing marketing agility. The opportunity cost of not launching, learning, and iterating adds up and can negatively affect the culture of the marketing team. 

Sure, it’s easy to point the finger at centralized data or IT teams that are usually operating under a different set of organizational priorities (hello cloud data migration 👋). But the reality is “I’m blocked by engineering” is not going to cut it as an excuse when the marketing team comes up short on their revenue goals.

Data technology is moving faster than MarTech, and marketers need to keep up

As Scott Brinker mentioned, digital marketing was evolving faster 5 years ago than the rest of the digital transformation happening in the business. That’s why Marketing Ops and Growth teams were built – and their technology was way ahead of the rest of the organization. 🔮

That’s now completely shifted. Companies are investing heavily in cloud data technology, as well as technical talent. In fact, last year the major cloud platforms reported revenue of $191 billion. 🤯 The data warehouse has become the foundational reporting layer for every team in the organization, but the data warehouse is usually owned by centralized data or IT teams. 

In order to keep up and continue to own their data, Marketing Ops and Growth teams have stepped up to become more data-literate. This increasing data literacy is happening at the same time as Marketing Ops becoming an incredibly strategic role in the organization. Now, 62% of CMOs rate Marketing Operations as the key area they wanted to spend their –  and their team’s – time on. 

But first, WTF is a Data Warehouse?

A data warehouse aggregates data from different sources across the company into a single, central, consistent data store for it to be queried. What happens to data once it lands in the warehouse? Today, it’s a whole lot of business intelligence (BI) and reporting, but data warehouses are becoming more sophisticated and now support more advanced use cases like machine learning. 

With the shift to the cloud, data warehouses like Snowflake, Databricks, and BigQuery are booming, with the data warehousing market predicted to reach $51.18 Billion by 2028. 📈 There’s also a whole suite of low-code tooling that has sprung up around the data warehouse known as the modern data stack, making it significantly easier and faster for companies to collect and centralize vast quantities of data at scale. 

The data warehouse acts as a single source of truth for all your data
The data warehouse acts as a single source of truth for all your data

Your data warehouse has all the data you need, you just can’t access it 

Companies with greater data ownership have seen a 14.9% improvement in overall marketing campaign effectiveness. Yet, most teams find themselves frustrated that they can’t activate personalized customer experiences because their customer data is fragmented across dozens of systems. 😠

The good news is the data warehouse removes data silos by consolidating data from your different sources into a single source of truth. 👍

The bad news is access to the data warehouse has been historically limited to data teams who speak SQL (Structured Query Language). 👎

Not only is it limited to users with SQL fluency, but the data warehouse is often a mess with hundreds, if not thousands, of tables that are not well documented. 🍝 Even if you were to get access to the data warehouse, it would take you a long time to find the data you need and can trust. 

Finally, the data warehouse isn’t immediately actionable. Today, the most common destination for data is a dashboard. But they don’t automatically drive action, so it tends to be where dashboards are where data goes to die. 🪦 Actually activating those insights still requires more complex pipelines and engineering effort. 

Data Activation: A new way for marketing teams to access the data warehouse

When it comes to the data warehouse, the challenge is no longer collecting the data – your data teams are already doing that. Instead, it’s:

  1. Not knowing what customer data is available 🤷
  2. Being able to access and activate that customer data across your marketing channels, without engineering effort. 🔑

The good news is SQL is no longer the only key to the data warehouse. There are now warehouse-native no-code tools like Census that enable marketers to unlock data directly from the warehouse – all without needing to know SQL. These warehouse-native activation tools enable marketers to:

  1. Sync customer attributes and lists from their data warehouse into “systems of action” such as Salesforce Marketing Cloud, Marketo & Hubspot, and update them in real-time as the data is transformed. This pattern is known as reverse ETL
  2. Build customer (and other object!) segments by filtering across trusted data models from the data warehouse and sync them to all their advertising and marketing automation platforms from a single place. 

What makes them even more powerful is that these segments are fully dynamic, which means new customers will automatically be added to your segment as they meet the criteria. You can also set up dynamic exclusions, so once a user has purchased a product, for example, they will no longer be targeted via ads. 

Census's visual segment builder

Bridging the gap between marketing and data teams 🤝

Warehouse-native data activation tools bridge the gap between marketing teams and data teams. 🌉 They allow data teams to define a set of trusted underlying tables or models that marketing teams can use to sync customer attributes, or combine to build dynamic segments of users. This brings the full power of SQL to marketers, resulting in better and more efficient campaigns and faster audience experimentation. And, ultimately, it enables marketing teams to get more out of their existing MarTech investments.

I started this piece by talking about the frustration marketing teams can face when they are blocked by data teams. But what I’m most excited about is how tools like Census will fundamentally change the way data and marketing teams collaborate for the better. 🙌

I regularly hear marketers describe the benefit of Census as, “We can now build segments during the meeting” or, “We had an idea for a campaign and were able to launch it in 30 minutes.” The goal isn’t for marketing teams to never talk to data teams ever again; it’s to remove false dependencies, make marketing teams more agile, and enable each side to spend more time on value-adding work. 

The future of the CDP is the data warehouse

You might be thinking this all sounds mighty familiar… Isn’t this what a Customer Data Platform (CDP) does? Well, CDPs emerged 5-6 years ago when cloud data warehouses were in their infancy, and marketing teams needed a way to centralize their customer data, build unified customer profiles, and activate that data across different marketing channels. In many ways, CDPs were a stopgap in the journey of marketing technology, but by trying to give fast access to data to marketing teams, the CDP created yet another silo of data

So, the data warehouse has all the data, but prior to data activation, it just wasn’t accessible to marketers. 🚫 Whereas CDPs were designed to be highly accessible to marketers, but they don’t have all the data and are therefore ineffective for data activation. 

CDP vendors like Twilio Segment and mParticle have recognized this deficiency by recently announcing their own versions of “reverse ETL connectors”. That sounds great in theory, but the reality is that these platforms were not natively designed for the data warehouse and any attempt to “slap on” reverse ETL will be an afterthought. 💭

The data warehouse, unlike the CDP, is not a hype cycle – and it’s certainly not going anywhere. Data warehouses have existed for over 40 years, and moving them to the cloud has made them infinitely more scalable and powerful. 🚀

Now, with data activation and reverse ETL democratizing access, the data warehouse is best positioned to become the system of customer records that powers not just your marketing technology, but the entire business operations of the company. ⭐

How to get value from your data warehouse today

As a marketer, you don’t need to boil the ocean. In fact, it’s possible for you to build out highly effective campaign experiments using only a few tables from your data warehouse that likely already exist. 

How? The first step is to chat with your friendly data person today. If you don’t already know, find out what data warehouse you’re using, how data lands there, and what data has already been modeled.

Hint: Usually these are the underlying tables that are joined together to power your BI reports.

For example, say you have a customer lifetime value dashboard. Some of the underlying tables powering that dashboard might be an “orders by customer”, and an “LTV by customer” table. You could use this data today to build a segment such as 👇

“People who purchased SKU A last year, have previously purchased 3 products, but haven’t yet purchased SKU B

Then you can start activating that audience across your marketing channels (a conveniently relevant example as we head into the holiday season). 🎄

Most marketers are generally not short of great ideas. It’s more about understanding what data is available and how easy it is to activate these segments on top of the data warehouse that completely shifts their thinking. With data activation you no longer need to become a SQL pro, you just need to know where your best data lives and how you can access it.

If you’d like to learn more about how Census works with marketing teams to activate data trapped in the data warehouse, book a demo with one of our product specialists. 💪

🙏 Plus, a very special thank you to Juan Mendoza and Scott Brinker. Reading a lot of their incredible MarTech content was the inspiration for this post.

Related articles

Customer Stories
Built With Census Embedded: Labelbox Becomes Data Warehouse-Native
Built With Census Embedded: Labelbox Becomes Data Warehouse-Native

Every business’s best source of truth is in their cloud data warehouse. If you’re a SaaS provider, your customer’s best data is in their cloud data warehouse, too.

Best Practices
Keeping Data Private with the Composable CDP
Keeping Data Private with the Composable CDP

One of the benefits of composing your Customer Data Platform on your data warehouse is enforcing and maintaining strong controls over how, where, and to whom your data is exposed.

Product News
Sync data 100x faster on Snowflake with Census Live Syncs
Sync data 100x faster on Snowflake with Census Live Syncs

For years, working with high-quality data in real time was an elusive goal for data teams. Two hurdles blocked real-time data activation on Snowflake from becoming a reality: Lack of low-latency data flows and transformation pipelines The compute cost of running queries at high frequency in order to provide real-time insights Today, we’re solving both of those challenges by partnering with Snowflake to support our real-time Live Syncs, which can be 100 times faster and 100 times cheaper to operate than traditional Reverse ETL. You can create a Live Sync using any Snowflake table (including Dynamic Tables) as a source, and sync data to over 200 business tools within seconds. We’re proud to offer the fastest Reverse ETL platform on the planet, and the only one capable of real-time activation with Snowflake. 👉 Luke Ambrosetti discusses Live Sync architecture in-depth on Snowflake’s Medium blog here. Real-Time Composable CDP with Snowflake Developed alongside Snowflake’s product team, we’re excited to enable the fastest-ever data activation on Snowflake. Today marks a massive paradigm shift in how quickly companies can leverage their first-party data to stay ahead of their competition. In the past, businesses had to implement their real-time use cases outside their Data Cloud by building a separate fast path, through hosted custom infrastructure and event buses, or piles of if-this-then-that no-code hacks — all with painful limitations such as lack of scalability, data silos, and low adaptability. Census Live Syncs were born to tear down the latency barrier that previously prevented companies from centralizing these integrations with all of their others. Census Live Syncs and Snowflake now combine to offer real-time CDP capabilities without having to abandon the Data Cloud. This Composable CDP approach transforms the Data Cloud infrastructure that companies already have into an engine that drives business growth and revenue, delivering huge cost savings and data-driven decisions without complex engineering. Together we’re enabling marketing and business teams to interact with customers at the moment of intent, deliver the most personalized recommendations, and update AI models with the freshest insights. Doing the Math: 100x Faster and 100x Cheaper There are two primary ways to use Census Live Syncs — through Snowflake Dynamic Tables, or directly through Snowflake Streams. Near real time: Dynamic Tables have a target lag of minimum 1 minute (as of March 2024). Real time: Live Syncs can operate off a Snowflake Stream directly to achieve true real-time activation in single-digit seconds. Using a real-world example, one of our customers was looking for real-time activation to personalize in-app content immediately. They replaced their previous hourly process with Census Live Syncs, achieving an end-to-end latency of <1 minute. They observed that Live Syncs are 144 times cheaper and 150 times faster than their previous Reverse ETL process. It’s rare to offer customers multiple orders of magnitude of improvement as part of a product release, but we did the math. Continuous Syncs (traditional Reverse ETL) Census Live Syncs Improvement Cost 24 hours = 24 Snowflake credits. 24 * $2 * 30 = $1440/month ⅙ of a credit per day. ⅙ * $2 * 30 = $10/month 144x Speed Transformation hourly job + 15 minutes for ETL = 75 minutes on average 30 seconds on average 150x Cost The previous method of lowest latency Reverse ETL, called Continuous Syncs, required a Snowflake compute platform to be live 24/7 in order to continuously detect changes. This was expensive and also wasteful for datasets that don’t change often. Assuming that one Snowflake credit is on average $2, traditional Reverse ETL costs 24 credits * $2 * 30 days = $1440 per month. Using Snowflake’s Streams to detect changes offers a huge saving in credits to detect changes, just 1/6th of a single credit in equivalent cost, lowering the cost to $10 per month. Speed Real-time activation also requires ETL and transformation workflows to be low latency. In this example, our customer needed real-time activation of an event that occurs 10 times per day. First, we reduced their ETL processing time to 1 second with our HTTP Request source. On the activation side, Live Syncs activate data with subsecond latency. 1 second HTTP Live Sync + 1 minute Dynamic Table refresh + 1 second Census Snowflake Live Sync = 1 minute end-to-end latency. This process can be even faster when using Live Syncs with a Snowflake Stream. For this customer, using Census Live Syncs on Snowflake was 144x cheaper and 150x faster than their previous Reverse ETL process How Live Syncs work It’s easy to set up a real-time workflow with Snowflake as a source in three steps: