dbt or data business transition: Reflecting on this year’s Coalesce highlights ✨
The Official Census Blog
After Coalesce in New Orleans last month, I went straight to Chicago. My mom picked me up from the airport, asked me where I was, and when I told her I was at a dbt conference, a familiar guessing game ensued: One where she tries to guess what the various jargon-y acronyms in data mean. For “dbt” she got “data” right away, with which we both were quite happy. But then she landed on “business transition” as the “bt” part.
While we all know that’s not actually correct, as I thought about it more, “dbt as data business transition” became a better and better summary of my Coalesce conference experience this year.
I joined the dbt community nearly two years ago while standing up the MDS (Fivetran, Snowflake, dbt, and Census) at Owner.com. Back then (in my day), the Slack community was only about half what it is now (if you want to learn more about how it’s changed, check out the work of Benn Stancil and Taylor Murphy). I immediately loved how welcomed I felt as I dug into new tools and the analytics engineering framework.
It’s wild to see how dbt has grown, especially at the end of Coalesce, which I think is one of the best – if not the best – conferences I’ve been to. The dbt Labs crew did a great job coordinating and scheduling talks (many of which were given by OA Club community members 🙌), running a fun Expo Hall, and weaving in New Orleans flair.
So, while I reflect on the many awesome community experiences I had this year, I wanted to highlight some of my favorite moments from the event (mostly focused on business applications of analytics engineering and dbt).
Bonus: If you’re looking for a great roundup of Coalesce talks on career growth, check out this post from Metaplane’s Founder & CEO Kevin Hu.
Tuesday was definitely my heaviest session day. Here are some of my favorites (in no particular order).
David’s blog post recap is here and the session recording is here.
In this talk, David speaks about three key benefits of analytics engineering
Overall, I would recommend this overview of analytics engineering’s impact from a technical and organizational perspective. I appreciated David’s coverage of data teams as cost centers and their technical impact, though I’d love to hear more of his perspective on an objective function around ROI versus KPIs.
You can find the talk recording for this session here.
Times have changed since Ralph Kimball published The Data Warehouse Toolkit in 1996, but a lot of the value of dimensional modeling has remained.
Now in a quasi-storage-unlimited world, some of the justifications of the techniques for minimizing storage requirements aren't as true. The age of ETL has become ELT, with transformation no longer happening in-flight, but rather within the warehouse with SQL and dbt.
The big pro of dimensional modeling is that it’s very understandable. Fact and dimension tables make sense of data structure, communicating them via two concepts (seen below):
The key takeaway from this talk? Just be thoughtful when data modeling. Don’t throw the baby out with the bathwater by ignoring the benefits of Kimball.
Bonus: If you want some additional reading, Sydney and Josh recommended reading Star Schema by Christopher Adamson.
You can find the talk recording here.
John and Lindsay opened their talk by diving into how data silos started in early-stage companies (illustrated by the great visual below).
But they really connected with the audience by turning focus to the first data person (or FDP) at a startup, a role many of us (including myself) have served in.
When it came to the responsibilities of the FDP, they called out two priorities:
And, beyond these two areas, Lindsay highlighted a hot take focus area I’m really in favor of: FDPs should teach tech skills when they join and have six months of lead time that includes three months hiring and three months of onboarding. I think this lets you teach SQL skills faster, given curiosity, analytical thinking, and communication skills are already present.
This is a must-watch for any FDPs at startups, including founders and ops leaders thinking about scaling the data team, and a great playbook on how to add value as soon as possible in a new role.
You can find this talk recording here.
Emilie did an awesome job laying the framework of how data and experimentation results don’t have full context via examples of Jane Fonda’s exclusion from a Netflix title card experiment where Jane Fonda was excluded from promotion materials based on misleading data (correlation =/= causation, anyone?).
Emilie used these examples to set up the takedown of Reforge’s post by Crystal Widjaja, which positions data-informed => data-driven => data-led as the transitive states of an organization’s journey to leveraging data. Emilie contests (and I wholeheartedly agree) that, ultimately, being data-led is dumb because data doesn’t tell the whole story. In fact, data:
Instead, Emilie proposed a new phase of data as both a substitute and the “right way” to evolve data teams: Data as a Product to Data as Operations (for context on getting to Data as a Product, see this Coalesce 2020 talk by Emilie and Taylor).
For any proactive data leader and practitioner, this is an absolute must-watch. Further reading on Data Business Partnerships can be found in this blog by Adam Stone.
You can find the talk recording here.
Kshitij did a great job explaining how to change an organization by bringing in best practices from software engineering. To start, he talked about change management in general (and why it’s important):
Note the “hit by a bus” analogy again 🚌, which was fitting as much of this talk touched on points from David’s earlier talk. The change management piece was around how to make sure new systems actually help teams ship analytics faster with confidence.
One great pattern he talked about was an Acceptance Criteria for new tickets that considered the following:
He chose these criteria because they focused on the “what” rather than the “how,” which gives everyone picking up the ticket the context needed to own its delivery.
He wrapped up by talking about guardrails for development and automated code review, specifically how analytics engineers have applied software engineering best practices (basically stating, “everything old is new again,” which I thought was a great final takeaway from the presentation). This is a great talk for data leaders building out a team to use as a north star template for a high-functioning team.
Transparently, I spent most of my Wednesday and Thursday on the conference floor talking with community members about their work, so I didn’t catch as many sessions as I would have liked (thankfully, they’re all on demand), but here are my favorites I was able to drop in on. 👇
You can find the talk recording here.
I always love catching Julia and Marc (and, of course, our very own Boris) on panels together. For this session, they touched on how to bridge data and analytics reporting with operational systems when stakeholders (like Sales) want to see how different data initiatives impact revenue.
Key takeaways from this session included:
This talk is excellent for data leaders looking to improve their working relationships with ops teams and better communicate the dollar value of their team’s work.
You can find the talk recording here.
This was an interesting talk on the various data assumptions that take place in the real world of clickstream data, including:
Emily gave great introductions to the “data is hard” feeling, jsonschema as a way to have validation of data types of fields, and the concept of an agreement.
Jake covered the best practices for getting in front of downstream problems, including defining, annotating, and evolving the jsonschema agreement with product engineering teams before putting the agreement into a registry. This enables product SDKs, as well as dbt, to read from the registry and apply validators to event definitions.
This talk resonated with me thanks to the event issues I’ve come across in the past. It seems like an elegant solution to solving them (and necessitates great cross-functional buy-in from engineers, product, and leadership). I’d recommend this talk to data leaders focused on event data, plus frustrated product analysts looking for a great end-vision for contracts.
Related reading: Sarah Krasnik’s article on an analytics requirements document (ARD).
You can find the session recording here.
Dylan did a great job outlining the pros and cons of scheduling in this talk, shown here:
I felt this talk at my core, given my experience scheduling questions. The key takeaway?
The real world operates in events, and operational data workflows should mirror this reality.
As Dylan explained, event-driven dataflows are best because they:
Thinking first about the organizational outcomes at the design phase of a pipeline helps future-proof the questions about data discrepancies and drive the largest impact possible.
I’d highly recommend watching this entire talk as a data engineer. Dylan included a spectacular example scenario of data flows from BlueJet that ensured unsafe aircraft do not take off.
The best part about Coalesce? Just because the event is over doesn’t mean the learning is! Here are the talks I’m sad I missed, but I'm excited to catch on-demand:
Overall, I enjoyed and learned a lot from all the talks I caught this year. All the presenters did a fantastic job covering the future-proofing analytics engineering and best data practices, and were quite reaffirming of the Operational Analytics (OA) space (peep the “data business transition” rebrand 👀) I spend a lot of time in at Census.
If you’re interested in learning more about OA and learning from the data community, join The OA Club, or reach out to me directly. I’m always happy to talk to other data nerds! 🤓
