Data teams today are drowning in a sea of unstructured, messy information, while the value of that data is becoming increasingly important for business teams. Simply moving raw data from Databricks to activation platforms isn't enough. Teams need to enrich, categorize, and transform data before it reaches business users, but doing so has traditionally required complex ETL processes or custom code.
That's why, alongside the launch of Databricks Data Intelligence for Marketing, we're excited to introduce Census AI Columns for Databricks, a powerful solution that helps you transform your data lake with AI before syncing it to your marketing, sales, and customer engagement platforms.
Bringing AI directly to your Lake
For data teams managing customer data in Databricks, AI Columns opens up powerful new capabilities without complex code. Now, you can apply LLM-based transformations directly to your Databricks tables and views, enhancing data quality and extracting insights at scale.
You have the choice of ChatGPT (Open AI), Claude (Anthropic), or Gemini (Google) to power AI Columns; you can also use Census credits or just bring your own API key to stay in complete control. It is easy to get started with AI Columns. You simply:
- Write liquid template prompts that turn raw data into columns in your dataset.
- Define validation rules and default values to ensure data quality.
- Write transformed data back to Databricks or sync directly to destination platforms.
How can teams use AI Columns with Databricks?
All teams can get value out of AI Columns with Databricks. You can create structured insights, enrich your customer data, and normalize your records with just a few clicks.
Extract structured insights from unstructured data
Databricks excels at processing large volumes of data, but extracting meaning from text fields often requires custom ML models. AI Columns simplifies this process. For example, data teams can extract product feedback themes from support tickets by pulling ticket data from Databricks tables, asking the LLM to categorize issues and extract sentiment scores, then syncing results to Salesforce for CS teams or Gainsight for product teams.
Enhance customer data for marketing personalization
The richest customer data is often locked in unstructured formats. AI Columns helps marketers access these insights with no code. You can generate personalized engagement recommendations by combining product usage data, purchase history, and website activity from Databricks. From there, you can use AI Columns to generate next-best-action recommendations, then sync personalized messaging to Braze, Iterable, or Customer.io.
Automate data classification for segmentation
Manual classification of customers and accounts is time-consuming and inconsistent. AI Columns can classify companies by industry and maturity by feeding company descriptions and website data from Databricks, determining industry, company size, and tech maturity, then syncing classifications to Facebook Ads, Google Ads, or LinkedIn for targeted campaigns.
Clean and normalize data without complex transformations
Data cleaning doesn't have to mean regex nightmares or custom Python. AI Columns makes it easy to standardize job titles and departments by inputting messy job titles from your Databricks tables, normalizing and classifying titles by function and seniority, then syncing clean data to Salesforce, HubSpot, or Outreach for better segmentation. You can also sync the normalized data back into Databricks for future use.
Top 5 AI Column templates to get started
Here are powerful prompts you can implement today with Databricks:
1. Customer Lifecycle Stage Classification
Analyze customer data and classify them into lifecycle stages like 'New', 'Growing', 'Mature', 'At Risk', or 'Churned' based on usage frequency, feature adoption, account age, last active date, and contract end date.
2. Product Feature Recommendations
Based on customer usage patterns, recommend the top features they should adopt next, providing brief explanations for each recommendation by analyzing current features used, industry, company size, and similar customers' feature adoption.
3. Customer Feedback Summarization
Summarize customer feedback into concise insights, categorizing it as 'Bug', 'Feature Request', 'UX Improvement', 'Performance Issue', or 'Other', then assigning priority levels based on sentiment and impact.
4. Marketing Campaign Personalization
Generate personalized product recommendations and messages for email campaigns based on customer behavior and profile, including past purchases, browsing history, cart abandons, and customer segments.
5. Lead Scoring Enhancement
Calculate enhanced lead scores based on job titles, company descriptions, email engagement, website visits, product usage, and firmographic data, with brief explanations for scores and recommended next actions for sales.
Getting started with AI Columns for Databricks
Setting up AI Columns with your Databricks environment is straightforward. Connect Census to your Databricks instance, select the table or view containing your source data, and add an AI Column and craft your prompt. That’s it!
To learn more about implementing AI Columns with Databricks, check out our documentation or book a demo with our team.