How deet fixes data

Features that keepdata clean

Explore the capabilities and receipts that keep broken data out of production.

Type SystemNullability TrackingMulti-BackendLSP & IDE SupportTesting & AssertionsLineage Graph

Why the presentation stuck

The features matter because the proof loop is fast, visible, and repeatable.

Proof in minutes

A loop that ends with a green check and real output files.

Receipts over claims

Commands and artifacts stand in for slides and opinions.

Portable outputs

SQL and contracts run anywhere, so adoption feels safe.

Shareable context

.deet and templates keep every demo consistent.

Read the method
Type System

Catch errors before they catch you

A powerful type system that infers types from your data sources and propagates them through every transformation. No more runtime surprises.

models/users.dt

Inferred Types

Types are automatically inferred from your data sources. No need for manual annotations.

Propagated Safety

Type information flows through every transformation, catching errors instantly.

Compile-Time Checks

All type errors are caught before deployment, not at 3am in production.

Nullability Tracking

Know exactly what can be null

Explicit nullable types (string?) and compile-time null checks eliminate the most common source of data bugs. Filter once, type-safe forever.

Runtime Error
query.sql
-- This will fail at runtime if email is NULL
SELECT
  id,
  UPPER(email) as email_upper,
  LENGTH(email) as email_length
FROM users
-- ERROR: null value in column "email"
-- Query failed after processing 1.2M rows
Compile-Time Safety
query.dt
<span class="text-deet-orange font-semibold">model</span> user_emails = users
  <span class="text-deet-green font-semibold">|></span> <span class="text-cyan-400">filter</span>(email != <span class="text-deet-orange font-semibold">null</span>)  <span class=<span class="text-amber-400">"text-slate-500 italic"</span>>// Now email: <span class="text-purple-400">string</span></span>
  <span class="text-deet-green font-semibold">|></span> <span class="text-cyan-400">derive</span>(
    email_upper: <span class="text-cyan-400">upper</span>(email),   <span class=<span class="text-amber-400">"text-slate-500 italic"</span>>// Safe!</span>
    email_length: length(email)  <span class=<span class="text-amber-400">"text-slate-500 italic"</span>>// Safe!</span>
  )

<span class=<span class="text-amber-400">"text-slate-500 italic"</span>>// Compiler guarantees: no <span class="text-deet-orange font-semibold">null</span> errors possible</span>

How Nullability Tracking Works

before_filter.dt
1<span class="text-deet-orange font-semibold">source</span> users: {
2  id: <span class="text-purple-400">i64</span>
3  email: <span class="text-purple-400">string</span>?  <span class=<span class="text-amber-400">"text-slate-500 italic"</span>>// Nullable</span>
4}
5 
6<span class=<span class="text-amber-400">"text-slate-500 italic"</span>>// email is <span class="text-purple-400">string</span>? here</span>
7users.email  <span class=<span class="text-amber-400">"text-slate-500 italic"</span>>// Type: <span class="text-purple-400">string</span>?</span>
after_filter.dt
1<span class="text-deet-orange font-semibold">model</span> active_users = users
2  <span class="text-deet-green font-semibold">|></span> <span class="text-cyan-400">filter</span>(email != <span class="text-deet-orange font-semibold">null</span>)
3 
4<span class=<span class="text-amber-400">"text-slate-500 italic"</span>>// email is now <span class="text-purple-400">string</span>!</span>
5active_users.email  <span class=<span class="text-amber-400">"text-slate-500 italic"</span>>// Type: <span class="text-purple-400">string</span></span>
Multi-Backend

One codebase, every warehouse

Write your transformations once, compile to DuckDB, PostgreSQL, BigQuery, or Snowflake. Switch backends without changing a line of code.

analytics.dt
1<span class="text-deet-orange font-semibold">module</span> analytics
2 
3<span class="text-deet-orange font-semibold">source</span> users = <span class="text-cyan-400">table</span>(<span class="text-amber-400">"users"</span>) : {
4  id: <span class="text-purple-400">i64</span>
5  name: <span class="text-purple-400">string</span>
6  email: <span class="text-purple-400">string</span>?
7  created_at: <span class="text-purple-400">timestamp</span>
8}
9 
10<span class="text-deet-orange font-semibold">model</span> active_users = users
11  <span class="text-deet-green font-semibold">|></span> <span class="text-cyan-400">filter</span>(email != <span class="text-deet-orange font-semibold">null</span>)
12  <span class="text-deet-green font-semibold">|></span> <span class="text-cyan-400">derive</span>(days_active: datediff(<span class="text-amber-400">'day'</span>, created_at, <span class="text-cyan-400">now</span>()))
13  <span class="text-deet-green font-semibold">|></span> <span class="text-cyan-400">filter</span>(days_active > 0)
14 
15<span class=<span class="text-amber-400">"text-slate-500 italic"</span>>// Single <span class="text-deet-orange font-semibold">source</span> of truth</span>
16<span class=<span class="text-amber-400">"text-slate-500 italic"</span>>// Compiles to any backend</span>
Compiled Output - DuckDBOptimized
-- Generated for DuckDB
SELECT
  u.id,
  u.name,
  COALESCE(u.email, '') AS email,
  DATE_DIFF('day', u.created_at, NOW()) AS days_active
FROM users u
WHERE u.email IS NOT NULL
  AND DATE_DIFF('day', u.created_at, NOW()) > 0
DuckDBLocal development
PostgreSQLProduction OLTP
BigQueryAnalytics at scale
SnowflakeEnterprise warehouse
LSP & IDE Support

Your editor, supercharged

Full Language Server Protocol support brings autocomplete, go-to-definition, hover types, and inline errors to VS Code, Neovim, and more.

analytics.dt - Visual Studio Code
1<span class="text-deet-orange font-semibold">module</span> analytics
2 
3<span class="text-deet-orange font-semibold">source</span> users = <span class="text-cyan-400">table</span>(<span class="text-amber-400">"users"</span>) : {
4 id: <span class="text-purple-400">i64</span>
5 name: <span class="text-purple-400">string</span>
6 email: <span class="text-purple-400">string</span>?
7}
8 
9<span class="text-deet-orange font-semibold">model</span> active = users <span class="text-deet-green font-semibold">|></span> fi
deet0 errors
Ln 9, Col 28UTF-8

Autocomplete

Context-aware suggestions for functions, columns, and models

Inline Errors

See type errors and warnings as you type, not after

Go to Definition

Jump to model definitions with a single click

Hover Types

See inferred types by hovering over any expression

Works with your favorite editor

VS Code Neovim JetBrains
Testing & Assertions

Data quality built-in

Built-in assertions and contracts validate your data at every stage. Test your transformations like you test your code.

tests/revenue_test.dt
1<span class="text-deet-orange font-semibold">module</span> tests
2 
3<span class="text-deet-orange font-semibold">import</span> analytics
4 
5<span class=<span class="text-amber-400">"text-slate-500 italic"</span>>// Inline assertions</span>
6<span class="text-deet-orange font-semibold">model</span> validated_revenue = analytics.revenue
7  <span class="text-deet-green font-semibold">|></span> <span class="text-deet-orange font-semibold">assert</span>(total_revenue >= 0, <span class="text-amber-400">"Revenue must be positive"</span>)
8  <span class="text-deet-green font-semibold">|></span> <span class="text-deet-orange font-semibold">assert</span>(order_count > 0, <span class="text-amber-400">"Must have orders"</span>)
9  <span class="text-deet-green font-semibold">|></span> <span class="text-deet-orange font-semibold">expect</span>(avg_order_value < 10000, <span class="text-amber-400">"Flag unusual values"</span>)
10 
11<span class=<span class="text-amber-400">"text-slate-500 italic"</span>>// Data contracts</span>
12<span class="text-deet-orange font-semibold">test</span> revenue_has_required_fields =
13  analytics.revenue
14    <span class="text-deet-green font-semibold">|></span> <span class="text-cyan-400">limit</span>(1)
15    <span class="text-deet-green font-semibold">|></span> <span class="text-deet-orange font-semibold">assert</span>(customer_id is <span class="text-deet-orange font-semibold">not</span> <span class="text-deet-orange font-semibold">null</span>, <span class="text-amber-400">"customer_id required"</span>)
16    <span class="text-deet-green font-semibold">|></span> <span class="text-deet-orange font-semibold">assert</span>(order_date is <span class="text-deet-orange font-semibold">not</span> <span class="text-deet-orange font-semibold">null</span>, <span class="text-amber-400">"order_date required"</span>)
17    <span class="text-deet-green font-semibold">|></span> <span class="text-deet-orange font-semibold">assert</span>(total_revenue is <span class="text-deet-orange font-semibold">not</span> <span class="text-deet-orange font-semibold">null</span>, <span class="text-amber-400">"total_revenue required"</span>)
18 
19<span class="text-deet-orange font-semibold">test</span> revenue_is_consistent =
20  analytics.revenue
21    <span class="text-deet-green font-semibold">|></span> <span class="text-cyan-400">group</span>(<span class="text-deet-orange font-semibold">by</span>: [order_date], measures: { daily_total = <span class="text-cyan-400">sum</span>(total_revenue) })
22    <span class="text-deet-green font-semibold">|></span> <span class="text-deet-orange font-semibold">assert</span>(daily_total >= 0, <span class="text-amber-400">"Daily total must be non-negative"</span>)

Test Results

$ deet test
revenue_has_required_fields (2ms)
revenue_is_consistent (45ms)
expect: 3 rows flagged for review
2 passed|1 warning|47ms
Assertions
Runtime checks
Expectations
Soft warnings
Contracts
Schema validation
Unit Tests
Isolated testing
Lineage Graph

See how data flows

Interactive column-level lineage tracking shows exactly how data moves through your pipeline. Debug with confidence, refactor with clarity.

Understand your data at a glance

deet automatically tracks how data flows through your models, from source tables to final outputs. See exactly which columns depend on which sources, and understand the impact of any change before you make it.

Column-level lineage tracking
Impact analysis for schema changes
Automatic documentation generation
Interactive graph visualization
Column-Level Lineage
Source Tables
Models
lineage_query.dt
1<span class=<span class="text-amber-400">"text-slate-500 italic"</span>>// Query lineage programmatically</span>
2let deps = lineage(revenue_report)
3 
4<span class=<span class="text-amber-400">"text-slate-500 italic"</span>>// What tables does this <span class="text-deet-orange font-semibold">model</span> depend <span class="text-deet-orange font-semibold">on</span>?</span>
5deps.sources      <span class=<span class="text-amber-400">"text-slate-500 italic"</span>>// [<span class="text-amber-400">"users"</span>, <span class="text-amber-400">"orders"</span>, <span class="text-amber-400">"products"</span>]</span>
6 
7<span class=<span class="text-amber-400">"text-slate-500 italic"</span>>// What columns flow into <span class="text-amber-400">'total_revenue'</span>?</span>
8deps.column(<span class="text-amber-400">"total_revenue"</span>).upstream
9<span class=<span class="text-amber-400">"text-slate-500 italic"</span>>// [<span class="text-amber-400">"orders.amount"</span>, <span class="text-amber-400">"orders.quantity"</span>, <span class="text-amber-400">"products.price"</span>]</span>
10 
11<span class=<span class="text-amber-400">"text-slate-500 italic"</span>>// What would break <span class="text-deet-orange font-semibold">if</span> we remove <span class="text-amber-400">'users.email'</span>?</span>
12deps.impact(<span class="text-amber-400">"users.email"</span>)
13<span class=<span class="text-amber-400">"text-slate-500 italic"</span>>// [<span class="text-amber-400">"user_orders.email"</span>, <span class="text-amber-400">"revenue_report.customer_email"</span>]</span>

Ready to experience the future of data engineering?

Try deet in our interactive playground - no signup required. See the type system, null safety, and multi-backend compilation in action.

Launch PlaygroundRead the Docs