Data Lakes vs Warehouses vs Lakehouses

The data storage landscape has three major architectures. Each one emerged because the previous one could not solve every problem. Understanding when and why to use each is a core data engineering skill.

Data warehouse

A data warehouse is a structured, schema-on-write database optimized for analytical SQL queries.

How it works: data is cleaned, validated, and loaded into predefined schemas (star or snowflake). The warehouse enforces types, constraints, and structure at write time.

Strengths:

• Excellent SQL query performance — columnar storage, query optimization, caching.
• Strong governance — schema enforcement, access control, audit trails.
• BI tools integrate natively — Looker, Tableau, Metabase all expect a warehouse.
• ACID transactions — consistent reads, no partial writes.

Weaknesses:

• Structured data only — no images, logs, videos, or raw JSON blobs.
• Schema must be defined upfront — changes require migration.
• Expensive storage — warehouse storage costs 10-100x more than object storage.
• Not ideal for ML workloads — data scientists need raw, flexible data.

Popular warehouses: Snowflake, Google BigQuery, Amazon Redshift, Databricks SQL.

Data lake

A data lake is a centralized repository that stores raw data in its native format on cheap object storage (S3, GCS, Azure Blob).

How it works: everything gets dumped in — CSVs, JSON, Parquet, images, logs, API responses. Schema is applied at read time, not write time (schema-on-read).

Strengths:

• Stores any data type — structured, semi-structured, unstructured.
• Extremely cheap storage — S3 costs ~$0.023/GB/month vs $23+/TB/month for warehouses.
• Ideal for ML — data scientists can access raw data in any format.
• Flexible — no schema migration needed, store now and figure out structure later.

Weaknesses:

• No ACID transactions — concurrent reads/writes can produce inconsistent results.
• No schema enforcement — data quality degrades over time (“data swamp”).
• Poor query performance for SQL — scanning raw files is slow without optimization.
• Weak governance — hard to track lineage, enforce access, or audit usage.

Common implementations: S3 + Athena, GCS + BigQuery external tables, ADLS + Databricks.

The data swamp problem

Without governance, a data lake becomes a data swamp:

• Files with no documentation — nobody knows what export_final_v2_FIXED.csv contains.
• Schema drift — the same file type has different column names across months.
• Stale data — files from 2019 sit next to files from today, all treated equally.
• No access control — sensitive PII mixed with public metrics.
• No lineage — impossible to trace where a number came from.

This is why the industry invented the lakehouse.

Data lakehouse

A lakehouse adds a metadata and transaction layer on top of a data lake, giving it warehouse-like capabilities while keeping the cheap storage and format flexibility of a lake.

How it works: data still lives as Parquet/ORC files on object storage. A table format layer (Delta Lake, Apache Iceberg, or Apache Hudi) adds ACID transactions, schema enforcement, time travel, and partition management on top of those files.

┌──────────────────────────────────────┐
│        Query Engines                  │
│   Spark / Trino / Presto / Flink     │
└────────────────┬─────────────────────┘
                 │
┌────────────────▼─────────────────────┐
│        Table Format Layer             │
│   Delta Lake / Iceberg / Hudi         │
│   (ACID, Schema, Time Travel)         │
└────────────────┬─────────────────────┘
                 │
┌────────────────▼─────────────────────┐
│        Object Storage                 │
│   S3 / GCS / ADLS                     │
│   (Parquet files)                     │
└──────────────────────────────────────┘

Strengths:

• ACID transactions on cheap object storage.
• Schema enforcement with evolution (add columns without rewriting).
• Time travel — query data as it existed at any past point.
• Works for both SQL analytics and ML workloads.
• Open formats — no vendor lock-in, multiple engines can read the same tables.

Weaknesses:

• More complex to operate than a managed warehouse.
• Query performance still lags behind purpose-built warehouses for complex SQL.
• Ecosystem is younger — tooling gaps compared to mature warehouse platforms.

Table format comparison

Feature	Delta Lake	Apache Iceberg	Apache Hudi
Creator	Databricks	Netflix	Uber
ACID	Yes	Yes	Yes
Time travel	Yes	Yes	Yes
Schema evolution	Yes	Yes (best)	Yes
Partition evolution	Limited	Yes (hidden partitions)	Limited
Engine support	Spark-first	Multi-engine (Spark, Trino, Flink)	Spark, Flink
Upserts	MERGE	MERGE	Native (optimized for CDC)
Ecosystem	Databricks-centric	Vendor-neutral	Smaller community

Apache Iceberg has the broadest engine support and best partition evolution. Delta Lake has the deepest Databricks integration. Apache Hudi was purpose-built for change data capture (CDC) and incremental processing.

Side-by-side comparison

Feature	Data Warehouse	Data Lake	Data Lakehouse
Data types	Structured only	All types	All types
Schema	On-write	On-read	Both
ACID	Yes	No	Yes
Storage cost	High	Low	Low
Query performance	Excellent	Variable	Good
Governance	Strong	Weak	Strong
ML support	Limited	Excellent	Excellent
Best for	BI / Reporting	Raw storage / ML	Unified analytics

When to use each

Use a warehouse when:

• Your team is SQL-first and BI-focused.
• You want a managed service with minimal operations overhead.
• Data is structured and schema changes are infrequent.
• Budget allows warehouse storage costs.

Use a data lake when:

• You need to store raw, unstructured data (images, logs, videos).
• Cost is the primary concern and you have petabytes of data.
• ML teams need flexible access to raw data.
• You are building a staging area for an ELT pipeline.

Use a lakehouse when:

• You want the flexibility of a lake with the governance of a warehouse.
• Both analytics and ML teams need to work from the same data.
• You need ACID transactions on large-scale data.
• You want to avoid vendor lock-in with open table formats.

The practical reality

Most organizations use a combination:

1. Data lake — raw landing zone where all source data arrives.
2. Lakehouse layer — Iceberg or Delta on top of the lake for curated, governed tables.
3. Data warehouse — Snowflake or BigQuery for the final analytics-ready layer consumed by BI tools.

The three architectures are not competitors. They are layers in a modern data platform.

Next steps

• Data Warehouse Concepts — star schemas and dimensional modeling.
• Batch vs Stream Processing — how data moves between these layers.
• What Is Apache Kafka? — real-time data streaming into lakes and warehouses.