DuckDB inside OLTP databases

DuckDB inside OLTP databases

A neutral comparison of database servers that embed DuckDB, against plain DuckDB at several versions and row-store baselines, on the same TPC-H data with the same container limits. There is no overall score and no ranking: read each measure with its failures and its n/a cells.

Marks on charts and tables: ✗ wrong result (never drawn as a bar), T/O timeout, E error, C crashed, ! failed to run, – n/a. Hover a mark for the reason.

Findings

Computed from the result records of this page, one sentence per measure, with every system in the same order. They state counts and ranges; they do not rank systems or give an overall verdict. Each finding links to the measure's charts.

Correct answers

11 of 13 systems answered all 22 queries with the reference result.

  • MySQL InnoDB: q01 timeout; q03 timeout; q05 timeout; q07 timeout; q08 timeout; q09 timeout; q10 timeout; q13 timeout; q15 timeout; q18 timeout; q21 timeout
  • PostgreSQL: q18 timeout; q20 timeout

Query time

On the 22 queries every integration answered correctly, the total warm time was 47.19 s to 1,481 s across all 6 integrations, and 28.20 s to 38.11 s for plain DuckDB.

  • AliSQL: 47.19 s
  • MariaDB: 49.17 s
  • Percona: 48.39 s
  • Percona (rebuilt on DuckDB 1.5.5): 55.28 s
  • pg_duckdb: 1,481 s
  • pg_ducklake: 73.53 s
  • DuckDB 1.4.3: 38.02 s
  • DuckDB 1.5.2-dev736: 38.11 s
  • DuckDB 1.5.3: 36.99 s
  • DuckDB 1.5.5: 36.92 s
  • DuckDB 2.0.0: 28.20 s
  • MySQL InnoDB: n/a (q01: timeout, q03: timeout, q05: timeout, q07: timeout, q08: timeout, q09: timeout, q10: timeout, q13: timeout, q15: timeout, q18: timeout, q21: timeout)
  • PostgreSQL: n/a (q18: timeout, q20: timeout)

Integration overhead

Against plain DuckDB at the same DuckDB version, the geometric-mean time ratio was 1.13x to 36.64x across all 6 integrations. 1x means no cost for running inside the server.

  • AliSQL: 1.13x against DuckDB 1.5.2-dev736
  • MariaDB: 1.21x against DuckDB 1.5.5
  • Percona: 1.49x against DuckDB 2.0.0
  • Percona (rebuilt on DuckDB 1.5.5): 1.34x against DuckDB 1.5.5
  • pg_duckdb: 36.64x against DuckDB 1.4.3
  • pg_ducklake: 1.92x against DuckDB 1.5.3

Concurrency

At 16 concurrent streams, throughput was 665 qph to 2,097 qph across 7 of 11 systems measured.

  • AliSQL: crashed
  • MariaDB: 750 qph
  • Percona: 1,442 qph
  • Percona (rebuilt on DuckDB 1.5.5): 1,427 qph
  • pg_duckdb: error
  • pg_ducklake: error
  • DuckDB 1.4.3: 665 qph
  • DuckDB 1.5.2-dev736: crashed
  • DuckDB 1.5.3: 800 qph
  • DuckDB 1.5.5: 830 qph
  • DuckDB 2.0.0: 2,097 qph
  • AliSQL, 8 streams: crashed (the container was killed by its memory limit during the run (OOMKilled, exit 137) (102 crashed, 7 timeout, 67 ok of 176 queries); no throughput is reported; first errors: [['q10', 'timeout', "OperationalError: (2013, 'Lost connection to MySQL server during query (The read operation timed out)')"], ['q17', 'crashed', "OperationalError: (2013, 'Lost connection to MySQL server during query')"], ['q15', 'crashed', 'StopIteration: ']])
  • AliSQL, 16 streams: crashed (the container was killed by its memory limit during the run (OOMKilled, exit 137) (323 crashed, 5 timeout, 24 ok of 352 queries); no throughput is reported; first errors: [['q21', 'timeout', "OperationalError: (2013, 'Lost connection to MySQL server during query (The read operation timed out)')"], ['q02', 'crashed', 'StopIteration: '], ['q06', 'crashed', 'StopIteration: ']])
  • AliSQL, 32 streams: crashed (the container was killed by its memory limit during the run (OOMKilled, exit 137) (642 crashed, 32 timeout, 30 ok of 704 queries); no throughput is reported; first errors: [['q21', 'timeout', "OperationalError: (2013, 'Lost connection to MySQL server during query (The read operation timed out)')"], ['q02', 'crashed', 'OperationalError: (2003, "Can\'t connect to MySQL server on \'172.17.0.3\' ([SSL: UNEXPECTED_EOF_WHILE_READING] EOF occurred in violation of protocol (_ssl.c:1011))")'], ['q06', 'crashed', 'StopIteration: ']])
  • MariaDB, 32 streams: 17 of 704 queries did not complete
  • pg_duckdb, 4 streams: 3 of 88 queries did not complete
  • pg_duckdb, 8 streams: error (queries failed during the run (15 error, 10 timeout, 151 ok of 176 queries); no throughput is reported; first errors: [['q21', 'error', 'InternalError_: (PGDuckDB/Duckdb_ExecCustomScan_Cpp) IO Error: Could not read enough bytes from file "/var/lib/postgresql/duckdb_tmp/duckdb_temp_storage_DEFAULT-0.tmp": attempted to read 262144 bytes from location 192937984'], ['q12', 'timeout', 'InternalError_: (PGDuckDB/Duckdb_ExecCustomScan_Cpp) Executor Error: Query cancelled'], ['q18', 'error', 'InternalError_: (PGDuckDB/Duckdb_ExecCustomScan_Cpp) Conversion Error: Casting value "-265845505630417986306734531451771944.96" to type DECIMAL(38,3) failed: value is out of range! when casting from source column sum(l_quantity)\n\nLINE 1: ....lineitem lineitem_1 GROUP BY lineitem_1.l_orderkey HAVING ']])
  • pg_duckdb, 16 streams: error (queries failed during the run (59 error, 79 timeout, 214 ok of 352 queries); no throughput is reported; first errors: [['q21', 'timeout', 'InternalError_: (PGDuckDB/Duckdb_ExecCustomScan_Cpp) Executor Error: Query cancelled'], ['q07', 'timeout', 'InternalError_: (PGDuckDB/Duckdb_ExecCustomScan_Cpp) Executor Error: Query cancelled'], ['q12', 'timeout', 'InternalError_: (PGDuckDB/Duckdb_ExecCustomScan_Cpp) Executor Error: Query cancelled']])
  • pg_duckdb, 32 streams: error (queries failed during the run (455 error, 162 timeout, 87 ok of 704 queries); no throughput is reported; first errors: [['q21', 'timeout', 'InternalError_: (PGDuckDB/Duckdb_ExecCustomScan_Cpp) Executor Error: Query cancelled'], ['q02', 'error', 'OperationalError: consuming input failed: server closed the connection unexpectedly\n\tThis probably means the server terminated abnormally\n\tbefore or while processing the request.'], ['q07', 'error', 'InternalError_: (PGDuckDB/Duckdb_ExecCustomScan_Cpp) IO Error: Could not read enough bytes from file "/var/lib/postgresql/duckdb_tmp/duckdb_temp_storage_S96K-0.tmp": attempted to read 98304 bytes from location 102531072']])
  • pg_ducklake, 8 streams: error (queries failed during the run (13 error, 1 timeout, 162 ok of 176 queries); no throughput is reported; first errors: [['q18', 'error', 'InternalError_: (PGDuckDB/Duckdb_ExecCustomScan_Cpp) IO Error: Could not read enough bytes from file "/var/lib/postgresql/duckdb_tmp/duckdb_temp_storage_S96K-0.tmp": attempted to read 98304 bytes from location 16220160'], ['q09', 'error', 'InternalError_: (PGDuckDB/Duckdb_ExecCustomScan_Cpp) IO Error: Could not read enough bytes from file "/var/lib/postgresql/duckdb_tmp/duckdb_temp_storage_S96K-0.tmp": attempted to read 98304 bytes from location 1474560'], ['q18', 'error', 'InternalError_: (PGDuckDB/Duckdb_ExecCustomScan_Cpp) IO Error: Could not read enough bytes from file "/var/lib/postgresql/duckdb_tmp/duckdb_temp_storage_S32K-0.tmp": attempted to read 32768 bytes from location 3670016']])
  • pg_ducklake, 16 streams: error (queries failed during the run (50 error, 1 timeout, 301 ok of 352 queries); no throughput is reported; first errors: [['q21', 'error', 'InternalError_: (PGDuckDB/Duckdb_ExecCustomScan_Cpp) IO Error: Could not read enough bytes from file "/var/lib/postgresql/duckdb_tmp/duckdb_temp_storage_S192K-0.tmp": attempted to read 196608 bytes from location 237502464'], ['q18', 'error', 'InternalError_: (PGDuckDB/Duckdb_ExecCustomScan_Cpp) Conversion Error: Casting value "1022467097382678256660134525695189319.68" to type DECIMAL(38,3) failed: value is out of range! when casting from source column sum(l_quantity)\n\nLINE 1: ....lineitem lineitem_1 GROUP BY lineitem_1.l_orderkey HAVING '], ['q13', 'error', 'InternalError_: (PGDuckDB/Duckdb_ExecCustomScan_Cpp) IO Error: Could not read enough bytes from file "/var/lib/postgresql/duckdb_tmp/duckdb_temp_storage_DEFAULT-0.tmp": attempted to read 262144 bytes from location 457179136']])
  • pg_ducklake, 32 streams: error (queries failed during the run (246 error, 458 ok of 704 queries); no throughput is reported; first errors: [['q21', 'error', 'InternalError_: (PGDuckDB/Duckdb_ExecCustomScan_Cpp) IO Error: Could not read enough bytes from file "/var/lib/postgresql/duckdb_tmp/duckdb_temp_storage_S192K-0.tmp": attempted to read 196608 bytes from location 54263808'], ['q12', 'error', 'OperationalError: consuming input failed: server closed the connection unexpectedly\n\tThis probably means the server terminated abnormally\n\tbefore or while processing the request.'], ['q16', 'error', 'OperationalError: consuming input failed: server closed the connection unexpectedly\n\tThis probably means the server terminated abnormally\n\tbefore or while processing the request.']])
  • DuckDB 1.4.3, 32 streams: crashed (the container was killed by its memory limit during the run (OOMKilled, exit 137) (219 crashed, 43 timeout, 442 ok of 704 queries); no throughput is reported; first errors: [['q21', 'timeout', 'InterruptException: INTERRUPT Error: Interrupted!'], ['q09', 'crashed', 'ConnectionError: server closed the connection'], ['q11', 'crashed', 'StopIteration: ']])
  • DuckDB 1.5.2-dev736, 16 streams: crashed (the container was killed by its memory limit during the run (OOMKilled, exit 137) (318 crashed, 34 ok of 352 queries); no throughput is reported; first errors: [['q07', 'crashed', 'ConnectionError: server closed the connection'], ['q22', 'crashed', 'StopIteration: '], ['q12', 'crashed', 'StopIteration: ']])
  • DuckDB 1.5.2-dev736, 32 streams: crashed (the container was killed by its memory limit during the run (OOMKilled, exit 137) (86 crashed, 551 error, 31 timeout, 36 ok of 704 queries); no throughput is reported; first errors: [['q21', 'timeout', 'no response from server within timeout + margin'], ['q02', 'error', 'TimeoutError: timed out'], ['q06', 'error', 'TimeoutError: timed out']])
  • DuckDB 1.5.3, 32 streams: crashed (the container was killed by its memory limit during the run (OOMKilled, exit 137) (123 crashed, 15 timeout, 566 ok of 704 queries); no throughput is reported; first errors: [['q09', 'timeout', 'InterruptException: INTERRUPT Error: Interrupted!'], ['q05', 'crashed', 'ConnectionError: server closed the connection'], ['q22', 'crashed', 'StopIteration: ']])
  • DuckDB 1.5.5, 32 streams: crashed (the container was killed by its memory limit during the run (OOMKilled, exit 137) (149 crashed, 18 timeout, 537 ok of 704 queries); no throughput is reported; first errors: [['q09', 'timeout', 'InterruptException: INTERRUPT Error: Interrupted!'], ['q17', 'crashed', 'ConnectionError: server closed the connection'], ['q09', 'timeout', 'InterruptException: INTERRUPT Error: Interrupted!']])
  • DuckDB 2.0.0, 32 streams: 3 of 704 queries did not complete

SQL compatibility

120 SQL statements an OLTP application would send, checked against the server's own row-store tables. A wrong result is silent: the statement succeeds and the answer differs.

  • AliSQL: 15 errors, 2 wrong results, 0 n/a
  • MariaDB: 10 errors, 4 wrong results, 3 n/a
  • Percona: 0 errors, 0 wrong results, 0 n/a
  • pg_duckdb: 0 errors, 0 wrong results, 7 n/a
  • pg_ducklake: 15 errors, 2 wrong results, 0 n/a
  • Wrong result, AliSQL: Generated primary key values for rows inserted without an id (dml_auto_increment)
  • Wrong result, AliSQL: Insert-or-update on a primary key conflict (dml_upsert)
  • Wrong result, MariaDB: Insert-or-update on a primary key conflict (dml_upsert)
  • Wrong result, MariaDB: String concatenation, NULL propagates (fn_concat)
  • Wrong result, MariaDB: EXISTS across the two (engine table with a row-store table) (join_mixed_exists)
  • Wrong result, MariaDB: The session time zone (UTC+2) decides how a timestamp converts to epoch seconds (var_session_time_zone)
  • Wrong result, pg_ducklake: TRUNCATE TABLE, then insert one row and count (ddl_truncate)
  • Wrong result, pg_ducklake: The session time zone (UTC+2) decides how a timestamp converts to epoch seconds (var_session_time_zone)

Silent slow paths

TPC-H queries seen on the row path: none on 5 of the 5 integrations that report it. Of the read probes, the share answered on the row path instead of in DuckDB was 0% to 86.7%.

  • AliSQL: TPC-H 0%, probes 0%
  • MariaDB: TPC-H 0%, probes 4.9%
  • Percona: TPC-H 0%, probes 86.7%
  • pg_duckdb: TPC-H 0%, probes 10.6%
  • pg_ducklake: TPC-H 0%, probes 0%

Loading

Bulk load of the TPC-H data took 125 s to 18,141 s across all 7 systems.

  • AliSQL: 6,235 s
  • MariaDB: 125 s
  • Percona: 1,999 s
  • pg_duckdb: 12,736 s
  • pg_ducklake: 195 s
  • MySQL InnoDB: 18,141 s
  • PostgreSQL: 12,537 s

Parquet entry points

4 of 4 systems answered all 22 queries with the reference result. On the 22 queries every entry point answered correctly, the total warm time was 71.26 s to 82.53 s across all 3 entry points, and 71.50 s for plain DuckDB.

  • MariaDB (run_in_duckdb): 82.53 s
  • pg_duckdb (duckdb.query): 71.78 s
  • VillageSQL: 71.26 s
  • DuckDB 1.5.5: 71.50 s

Headlines: tables stored in the database

Systems appear in the same order everywhere: database integrations A to Z, plain DuckDB, row-store baselines. Totals use only the queries every integration completed correctly, so everyone is scored on the same work. VillageSQL has no DuckDB table type, so its row here is n/a; its numbers are under Parquet entry points below.

Total query time, common queries (s)

Sum of warm medians over the queries that every integration ran correctly. Lower is better.

Throughput at 16 streams (queries per hour)

Concurrent TPC-H streams, warm cache. Higher is better.

Integration overhead vs plain DuckDB (geomean, common queries)

Ratio to plain DuckDB at the same version; the headline uses the same queries for every system. Each system's own geomean and its coverage are in the table below.

Headline numbers (SF100)

SystemDuckDBTotal, common queriesOwn total (ok/total) Throughput, 16 streams (qph)Overhead, commonOverhead, own geomean (ok/total)Plain baseline used
AliSQL 1.5.2-dev736 47.19 s 47.19 s (22/22) C 1.13x 1.13x (22/22)
MariaDB 1.5.5 49.17 s 49.17 s (22/22) 750 1.21x 1.21x (22/22)
Percona 2.0.0 48.39 s 48.39 s (22/22) 1,442 1.49x 1.49x (22/22)
Percona (rebuilt on DuckDB 1.5.5) 1.5.5 55.28 s 55.28 s (22/22) 1,427 1.34x 1.34x (22/22)
pg_duckdb 1.4.3 1,481 s 1,481 s (22/22) E 36.64x 36.64x (22/22)
pg_ducklake 1.5.3 73.53 s 73.53 s (22/22) E 1.92x 1.92x (22/22)
DuckDB 1.4.3 v1.4.3 38.02 s 38.02 s (22/22) 665 –
DuckDB 1.5.2-dev736 v1.5.2-dev736 38.11 s 38.11 s (22/22) C –
DuckDB 1.5.3 v1.5.3 36.99 s 36.99 s (22/22) 800 –
DuckDB 1.5.5 v1.5.5 36.92 s 36.92 s (22/22) 830 –
DuckDB 2.0.0 v2.0.0 28.20 s 28.20 s (22/22) 2,097 –
MySQL InnoDB n/a – 2,846 s (11/22) – –
PostgreSQL n/a – 1,662 s (20/22) – –
VillageSQL 1.5.5 – – – measured on Parquet files only; see Parquet entry points

Parquet entry points

A second workload on the same TPC-H data: DuckDB SQL over Parquet files, sent through each system's raw DuckDB entry point (VillageSQL duckdb_query(), pg_duckdb duckdb.query(), MariaDB run_in_duckdb()) and plain DuckDB. These numbers are not comparable with the table-based ones above and are never drawn on the same chart.

Total query time over Parquet, common queries (s)

Sum of warm medians over the queries every entry point ran correctly. Lower is better.

Details: explore the data, all charts, methodology and every setting.