
1.在DuckDB中生成tcph数据和查询语句并导出schema脚本和csv数据。rootDESKTOP-59T6U68:/mnt/c/d# ./duckdb140 DuckDB v1.4.0 (Andium) b8a06e4a22 Enter .help for usage hints. Connected to a transient in-memory database. Use .open FILENAME to reopen on a persistent database. D load tpch; D call dbgen(sf0.1); ┌─────────┐ │ Success │ │ boolean │ ├─────────┤ │ 0 rows │ └─────────┘ D export database tpch1; D .mode list D select query FROM tpch_queries() limit 2; query SELECT l_returnflag, l_linestatus, sum(l_quantity) AS sum_qty, ... FROM lineitem WHERE l_shipdate CAST(1998-09-02 AS date) GROUP BY l_returnflag, l_linestatus ORDER BY l_returnflag, l_linestatus; SELECT s_acctbal, s_name, n_name, p_partkey, p_mfgr, s_address, s_phone, s_comment FROM part, supplier, ... AND r_name EUROPE) ORDER BY s_acctbal DESC, n_name, s_name, p_partkey LIMIT 100;用如下命令把所有查询语句保存到文件。D .header off D .mode list D .output q.sql D select query from tpch_queries(); D .output D .exit2.在umbra-db中执行schema脚本用copy命令复制csv文件中的数据。注意不能用load.sql里的语法umbra-db不能识别。rootDESKTOP-59T6U68:/mnt/c/Users/lt# docker start umbra26 umbra26 rootDESKTOP-59T6U68:/mnt/c/Users/lt# docker exec -it umbra26 bash umbraDESKTOP-59T6U68:/var/db$ umbra-sql WARNING: Could not extract threads for numa node 0! \i /par/tpch1/schema.sql \d List of relations Schema | Name | Type | Owner ----------------------------------- public | nation | table | postgres public | orders | table | postgres public | part | table | postgres public | lineitem | table | postgres public | region | table | postgres public | partsupp | table | postgres public | customer | table | postgres public | supplier | table | postgres COPY nation FROM /par/tpch1/nation.csv (FORMAT csv, force_not_null (n_nationkey, n_name, n_regionkey, n_comment), quote , delimiter ,, header 1); ERROR: invalid number format for integer: no digits found in n_nationkey COPY nation FROM /par/tpch1/nation.csv with csv header; COPY lineitem FROM /par/tpch1/lineitem.csv with csv header; COPY orders FROM /par/tpch1/orders.csv with csv header; COPY part FROM /par/tpch1/part.csv with csv header; COPY partsupp FROM /par/tpch1/partsupp.csv with csv header; COPY customer FROM /par/tpch1/customer.csv with csv header; COPY region FROM /par/tpch1/region.csv with csv header;执行第二个tpch查询的计划居然是0行实际查询也是0行。原来是忘记复制supplier表。 explain SELECT s_acctbal, s_name, n_name, p_partkey, p_mfgr, s_address, s_phone, s_comment FROM part, supplier, partsupp, nation, region WHERE p_partkey ps_partkey ... AND r_name EUROPE) ORDER BY s_acctbal DESC, n_name, s_name, p_partkey LIMIT 100; ┌─────────────┐ │ inlinetable │ │ card 0 │ └─────────────┘ INFO: [s] execution: (0.000028 min, 0.000028 max, 0.000028 median, 0.0% relMAD, 0.000028 avg, 0.000000 sdev, 2 scale, nan IPC, nan CPUs, nan GHz) compilation: (0.010586 min, 0.010586 max, 0.010586 median, 0.0% relMAD, 0.010586 avg, 0.000000 sdev) SELECT s_acctbal, s_name, n_name, p_partkey, p_mfgr, s_address, s_phone, s_comment FROM part, ... AND r_name EUROPE) ORDER BY s_acctbal DESC, n_name, s_name, p_partkey LIMIT 10; s_acctbal s_name n_name p_partkey p_mfgr s_address s_phone s_comment INFO: [s] execution: (0.022432 min, 0.022432 max, 0.022432 median, 0.0% relMAD, 0.022432 avg, 0.000000 sdev, 1 scale, nan IPC, nan CPUs, nan GHz) compilation: (0.003209 min, 0.003209 max, 0.003209 median, 0.0% relMAD, 0.003209 avg, 0.000000 sdev)复制supplier表以后就正常输出查询结果了也能显示复杂的执行计划。 COPY supplier FROM /par/tpch1/supplier.csv with csv header; INFO: [s] execution: (0.008510 min, 0.008510 max, 0.008510 median, 0.0% relMAD, 0.008510 avg, 0.000000 sdev, 1 scale, nan IPC, nan CPUs, nan GHz) compilation: (0.006464 min, 0.006464 max, 0.006464 median, 0.0% relMAD, 0.006464 avg, 0.000000 sdev) SELECT s_acctbal, s_name, n_name, p_partkey, p_mfgr, s_address, s_phone, s_comment FROM part, supplier, partsupp, nation, region WHERE p_partkey ps_partkey AND s_suppkey ps_suppkey AND p_size 15 AND p_type LIKE %BRASS AND s_nationkey n_nationkey AND n_regionkey r_regionkey AND r_name EUROPE AND ps_supplycost ( SELECT min(ps_supplycost) FROM partsupp, supplier, nation, region WHERE p_partkey ps_partkey AND s_suppkey ps_suppkey AND s_nationkey n_nationkey AND n_regionkey r_regionkey AND r_name EUROPE) ORDER BY s_acctbal DESC, n_name, s_name, p_partkey LIMIT 10; s_acctbal s_name n_name p_partkey p_mfgr s_address s_phone s_comment 9828.21 Supplier#000000647 UNITED KINGDOM 13120 Manufacturer#5 vV6Teq1EvLlR 33-258-202-4782 mong the carefully quiet accounts slee ... 8615.50 Supplier#000000812 FRANCE 13811 Manufacturer#4 TAJWyNst8OGVPINgqtzwyyp002iYNDVub 16-585-724-6633 ress ideas eat quickly. blithely express deposits was slyly. final, INFO: [s] execution: (0.003217 min, 0.003217 max, 0.003217 median, 0.0% relMAD, 0.003217 avg, 0.000000 sdev, 182001 scale, nan IPC, nan CPUs, nan GHz) compilation: (0.010193 min, 0.010193 max, 0.010193 median, 0.0% relMAD, 0.010193 avg, 0.000000 sdev) explain SELECT s_acctbal, s_name, n_name, p_partkey, p_mfgr, s_address, s_phone, s_comment FROM part, supplier, partsupp, nation, region WHERE p_partkey ps_partkey AND s_suppkey ps_suppkey AND p_size 15 AND p_type LIKE %BRASS AND s_nationkey n_nationkey AND n_regionkey r_regionkey AND r_name EUROPE AND ps_supplycost ( SELECT min(ps_supplycost) FROM partsupp, supplier, nation, region WHERE p_partkey ps_partkey AND s_suppkey ps_suppkey AND s_nationkey n_nationkey AND n_regionkey r_regionkey AND r_name EUROPE) ORDER BY s_acctbal DESC, n_name, s_name, p_partkey LIMIT 100; ┌────────┐ │ sort │ │ card 7 │ └────┬───┘ │ ┌──────┴──────┐ │ join (hash) │ │ card 7 │ └──────┬──────┘ │ ┌────┴──────────────────────────┐ │ │ ┌──────┴──────┐ ┌─────┴────┐ │ join (hash) │ │ map │ │ card 34 │ │ card 204 │ └──────┬──────┘ └─────┬────┘ │ │ ┌─────┴────────────────────┐ ┌─────┴─────┐ │ │ │ tablescan │ ┌──────┴──────┐ ┌──────┴────│─supplier │ │ join (hash) │ │ tablescan │ card 204 │ │ card 19 │ │ partsupp └───────────┘ └──────┬──────┘ │ card 80000 │ │ └─────────────┘ ┌─────────┴──────────┐ │ │ ┌─────┴─────┐ ┌────┴────┐ │ tablescan │ │ groupby │ │ part │ │ card 19 │ │ card 39 │ └────┬────┘ └───────────┘ │ ┌──────┴──────┐ │ join (hash) │ │ card 21 │ └──────┬──────┘ │ ┌────┴──────────┐ │ │ ┌─────┴─────┐ ┌─────┴─────┐ │ tablescan │ │ tablescan │ │ supplier │ │ partsupp │ │ card 204 │ │ card 109 │ └───────────┘ └───────────┘ INFO: [s] execution: (0.000079 min, 0.000079 max, 0.000079 median, 0.0% relMAD, 0.000079 avg, 0.000000 sdev, 2 scale, nan IPC, nan CPUs, nan GHz) compilation: (0.003266 min, 0.003266 max, 0.003266 median, 0.0% relMAD, 0.003266 avg, 0.000000 sdev)3.umbra-db和duckdb比较在刚才DuckDB输出的查询脚本开头添加以下行.read /par/tpch1/schema.sql -- umbra-db用\i /par/tpch1/schema.sql .timer on -- umbra-db不用这一行 COPY nation FROM /par/tpch1/nation.csv with csv header; COPY lineitem FROM /par/tpch1/lineitem.csv with csv header; COPY customer FROM /par/tpch1/customer.csv with csv header; COPY orders FROM /par/tpch1/orders.csv with csv header; COPY part FROM /par/tpch1/part.csv with csv header; COPY partsupp FROM /par/tpch1/partsupp.csv with csv header; COPY supplier FROM /par/tpch1/supplier.csv with csv header; COPY region FROM /par/tpch1/region.csv with csv header;然后执行./duckdb /par/q.sql /tmp/dkresult.txtduckdb的计时信息被重定向到了文件整理如下umbrakylin-pc:/par$ cat /tmp/dkresult.txt|grep Run Run Time (s): real 0.016 user 0.007849 sys 0.005164 --copy Run Time (s): real 2.686 user 18.787420 sys 1.452828 Run Time (s): real 0.126 user 0.213403 sys 0.040328 Run Time (s): real 0.638 user 3.076135 sys 0.211725 Run Time (s): real 0.138 user 0.287571 sys 0.008605 Run Time (s): real 0.273 user 0.882975 sys 0.139947 Run Time (s): real 0.033 user 0.032843 sys 0.000077 Run Time (s): real 0.003 user 0.003218 sys 0.000257 Run Time (s): real 0.079 user 0.588044 sys 0.007882 --query Run Time (s): real 0.024 user 0.073599 sys 0.003260 Run Time (s): real 0.030 user 0.160817 sys 0.006042 Run Time (s): real 0.033 user 0.187327 sys 0.011040 Run Time (s): real 0.030 user 0.171901 sys 0.005003 Run Time (s): real 0.008 user 0.062233 sys 0.001421 Run Time (s): real 0.035 user 0.215051 sys 0.007385 Run Time (s): real 0.030 user 0.130061 sys 0.000241 Run Time (s): real 0.100 user 0.675368 sys 0.034883 Run Time (s): real 0.066 user 0.400028 sys 0.010151 Run Time (s): real 0.012 user 0.044323 sys 0.003946 Run Time (s): real 0.032 user 0.277101 sys 0.001190 Run Time (s): real 0.096 user 0.854122 sys 0.023407 Run Time (s): real 0.019 user 0.105275 sys 0.003594 Run Time (s): real 0.016 user 0.121632 sys 0.003646 Run Time (s): real 0.036 user 0.161581 sys 0.001646 Run Time (s): real 0.018 user 0.111958 sys 0.001223 Run Time (s): real 0.092 user 0.829879 sys 0.034265 Run Time (s): real 0.058 user 0.432020 sys 0.000135 Run Time (s): real 0.025 user 0.170460 sys 0.008070 Run Time (s): real 0.097 user 0.637837 sys 0.020801 Run Time (s): real 0.030 user 0.083713 sys 0.005913再执行umbra-sql /par/q.sql /tmp/umresult.txt计时信息直接输出到控制台应该是用标准错误stderr输出的。将它保存到文件然后用如下命令截取时间。cut -d -f 7 umb.txt (0.004464 --create table (0.000146 (0.000053 (0.000080 (0.000125 (0.000052 (0.000035 (0.000110 (0.001303 --copy (18.919193 (0.639593 (4.903765 (0.916798 (1.817959 (0.015532 (0.000299 (0.039141 --query (0.004482 (0.032046 (0.008398 (0.016439 (0.002618 (0.009787 (0.025687 (0.061944 (0.020215 (0.003550 (0.015802 (0.057290 (0.004152 (0.006184 (0.017023 (0.012815 (0.053271 (0.003930 (0.003874 (0.033213 (0.010367比较发现duckdb的copy启用了并行速度更快导入最大的lineitem.csv只用了2秒而查询则基本上都是umbra-db更快。