Postgresql：創建表時，UNION ALL比UNION慢？

我有一個UNION查詢。在EXPLAIN輸出中，我注意到UNION總是在追加後進行排序。所以我把它改爲UNION ALL。Postgresql：創建表時，UNION ALL比UNION慢？

據說查詢應該更快？但實際執行表明，使用UNION ALL查詢總是在幾秒後返回。（也許我的數據集不夠大，差別看起來很小，但我仍然沒有意義。）我錯過了什麼？

QUERY

WITH wcal AS (...), wlog AS (...) 
SELECT * INTO temp.utlog FROM (
    SELECT * FROM wcal 
    UNION SELECT * FROM wlog 
) as t2;

EXPLAIN UNION

QUERY PLAN 
Unique (cost=691313.04..737080.66 rows=3051175 width=80) 
    CTE wcal 
    CTE wlog 
    -> Sort (cost=691313.04..698940.98 rows=3051175 width=80) 
     Sort Key: wcal.wts, wcal.wdate, wcal.wstate, wcal.wln, wcal.wid 
     -> Append (cost=0.00..91535.25 rows=3051175 width=80) 
       -> CTE Scan on wcal (cost=0.00..52600.00 rows=2630000 width=60) 
       -> Subquery Scan on "*SELECT* 2" (cost=0.00..12635.25 rows=421175 width=64) 
        -> CTE Scan on wlog (cost=0.00..8423.50 rows=421175 width=64)

EXPLAIN UNION ALL

QUERY PLAN 
Append (cost=0.00..91535.25 rows=3051175 width=80) 
    CTE wcal ... 
    CTE wlog ... 
    -> CTE Scan on wcal (cost=0.00..52600.00 rows=2630000 width=60) 
    -> Subquery Scan on "*SELECT* 2" (cost=0.00..12635.25 rows=421175 width=64) 
     -> CTE Scan on wlog (cost=0.00..8423.50 rows=421175 width=64)

實際結果;

與UNION

Table temp.utlog dropped 

Execution time: 0.02s 
Statement 1 of 5 finished 

0 rows affected 
WITH executed successfully 

Execution time: 6.05s 
Statement 2 of 5 finished

與UNION ALL

Table temp.utlog1 dropped 

Execution time: 0.02s 
Statement 1 of 5 finished 

0 rows affected 
WITH executed successfully 

Execution time: 6.81s 
Statement 2 of 5 finished

在新表

結果計數是一樣的...

SELECT count(*) FROM temp.utlog1 
UNION ALL 
SELECT count(*) FROM temp.utlog 

count 
364414 
364414

EXPLAIN ANALYZEUNION ALL

QUERY PLAN 
Append (cost=0.00..11658.36 rows=388612 width=80) (actual time=0.670..3382.220 rows=364414 loops=1) 
    CTE wcal ... 
    CTE wlog ... 
    -> CTE Scan on wcal (cost=0.00..720.00 rows=36000 width=60) (actual time=0.668..140.251 rows=33624 loops=1) 
    -> Subquery Scan on "*SELECT* 2" (cost=0.00..10578.36 rows=352612 width=64) (actual time=2635.901..3179.006 rows=330790 loops=1) 
     -> CTE Scan on wlog (cost=0.00..7052.24 rows=352612 width=64) (actual time=2635.897..3043.421 rows=330790 loops=1) 
Planning time: 1.759 ms 
Execution time: 6570.793 ms 
... 
Planning time: 1.713 ms 
Execution time: 5883.275 ms 
... 
Planning time: 1.351 ms 
Execution time: 7092.713 ms 
... 
Planning time: 1.318 ms 
Execution time: 5803.567 ms

EXPLAIN ANALYZEUNION

QUERY PLAN 
Unique (cost=65006.04..70835.22 rows=388612 width=80) (actual time=4259.243..4534.226 rows=364414 loops=1) 
    CTE wcal ... 
    CTE wlog ... 
    -> Sort (cost=65006.04..65977.57 rows=388612 width=80) (actual time=4259.239..4370.691 rows=364414 loops=1) 
     Sort Key: wcal.wts, wcal.wdate, wcal.wstate, wcal.wln, wcal.wid 
     Sort Method: external sort Disk: 11832kB 
     -> Append (cost=0.00..11658.36 rows=388612 width=80) (actual time=0.677..3486.529 rows=364414 loops=1) 
       -> CTE Scan on wcal (cost=0.00..720.00 rows=36000 width=60) (actual time=0.676..120.320 rows=33624 loops=1) 
       -> Subquery Scan on "*SELECT* 2" (cost=0.00..10578.36 rows=352612 width=64) (actual time=2760.279..3309.084 rows=330790 loops=1) 
        -> CTE Scan on wlog (cost=0.00..7052.24 rows=352612 width=64) (actual time=2760.275..3189.444 rows=330790 loops=1) 
Planning time: 1.500 ms 
Execution time: 7577.380 ms 
... 
Planning time: 1.375 ms 
Execution time: 6777.699 ms 
... 
Planning time: 1.375 ms 
Execution time: 6777.699 ms 
... 
Planning time: 1.340 ms 
Execution time: 6964.954 ms

UNION（0._s）和UNION ALL（4。_s），而不SELECT INTO

WITH executed successfully 

Execution time: 0.41s 
Statement 1 of 4 finished 

WITH executed successfully 

Execution time: 0.28s 
Statement 1 of 4 finished 

WITH executed successfully 

Execution time: 0.33s 
Statement 1 of 4 finished 

WITH executed successfully 

Execution time: 0.25s 
Statement 1 of 4 finished 

WITH executed successfully 

Execution time: 0.28s 
Statement 1 of 5 finished 

WITH executed successfully 

Execution time: 4.67s 
Statement 1 of 5 finished 

WITH executed successfully 

Execution time: 4.17s 
Statement 1 of 5 finished 

WITH executed successfully 

Execution time: 4.17s 
Statement 1 of 5 finished 

WITH executed successfully 

Execution time: 4.33s 
Statement 1 of 5 finished

UNION而不CTE

QUERY PLAN 
Unique (cost=699529.52..752925.08 rows=3051175 width=65) (actual time=3041.962..3488.246 rows=364414 loops=1) 
    -> Sort (cost=699529.52..707157.46 rows=3051175 width=65) (actual time=3041.958..3296.505 rows=364414 loops=1) 
     Sort Key: s1.s1, ((date_trunc('day'::text, s1.s1))::date), (NULL::character varying(3)), s2.wline, (NULL::boolean), (NULL::integer) 
     Sort Method: external merge Disk: 12544kB 
     -> Append (cost=0.00..120611.73 rows=3051175 width=65) (actual time=0.729..2218.973 rows=364414 loops=1) 
       -> Nested Loop (cost=0.00..46077.88 rows=2630000 width=36) (actual time=0.727..94.359 rows=33624 loops=1) 
       ... 
       -> WindowAgg (cost=198.08..44022.11 rows=421175 width=24) (actual time=0.737..2049.979 rows=330790 loops=1) 
       ... 

Planning time: 1.318 ms 
Execution time: 6438.852 ms

UNION與CTE

QUERY PLAN 
Unique (cost=670453.04..723848.60 rows=3051175 width=65) (actual time=3726.279..3973.870 rows=364414 loops=1) 
    CTE wcal 
    -> Nested Loop (cost=0.00..46077.88 rows=2630000 width=36) (actual time=1.788..82.337 rows=33624 loops=1) 
    ... 
    CTE wlog 
    -> Sort (cost=92006.40..93059.34 rows=421175 width=24) (actual time=2542.472..2743.255 rows=330790 loops=1) 
      Sort Key: tbl_line_evts.evt_time 
      Sort Method: external merge Disk: 11712kB 
      -> WindowAgg (cost=198.08..44022.11 rows=421175 width=24) (actual time=0.932..1982.497 rows=330790 loops=1) 
      ... 
    -> Sort (cost=670453.04..678080.98 rows=3051175 width=65) (actual time=3726.275..3823.236 rows=364414 loops=1) 
     Sort Key: wcal.wts, wcal.wdate, wcal.wstate, wcal.wln, wcal.wdirty, wcal.wid 
     Sort Method: external sort Disk: 12584kB 
     -> Append (cost=0.00..91535.25 rows=3051175 width=65) (actual time=1.800..3082.954 rows=364414 loops=1) 
       -> CTE Scan on wcal (cost=0.00..52600.00 rows=2630000 width=57) (actual time=1.798..125.699 rows=33624 loops=1) 
       -> CTE Scan on wlog (cost=0.00..8423.50 rows=421175 width=49) (actual time=2542.476..2915.524 rows=330790 loops=1) 
Planning time: 2.381 ms 
Execution time: 5950.324 ms

UNION ALL沒有CTE

QUERY PLAN 
Sort (cost=721442.46..729070.40 rows=3051175 width=80) (actual time=3416.176..3627.253 rows=364414 loops=1) 
    Sort Key: s1.s1 
    Sort Method: external merge Disk: 11816kB 
    -> Append (cost=0.00..121664.67 rows=3051175 width=80) (actual time=0.675..2726.980 rows=364414 loops=1) 
     -> Nested Loop (cost=0.00..46077.88 rows=2630000 width=39) (actual time=0.674..118.694 rows=33624 loops=1) 
     ... 
     -> Subquery Scan on "*SELECT* 2" (cost=198.08..49286.79 rows=421175 width=51) (actual time=1.223..2517.292 rows=330790 loops=1) 
       -> WindowAgg (cost=198.08..45075.04 rows=421175 width=51) (actual time=1.218..2334.702 rows=330790 loops=1) 
       ... 
Planning time: 1.360 ms 
Execution time: 5795.449 ms

UNION ALL與CTE

QUERY PLAN 
Append (cost=0.00..91535.25 rows=3051175 width=80) (actual time=0.718..3595.377 rows=364414 loops=1) 
    CTE wcal 
    -> Nested Loop (cost=0.00..46077.88 rows=2630000 width=39) (actual time=0.709..86.700 rows=33624 loops=1) 
    ... 
    CTE wlog 
    -> Sort (cost=98820.34..99873.28 rows=421175 width=51) (actual time=2818.220..3049.432 rows=330790 loops=1) 
      Sort Key: tbl_line_evts.evt_time 
      Sort Method: external merge Disk: 10952kB 
      -> WindowAgg (cost=198.08..45075.04 rows=421175 width=51) (actual time=0.884..2224.477 rows=330790 loops=1) 
      ... 
    -> CTE Scan on wcal (cost=0.00..52600.00 rows=2630000 width=60) (actual time=0.716..129.933 rows=33624 loops=1) 
    -> Subquery Scan on "*SELECT* 2" (cost=0.00..12635.25 rows=421175 width=64) (actual time=2818.229..3400.468 rows=330790 loops=1) 
     -> CTE Scan on wlog (cost=0.00..8423.50 rows=421175 width=64) (actual time=2818.224..3250.670 rows=330790 loops=1) 
Planning time: 2.415 ms 
Execution time: 6223.041 ms

來源

2017-04-09 Ben

'UNION'是否減小返回集的大小？即，是否有重複？此外，該排序可能會影響INSERT期間的索引構建。 – jcaron

否定的，他們產生了相同數量的記錄。你可以詳細說明索引嗎？ – Ben

將數據插入索引時，如果按索引順序插入，而不是由於I/O（尤其是在旋轉光盤（而不是SSD）上）的不同順序或隨機順序，則通常會更快。在沒有插入的查詢時間中，Ines是UNION，哪些是UNION ALL？ – jcaron

你的假設調和docs

UNION ......從它的結果消除了重複的行，以同樣的方式作爲 DISTINCT，除非使用UNION ALL 。

（......）我的

如果您選中這兩個查詢的費用，你會發現UNION ALL價格要便宜得多。 91K vs 737K。現在來解釋一下實際結果的差異，我假設這是io緩存和寫入操作的結果，嘗試測量時間而不創建表格，但只是運行10解釋分析與UNION，然後與UNION ALL 10次解釋。應該不那麼令人驚訝。

來源

2017-04-09 08:31:12

我更新了結果。你是對的，沒有創建表，差別很大，用ANALYZE，結果不穩定。但沒有ANALYZE，有線結果仍然是可重複的。那麼我應該學什麼？創建io重型桌子時，我應該更喜歡'UNION'到'UNION ALL'嗎？ – Ben

'explain analyze'實際上執行查詢，而只是'explain'估計它。當你想要一個沒有「DISTINCT」的快速聯合時，你應該使用'union all';如果你想要慢得多但只有不同的行，則應該使用'union'... –

謝謝。我想我明白了。在我的情況下，如果在緩存中存在任何低效率，'union all'返回比'union'快14倍的結果，1s的差異並不是那麼糟糕。「分析」引入了更多的延遲/中斷，可能會減輕強度，所以多次查看結果比單純查詢更好，但它不穩定。 – Ben

Postgresql：創建表時，UNION ALL比UNION慢？

回答

相關問題