1. 首頁
  2. 問答
  3. Postgresql - LEFT OUTER JOIN性能問題

Postgresql - LEFT OUTER JOIN性能問題

2016-06-07 247 views
1

這是關於postgres實際工作方式的更多理論問題。我有四張表，分別是A，B，C和D，分別爲20k，870k，770k和1.5mln。 ID是主鍵，正在對外鍵進行連接（默認爲索引，不允許爲空）。這裏是我的查詢：Postgresql - LEFT OUTER JOIN性能問題

SELECT "A"."id" AS "a", 
    COUNT("B"."id") AS "b_count", 
    COUNT("C"."id") AS "c_count", 
    COUNT("D"."id") AS "d_count" 
FROM "A" 
LEFT OUTER JOIN "B" ON ("A"."id" = "B"."A_id") 
LEFT OUTER JOIN "C" ON ("B"."id" = "C"."B_id") 
LEFT OUTER JOIN "D" ON ("C"."id" = "D"."C_id") 
GROUP BY "A"."id"

表和查詢是由django框架生成的。由於不知道的原因，它運行了很長時間，服務器崩潰了。但是，如果我將這些LEFT OUTER JOIN中的任何一個更改爲INNER JOIN，則查詢運行時間爲4.2秒。我知道LEFT和INNER JOIN在實踐中是如何工作的，但爲什麼在這種情況下性能差別如此之大？

附加信息： 解釋查詢使用LEFT OUTER JOIN（EXPLAIN（分析一下，緩衝區）不起作用，因爲查詢沒有結束）：

"GroupAggregate (cost=606144.91..628195.81 rows=20784 width=16)" 
" -> Sort (cost=606144.91..610513.52 rows=1747445 width=16)" 
"  Sort Key: A.id" 
"  -> Hash Right Join (cost=282896.74..365231.69 rows=1747445 width=16)" 
"    Hash Cond: (D.C_id = C.id)" 
"    -> Seq Scan on D (cost=0.00..32113.94 rows=1494094 width=8)" 
"    -> Hash (cost=267717.03..267717.03 rows=873257 width=12)" 
"     -> Hash Right Join (cost=208924.92..267717.03 rows=873257 width=12)" 
"       Hash Cond: (B.A_id = A.id)" 
"       -> Hash Right Join (cost=207935.28..250353.82 rows=873257 width=12)" 
"        Hash Cond: (C.B_id = B.id)" 
"        -> Seq Scan on C (cost=0.00..21039.49 rows=746649 width=8)" 
"        -> Hash (cost=193607.57..193607.57 rows=873257 width=8)" 
"          -> Seq Scan on B (cost=0.00..193607.57 rows=873257 width=8)" 
"       -> Hash (cost=729.84..729.84 rows=20784 width=4)" 
"        -> Seq Scan on A (cost=0.00..729.84 rows=20784 width=4)"

EXPLAIN（分析一下，緩衝區）查詢一個INNER JOIN代替LEFT OUTER JOIN的：

"GroupAggregate (cost=559935.67..578819.69 rows=20784 width=16) (actual time=4565.338..5090.632 rows=19567 loops=1)" 
" Buffers: shared hit=10625 read=205521, temp read=27640 written=27388" 
" -> Sort (cost=559935.67..563670.91 rows=1494094 width=16) (actual time=4565.244..4832.596 rows=1494094 loops=1)" 
"  Sort Key: A.id" 
"  Sort Method: external merge Disk: 37992kB" 
"  Buffers: shared hit=10625 read=205521, temp read=27640 written=27388" 
"  -> Hash Join (cost=278322.24..355638.06 rows=1494094 width=16) (actual time=2274.363..3341.921 rows=1494094 loops=1)" 
"    Hash Cond: (D.C_id = C.id)" 
"    Buffers: shared hit=10622 read=205521, temp read=13681 written=13429" 
"    -> Seq Scan on D (cost=0.00..32113.94 rows=1494094 width=8) (actual time=0.007..270.841 rows=1494094 loops=1)" 
"     Buffers: shared hit=3828 read=13345" 
"    -> Hash (cost=265343.13..265343.13 rows=746649 width=12) (actual time=2271.959..2271.959 rows=746649 loops=1)" 
"     Buckets: 4096 Batches: 64 Memory Usage: 512kB" 
"     Buffers: shared hit=6794 read=192176, temp read=5640 written=8351" 
"     -> Hash Join (cost=208924.92..265343.13 rows=746649 width=12) (actual time=1107.516..2138.249 rows=746649 loops=1)" 
"       Hash Cond: (B.A_id = A.id)" 
"       Buffers: shared hit=6794 read=192176, temp read=5640 written=5514" 
"       -> Hash Join (cost=207935.28..250353.82 rows=746649 width=12) (actual time=1099.799..1784.403 rows=746649 loops=1)" 
"        Hash Cond: (C.B_id = B.id)" 
"        Buffers: shared hit=6272 read=192176, temp read=5640 written=5514" 
"        -> Seq Scan on C (cost=0.00..21039.49 rows=746649 width=8) (actual time=0.005..203.923 rows=746649 loops=1)" 
"          Buffers: shared hit=4600 read=8973" 
"        -> Hash (cost=193607.57..193607.57 rows=873257 width=8) (actual time=1095.407..1095.407 rows=873453 loops=1)" 
"          Buckets: 4096 Batches: 64 Memory Usage: 547kB" 
"          Buffers: shared hit=1672 read=183203, temp written=2907" 
"          -> Seq Scan on B (cost=0.00..193607.57 rows=873257 width=8) (actual time=0.004..939.839 rows=873453 loops=1)" 
"           Buffers: shared hit=1672 read=183203" 
"       -> Hash (cost=729.84..729.84 rows=20784 width=4) (actual time=7.701..7.701 rows=20784 loops=1)" 
"        Buckets: 4096 Batches: 1 Memory Usage: 731kB" 
"        Buffers: shared hit=522" 
"        -> Seq Scan on A (cost=0.00..729.84 rows=20784 width=4) (actual time=0.004..4.661 rows=20784 loops=1)" 
"          Buffers: shared hit=522" 
"Total runtime: 5099.046 ms"

和獎金：如果我用稍微修改查詢：

SELECT "A"."id" AS "a", 
    COUNT("B"."id") AS "b_count", 
    COUNT("C"."id") AS "c_count", 
    COUNT("D"."id") AS "d_count" 
FROM "A" 
LEFT OUTER JOIN "B" ON ("A"."id" = "B"."A_id") 
LEFT OUTER JOIN "C" ON ("B"."id" = "C"."B_id") 
LEFT OUTER JOIN "D" ON ("C"."id" = "D"."D_id") 
WHERE "D"."id" < 1500000 -- this is always true 
GROUP BY "A"."id"

然後它也可以工作，但比使用至少一個INNER JOIN長一點點。這種情況的解釋（分析，緩衝區）：

"GroupAggregate (cost=563670.91..582554.92 rows=20784 width=16) (actual time=6779.121..7286.640 rows=19567 loops=1)" 
" Buffers: shared hit=10814 read=205329, temp read=27640 written=27388" 
" -> Sort (cost=563670.91..567406.14 rows=1494094 width=16) (actual time=6777.606..7033.885 rows=1494094 loops=1)" 
"  Sort Key: A.id" 
"  Sort Method: external merge Disk: 37992kB" 
"  Buffers: shared hit=10814 read=205329, temp read=27640 written=27388" 
"  -> Hash Join (cost=278322.24..359373.29 rows=1494094 width=16) (actual time=4601.432..5674.321 rows=1494094 loops=1)" 
"    Hash Cond: (D.C_id = C.id)" 
"    Buffers: shared hit=10814 read=205329, temp read=13681 written=13429" 
"    -> Seq Scan on D (cost=0.00..35849.18 rows=1494094 width=8) (actual time=0.005..374.660 rows=1494094 loops=1)" 
"     Filter: (id < 1500000)" 
"     Buffers: shared hit=3892 read=13281" 
"    -> Hash (cost=265343.13..265343.13 rows=746649 width=12) (actual time=4600.782..4600.782 rows=746649 loops=1)" 
"     Buckets: 4096 Batches: 64 Memory Usage: 512kB" 
"     Buffers: shared hit=6922 read=192048, temp read=5640 written=8351" 
"     -> Hash Join (cost=208924.92..265343.13 rows=746649 width=12) (actual time=3363.352..4469.474 rows=746649 loops=1)" 
"       Hash Cond: (B.A_id = A.id)" 
"       Buffers: shared hit=6922 read=192048, temp read=5640 written=5514" 
"       -> Hash Join (cost=207935.28..250353.82 rows=746649 width=12) (actual time=3257.869..4066.067 rows=746649 loops=1)" 
"        Hash Cond: (C.B_id = B.id)" 
"        Buffers: shared hit=6400 read=192048, temp read=5640 written=5514" 
"        -> Seq Scan on C (cost=0.00..21039.49 rows=746649 width=8) (actual time=0.006..372.317 rows=746649 loops=1)" 
"          Buffers: shared hit=4664 read=8909" 
"        -> Hash (cost=193607.57..193607.57 rows=873257 width=8) (actual time=3257.327..3257.327 rows=873453 loops=1)" 
"          Buckets: 4096 Batches: 64 Memory Usage: 547kB" 
"          Buffers: shared hit=1736 read=183139, temp written=2907" 
"          -> Seq Scan on B (cost=0.00..193607.57 rows=873257 width=8) (actual time=0.004..3097.367 rows=873453 loops=1)" 
"           Buffers: shared hit=1736 read=183139" 
"       -> Hash (cost=729.84..729.84 rows=20784 width=4) (actual time=105.467..105.467 rows=20784 loops=1)" 
"        Buckets: 4096 Batches: 1 Memory Usage: 731kB" 
"        Buffers: shared hit=522" 
"        -> Seq Scan on A (cost=0.00..729.84 rows=20784 width=4) (actual time=0.002..101.506 rows=20784 loops=1)" 
"          Buffers: shared hit=522" 
"Total runtime: 7294.388 ms"

爲什麼原始查詢中斷postgres，而其他兩個工作像一個魅力？

編輯： 即使沒有count（）子句，所有查詢的行爲都是相同的。問題僅在於聯接

來源

2016-06-07 Damian Malarczyk

+3

http://wiki.postgresql.org/wiki/SlowQueryQuestions –

+0

謝謝，我增加了更多的信息，以我的問題 –

回答

0

因爲LEFT JOIN可以將結果集乘以很多，具體取決於表內容。

首先，你的查詢似乎是錯誤的，你也算重複，所以你應該使用COUNT(DISTINCT columnName)

SELECT "A"."id" AS "a", 
    COUNT(distinct "B"."id") AS "b_count", 
    COUNT(distinct "C"."id") AS "c_count", 
    COUNT(distinct "D"."id") AS "d_count" 
FROM "A" 
LEFT OUTER JOIN "B" ON ("A"."id" = "B"."A_id") 
LEFT OUTER JOIN "C" ON ("B"."id" = "C"."B_id") 
LEFT OUTER JOIN "D" ON ("C"."id" = "D"."C_id") 
GROUP BY "A"."id

其次，考慮增加適當的索引。

A - (id) 
B - (id,A_id) 
C - (id,B_id) 
D - (id,C_id)

來源

2016-06-07 10:14:41 sagi

+0

DISTINCT因爲這些ID是主鍵在這裏不需要。索引已經存在，因爲JOIN是在外鍵（默認爲索引）上製作的 – PawelRoman

+0

如果'ID'是主鍵，例如'c'表有2個記錄具有相同的'b_id'，那麼這並不重要。而表'd'有兩個記錄具有相同的'c_id'，'C.id'的計數將是4而不是2. @PawelRoman – sagi

+0

你是對的，對不起，我的壞。儘管如此，它並沒有改變我們的觀點。 COUNTs根本就不是問題。如果我們刪除COUNT，則會出現同樣的問題：LEFT JOINS需要很長時間才能完成，而用INNER JOIN替換一個LEFT JOIN可以將查詢執行時間縮短爲5秒。爲什麼？連接正在進行外鍵（空值不允許，使用默認索引） – PawelRoman

相關問題

 相關問題