如何識別和刪除重複行，除了最近

Question

我在HeidiSQL中工作，我試圖找出如何刪除除最近的所有重複行。 「重複」之間有一些細微的差別，但是每當有四個以上特定值相同（即UserID，ContactID，SMSID和EventID）時，該行就被認爲是重複的。我需要根據最近的行（由CreatedDate標識）刪除這些。

下面的查詢識別這些行：

SELECT a.UserID, a.ContactID, a.SMSID, a.EventID, CreatedDate 
FROM WhenToText a 
JOIN (SELECT UserID, ContactID, SMSID, EventID 
     FROM WhenToText 
     GROUP BY UserID, ContactID, SMSID, EventID 
     HAVING COUNT(*) > 1) b 
ON a.UserID = b.UserID 
AND a.ContactID = b.ContactID 
AND a.SMSID = b.SMSID 
AND a.EventID = b.EventID 
ORDER BY UserID, ContactID, SMSID, EventID, CreatedDate DESC 

但是，我不知道我已經確定了他們後如何刪除這些重複。

下面是一些樣本數據：

Answer 1

這裏有一個辦法：

DELETE FROM WhenToText w1 
INNER JOIN 
(
    SELECT UserID, ContactID, SMSID, EventID, MAX(CreatedDate) AS MaxDate 
    FROM WhenToText 
    GROUP BY UserID, ContactID, SMSID, EventID 
) w2 
    ON w1.UserID = w2.UserID AND w1.ContactID = w2.ContactID AND w1.SMSID = w2.SMSID 
     AND w1.EventID = w2.EventID 
     AND w1.CreatedDate != w2.MaxDate 

這將刪除CreatedDate不是最新的給定(UserID, ContactID, SMSID, EventID)組的任何記錄。請記住，如果共享最新的CreatedDate，這可能會爲每個組留下多個記錄。

如果要首先測試哪個查詢以查看哪些記錄將作爲刪除目標，那麼可以用SELECT w1.* FROM WhenToText w1替換DELETE FROM WhenToText w1。

這裏是一個SQL小提琴一個鏈接，演示了查詢如何確定刪除記錄：

SQLFiddle

Answer 2

下面是一個使用DELETE FROM JOIN，W /與您的數據的完整演示解決方案。

SQL：

-- Data preparation 
create table WhenToText(UserID int, ContactID int, SMSID int, EventID int, CreatedDate datetime); 
insert into WhenToText values 
    (4, 25, 7934, 7407, '2016-02-10 00:00:11'), 
    (4, 25, 7934, 7407, '2016-02-09 00:00:12'), 
    (4, 29, 5132, 7407, '2016-02-10 00:00:11'), 
    (4, 29, 5132, 7407, '2016-02-09 00:00:12'), 
    (4, 31, 12944, 7405, '2016-02-10 07:03:02'), 
    (4, 31, 12944, 7405, '2016-02-10 05:03:02'), 
    (4, 146, 12908, 7405, '2016-02-10 06:52:02'), 
    (4, 146, 12908, 7405, '2016-02-10 04:52:02'), 
    (15, 63, 12964, 7401, '2016-02-10 03:42:04'), 
    (15, 63, 12964, 7401, '2016-02-10 03:41:04'), 
    (15, 64, 12326, 7401, '2016-02-07 03:01:03'), 
    (15, 64, 12326, 7401, '2016-02-07 03:00:03'); 
SELECT * FROM WhenToText; 

-- SQL needed 
DELETE a FROM 
    WhenToText a INNER JOIN 
    (
    SELECT UserID, ContactID, SMSID, EventID, MAX(CreatedDate) CreatedDate 
    FROM WhenToText 
    GROUP BY UserID, ContactID, SMSID, EventID 
    ) b 
    USING(UserID, ContactID, SMSID, EventID) 
WHERE 
    a.CreatedDate != b.CreatedDate; 

SELECT * FROM WhenToText; 

輸出：

mysql> SELECT * FROM WhenToText; 
+--------+-----------+-------+---------+---------------------+ 
| UserID | ContactID | SMSID | EventID | CreatedDate   | 
+--------+-----------+-------+---------+---------------------+ 
|  4 |  25 | 7934 | 7407 | 2016-02-10 00:00:11 | 
|  4 |  25 | 7934 | 7407 | 2016-02-09 00:00:12 | 
|  4 |  29 | 5132 | 7407 | 2016-02-10 00:00:11 | 
|  4 |  29 | 5132 | 7407 | 2016-02-09 00:00:12 | 
|  4 |  31 | 12944 | 7405 | 2016-02-10 07:03:02 | 
|  4 |  31 | 12944 | 7405 | 2016-02-10 05:03:02 | 
|  4 |  146 | 12908 | 7405 | 2016-02-10 06:52:02 | 
|  4 |  146 | 12908 | 7405 | 2016-02-10 04:52:02 | 
|  15 |  63 | 12964 | 7401 | 2016-02-10 03:42:04 | 
|  15 |  63 | 12964 | 7401 | 2016-02-10 03:41:04 | 
|  15 |  64 | 12326 | 7401 | 2016-02-07 03:01:03 | 
|  15 |  64 | 12326 | 7401 | 2016-02-07 03:00:03 | 
+--------+-----------+-------+---------+---------------------+ 
12 rows in set (0.00 sec) 

mysql> 
mysql> -- SQL needed 
mysql> DELETE a FROM 
    ->  WhenToText a INNER JOIN 
    ->  (
    ->  SELECT UserID, ContactID, SMSID, EventID, MAX(CreatedDate) CreatedDate 
    ->  FROM WhenToText 
    ->  GROUP BY UserID, ContactID, SMSID, EventID 
    ->  ) b 
    ->  USING(UserID, ContactID, SMSID, EventID) 
    -> WHERE 
    ->  a.CreatedDate != b.CreatedDate; 

SELECT * FQuery OK, 6 rows affected (0.00 sec) 

mysql> 
mysql> SELECT * FROM WhenToText; 
+--------+-----------+-------+---------+---------------------+ 
| UserID | ContactID | SMSID | EventID | CreatedDate   | 
+--------+-----------+-------+---------+---------------------+ 
|  4 |  25 | 7934 | 7407 | 2016-02-10 00:00:11 | 
|  4 |  29 | 5132 | 7407 | 2016-02-10 00:00:11 | 
|  4 |  31 | 12944 | 7405 | 2016-02-10 07:03:02 | 
|  4 |  146 | 12908 | 7405 | 2016-02-10 06:52:02 | 
|  15 |  63 | 12964 | 7401 | 2016-02-10 03:42:04 | 
|  15 |  64 | 12326 | 7401 | 2016-02-07 03:01:03 | 
+--------+-----------+-------+---------+---------------------+ 
6 rows in set (0.00 sec) 

Answer 3

這應該提供您正在尋找的解決辦法，因爲CreatedDate是一個日期數據類型。這也是基於最近的行在技術上是最近的CreatedDate的假設。

SELECT UserID, ContactID, SMSID, EventID, MAX(CreatedDate) AS CreatedDate 
FROM WhenToText 
GROUP BY 1, 2, 3, 4; 

有了這些值，你可以只覆蓋WhenToText表...這將是這個樣子......

CREATE TABLE tmp_table LIKE WhenToText; 

INSERT INTO tmp_table (SELECT UserID, ContactID, SMSID, EventID, MAX(CreatedDate) AS CreatedDate 
          FROM WhenToText 
          GROUP BY 1, 2, 3, 4); 

TRUNCATE WhenToText; 

INSERT INTO WhenToText (SELECT * FROM tmp_table); 

DROP TABLE tmp_table;