如何導入數據庫，更新已更改的產品，刪除已刪除的產品

Question

在基本MySQL中，我是「好的」，但這是「我的頭」！

目標：已更改

已被刪除

快速刪除產品

• 導入數據庫
• 更新的產品和高效

數據庫表（或多個）巨大的，速度是一個問題。

不需要MyISAM inoDB會更快嗎？每個數據庫將在一個唯一的表中。

我得到了這是一個開始的地方，我想要做的事：

CREATE TABLE `table` LIKE LiveTable 
LOAD DATA INFILE..... INTO `table` 
UPDATE `table` SET delete=1; -- Set the delete field to true because it will not have been updated 
UPDATE `table` INNER JOIN`table`ON `LiveTable.ID`=`table.ID` 
SET LiveTable.Col1=table.Col1, LiveTable.Col2=table.Col2….. delete=0 
INSERT INTO LiveTable(ID,Col1,Col2,… delete=0) 
SELECT ID,Col1,Col2,...FROM `table` 
LEFT JOIN LiveTable 
ON table.ID = LiveTable.ID 
WHERE LiveTable.ID IS NULL 
DELETE FROM LiveTableWHERE delete = 0 
EMPTY TABLE `table` 

---

> CREATE TABLE `product_table` (
>  `programname` VARCHAR(100) NOT NULL, 
>  `name`  VARCHAR(160) NOT NULL, 
>  `keywords` VARCHAR(300) NOT NULL, 
>  `description` TEXT NOT NULL, 
>  `sku`   VARCHAR(100) NOT NULL, 
>  -- This is the only "unique identifier given, none will be duplicates" 
>  `price`  DECIMAL(10, 2) NOT NULL, 
>  `created`  TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP, 
>  `updatedat` TIMESTAMP NOT NULL DEFAULT '0000-00-00 00:00:00', 
>  `delete`  TINYINT(4) NOT NULL DEFAULT '0', 
>  PRIMARY KEY (`sku`) ) ENGINE=myisam DEFAULT CHARSET=latin1; 
> 
> CREATE TABLE IF NOT EXISTS `temptable` LIKE `product_table`; 
> 
> TRUNCATE TABLE `temptable`; -- Remove data from temp table if for some 
> reason it has data in it. 
> 
> LOAD DATA LOW_PRIORITY LOCAL INFILE "catalog.csv" INTO TABLE 
> `temptable` FIELDS TERMINATED BY "," OPTIONALLY ENCLOSED BY """" 
> LINES TERMINATED BY "\n" IGNORE 1 LINES (`PROGRAMNAME`, `NAME`, 
> `KEYWORDS`, `DESCRIPTION`, `SKU`, `PRICE`); 
> 
> 
> UPDATE `temptable` SET `delete` = 1; -- Set the delete field to 
> true UPDATE `temptable` ttable 
>  INNER JOIN `product_table` mtable 
>   ON (mtable.sku = ttable.sku) SET mtable.programname = ttable.programname, 
>  mtable.name = ttable.name, 
>  mtable.keywords = ttable.keywords, 
>  mtable.description = ttable.description, 
>  mtable.sku = ttable.sku, 
>  mtable.price = ttable.price, 
>  mtable.created = ttable.created, 
>  mtable.updatedat = NOW(),-- Set Last Update 
>  mtable.delete = 0; -- Set Delete to NO 
> 
> -- Not sure what this is for... I'm LOST at this part... 
> INSERT INTO `product_table` VALUES  (`programname`, 
>    `name`, 
>    `keywords`, 
>    `description`, 
>    `sku`, 
>    `price`, 
>    `created`, 
>    `updatedat`, 
>    `delete`); 
> 
> -- This type of join requires alias as far as I know? 
> SELECT `programname`, 
>  `name`, 
>  `keywords`, 
>  `description`, 
>  `sku`, 
>  `price`, 
>  `created`, 
>  `updatedat`, 
>  `delete` FROM `temptable` tmptable 
>  LEFT JOIN `product_table` maintbl 
>   ON tmptable.sku = maintbl.sku WHERE maintbl.sku IS NULL; 
> 
> DELETE FROM `product_table` WHERE `delete` = 0; 
> 
> TRUNCATE TABLE `temptable`; `` remove all the data from temporary 
> table. 

Answer 1

我在這裏回答了這個問題自己： https://dba.stackexchange.com/questions/16197/innodb-update-slow-need-a-better-option/16283#16283

使用我從這裏得到的消息，在網絡和一些網絡聊天室，我想出了。網源：http://www.softwareprojects.com/resources/programming/t-how-to-use-mysql-fast-load-data-for-updates-1753.html

[DEMO][1] http://sqlfiddle.com/#!2/4ebe0/1 

的過程是：

Import into a new temp table. 
Update The old table information with information in Temp table. 
Insert new data into the table. (Real world I'm making a new CSV file and using LOAD INTO for the insert) 
delete everything that is no longer in the data feed. 
delete the temp table. 

這似乎是最快的方法爲止。

讓我知道你的意見是什麼。

Answer 2

的InnoDB通常要在表比MyISAM數據更好的是可用而INSERT，UPDATE和DELETE正在發生，因爲InnoDB使用行級鎖定進行更新，而MyISAM使用表級鎖定。

這是第一步。

第二步是在使用ALTER TABLE .. DISABLE KEYS將數據加載到表中之前禁用表中的所有索引，然後在使用ALTER TABLE .. ENABLE KEYS加載後將其重新啓用。

以上兩項顯示了您的性能大幅提升。

作爲另一種優化，在執行大規模更新時，將它們分成批（可能基於主鍵），以便所有行不會同時被鎖定。