如何使用附加列顯示文件名進行BULK INSERT？

Question

我還是習慣了SQL，所以在開始使用存儲過程之前，我想先了解如何有效地使用BULK INSERT。

我需要合併50個csv文件並將它們轉儲到SQL表中。問題是，我希望能夠區分每個記錄（如每個記錄屬於某個csv文件，我將通過文件名識別）。

這裏有我想要的一個小例子：

CREATE TABLE ResultsDump 
(
    PC FLOAT, 
    Amp VARCHAR(50), 
    RCS VARCHAR(50), 
    CW VARCHAR(50), 
    State0 VARCHAR(50), 
    State1 VARCHAR(50), 
) 
BULK INSERT ResultsDump 
    FROM 'c:\distance1000_7_13_2010_1_13PM_Avery DennisonAD_2300008_10S_Lock.csv' 
    WITH 
    ( 
     FIRSTROW = 2, 
     MAXERRORS = 0, 
     FIELDTERMINATOR = ',', 
     ROWTERMINATOR = '\n' 
    ) 
BULK INSERT ResultsDump 
    FROM 'c:\distance1000_7_13_2010_2_27PM_Avery DennisonAD_2300009_10S_Lock.csv' 
    WITH 
    ( 
     FIRSTROW = 2, 
     MAXERRORS = 0, 
     FIELDTERMINATOR = ',', 
     ROWTERMINATOR = '\n' 
    ) 
BULK INSERT ResultsDump 
    FROM 'C:\distance1000_7_13_2010_2_58PM_Avery DennisonAD_230000A_10S_Lock.csv' 
    WITH 
    ( 
     FIRSTROW = 2, 
     MAXERRORS = 0, 
     FIELDTERMINATOR = ',', 
     ROWTERMINATOR = '\n' 
    ) 
BULK INSERT ResultsDump 
    FROM 'c:\distance1000_7_13_2010_3_21PM_Avery DennisonAD_230000B_10S_Lock.csv' 
    WITH 
    ( 
     FIRSTROW = 2, 
     MAXERRORS = 0, 
     FIELDTERMINATOR = ',', 
     ROWTERMINATOR = '\n' 
    ) 
BULK INSERT ResultsDump 
    FROM 'c:\distance1000_7_13_2010_3_41PM_Avery DennisonAD_230000C_10S_Lock.csv' 
    WITH 
    ( 
     FIRSTROW = 2, 
     MAXERRORS = 0, 
     FIELDTERMINATOR = ',', 
     ROWTERMINATOR = '\n' 
    ) 

我知道這是做事情的方式效率極低，但我絕對想弄清楚如何手工轉儲一個文件中的SQL表中格式我想在開始創建存儲過程之前。

在新表，我想是這樣的：

FileName,PC,Amp,RCS,CW,State0,State1 
c:\distance1000_7_13_2010_1_13PM_Avery DennisonAD_2300008_10S_Lock.csv, ... 
... 
... 
c:\distance1000_7_13_2010_2_27PM_Avery DennisonAD_2300009_10S_Lock.csv, ... 
... 
... 
c:\distance1000_7_13_2010_2_58PM_Avery DennisonAD_230000A_10S_Lock.csv, ... 
... 
... 

任何簡單的建議或推薦特定的功能將是巨大的！請記住，我已經習慣了SQL，如果我可以一次完成這一步，那將是非常棒的，這就是爲什麼我要從這樣一個簡單的問題開始。

提前致謝！

Answer 1

您可以將列FileName varchar(max)添加到ResultsDump表中，使用新列創建表的視圖，批量插入視圖中，並且在每次插入之後，爲仍具有其默認值的列設置文件名稱null ：

CREATE TABLE dbo.ResultsDump 
(
    PC FLOAT, 
    Amp VARCHAR(50), 
    RCS VARCHAR(50), 
    CW VARCHAR(50), 
    State0 VARCHAR(50), 
    State1 VARCHAR(50), 
) 
GO 

ALTER TABLE dbo.ResultsDump ADD [FileName] VARCHAR(300) NULL 
GO 

CREATE VIEW dbo.vw_ResultsDump AS 
SELECT 
    PC, 
    Amp, 
    RCS, 
    CW, 
    State0, 
    State1 
FROM 
    ResultsDump 
GO 

BULK INSERT vw_ResultsDump 
    FROM 'c:\distance1000_7_13_2010_1_13PM_Avery DennisonAD_2300008_10S_Lock.csv' 
    WITH 
    ( 
     FIRSTROW = 2, 
     MAXERRORS = 0, 
     FIELDTERMINATOR = ',', 
     ROWTERMINATOR = '\n' 
    ) 

UPDATE dbo.ResultsDump 
SET [FileName] = 'c:\distance1000_7_13_2010_1_13PM_Avery DennisonAD_2300008_10S_Lock.csv' 
WHERE [FileName] IS NULL 

BULK INSERT vw_ResultsDump 
    FROM 'c:\distance1000_7_13_2010_2_27PM_Avery DennisonAD_2300009_10S_Lock.csv' 
    WITH 
    ( 
     FIRSTROW = 2, 
     MAXERRORS = 0, 
     FIELDTERMINATOR = ',', 
     ROWTERMINATOR = '\n' 
    ) 

UPDATE dbo.ResultsDump 
SET [FileName] = 'distance1000_7_13_2010_2_27PM_Avery DennisonAD_2300009_10S_Lock.csv' 
WHERE [FileName] IS NULL 

Answer 2

試試這個，

    ALTER PROCEDURE [dbo].[ReadandUpdateFileNames_SP] 
      (
      @spRequestId NVARCHAR(50) 
      ,@LoopCounter INT =0 
      ,@MaxFIVId INT=0 
      ,@spFileName NVARCHAR(100)=NULL) 
      AS 
      BEGIN 
      SET NOCOUNT ON 
       BEGIN TRY 
        BEGIN TRAN 
        -- To read filename's from the Request Id and store it in a temp table 
         DECLARE @cmd nvarchar(500) 
         SET @cmd = 'dir D:\Input\REQUEST-'+@spRequestId+' /b ' 
         --PRINT @cmd 
         DECLARE @DirOutput TABLE(
             ID INT IDENTITY 
            , files varchar(500)) 

         INSERT INTO @DirOutput 
         EXEC master.dbo.xp_cmdshell @cmd 
         SELECT * FROM @DirOutput WHERE files IS NOT NULL ORDER BY ID 

         ----Read files by RequestId BEGIN 

         SELECT @LoopCounter = min(ID) , @MaxFIVId = max(ID) 
          FROM @DirOutput 

         WHILE(@LoopCounter IS NOT NULL AND @LoopCounter<@MaxFIVId) 
         BEGIN 
         -- Create temp table to store FIVItems 
          CREATE TABLE Items_TEMP 
          (
            ControlID NVARCHAR(50) 
           , UNRS_Code NUMERIC(18,0) 
           , UNRS_Code_S NUMERIC(18,0) 
           , Ordered_Quantity NUMERIC(18,3) 
           , Sent_Quantity NUMERIC(18,3) 
           , Accepted_Quantity NUMERIC(18,3) 
           , Unit_Food_Price NUMERIC(18,2) 
           , Total_Price NUMERIC(18,2) 
          ) 
         SELECT @spFileName=files FROM @DirOutput WHERE ID=@LoopCounter 
          PRINT @LoopCounter 
          DECLARE @spControlId NVARCHAR(50) 
          SET @spControlId='FFO'+ Substring(@spFileName, 4, (len(@spFileName)-7)) 
          --PRINT @spControlId 
          DECLARE @sqlCmd NVARCHAR(MAX) 
          SET @sqlCmd='BULK INSERT 
          Items_TEMP 
          FROM ''D:\Input\REQUEST-'+@spRequestId+'\'+@spFileName+''' 
          WITH(
            FIELDTERMINATOR='','' 
           , ROWTERMINATOR=''\n'' 
          )' 
          PRINT @sqlCmd 
          EXECUTE sp_executesql @sqlCmd 
          ---Add a new column to the table which is not present in the CSV 
          ALTER TABLE Items_TEMP ADD OrderId NUMERIC(18,0)NULL 
          UPDATE Items_TEMP SET ControlID=@spControlId,OrderId=(SELECT OrderId FROM dbo.Orders WHERE ControlID=@spControlId) 
          SELECT * FROM Items_TEMP 

          --DROP FIVItems_TEMP table once CSV output generated 
          DROP TABLE Items_TEMP 


          SET @LoopCounter=@LoopCounter+1 
         END 
        ----****END*** 
        COMMIT TRAN 
       END TRY 
       BEGIN CATCH 
        ROLLBACK TRAN 
        PRINT 'ROLLBACK' 
        PRINT Error_Message() 
        SELECT ERROR_LINE() AS ErrorLine; 
       END CATCH 
      SET NOCOUNT OFF 
      END