Rails + Sidekiq Csv導入錯誤

我只是將我的CSV上傳過程切換到工作人員上運行。它在本地工作正常，但是當我嘗試在生產中上傳文件時，出現此錯誤。在我看來，它只是不知道從哪裏搶文件Rails + Sidekiq Csv導入錯誤

017-02-22T16:32:48.914560+00:00 app[worker.1]: 4 TID-os5wk7tgo InventoryUploadWorker JID-f5be1032c019c28684582427 INFO: start 
2017-02-22T16:32:49.224819+00:00 heroku[worker.1]: source=worker.1 dyno=heroku.53973862.c2c36482-5d99-4a68-a399-0918d1ed36d2 sample#load_avg_1m=0.29 sample#load_avg_5m=0.07 sample#load_avg_15m=0.02 
2017-02-22T16:32:49.224900+00:00 heroku[worker.1]: source=worker.1 dyno=heroku.53973862.c2c36482-5d99-4a68-a399-0918d1ed36d2 sample#memory_total=144.37MB sample#memory_rss=134.18MB sample#memory_cache=6.66MB sample#memory_swap=3.54MB sample#memory_pgpgin=55377pages sample#memory_pgpgout=19323pages sample#memory_quota=512.00MB 
2017-02-22T16:32:49.167416+00:00 app[worker.1]: Company Load (0.6ms) SELECT "companies".* FROM "companies" WHERE "companies"."id" = $1 LIMIT 1 [["id", 32]] 
2017-02-22T16:32:49.246868+00:00 app[worker.1]: 4 TID-os5wk7tgo InventoryUploadWorker JID-f5be1032c019c28684582427 INFO: fail: 0.332 sec 
2017-02-22T16:32:49.247408+00:00 app[worker.1]: 4 TID-os5wk7tgo WARN: {"class":"InventoryUploadWorker","args":["/tmp/RackMultipart20170222-4-1jaehp1.csv","32"],"retry":false,"queue":"default","jid":"f5be1032c019c28684582427","created_at":1487781168.915459,"enqueued_at":1487781168.9161458} 
2017-02-22T16:32:49.247452+00:00 app[worker.1]: 4 TID-os5wk7tgo WARN: Errno::ENOENT: No such file or directory @ rb_sysopen - /tmp/RackMultipart20170222-4-1jaehp1.csv

工人：

class InventoryUploadWorker 
    include Sidekiq::Worker 
    sidekiq_options retry: false 

    Sidekiq.configure_server do |config| 
    config.redis = { url: ENV["REDISTOGO_URL"], network_timeout: 5 } 
    end 

    Sidekiq.configure_client do |config| 
    config.redis = { url: ENV["REDISTOGO_URL"], network_timeout: 5 } 
    end 

    def perform(file_path, company_id) 
    CsvImport.csv_import(file_path, Company.find(company_id)) 
    end 
end

導入方法：

class CsvImport 

    def self.csv_import(filename, company) 
     time = Benchmark.measure do 
      File.open(filename) do |file| 
       headers = file.first 
       file.lazy.each_slice(150) do |lines| 
        Part.transaction do 
         inventory = [] 
         insert_to_parts_db = [] 
         rows = CSV.parse(lines.join, write_headers: true, headers: headers) 
         rows.map do |row| 
          part_match = Part.find_by(part_num: row['part_num']) 
          new_part = build_new_part(row['part_num'], row['description']) unless part_match 
          quantity = row['quantity'].to_i 
          row.delete('quantity') 
          row["condition"] = match_condition(row) 
          quantity.times do 
           part = InventoryPart.new(
            part_num: row["part_num"], 
            description: row["description"], 
            condition: row["condition"], 
            serial_num: row["serial_num"], 
            company_id: company.id, 
            part_id: part_match ? part_match.id : new_part.id 
            )   
           inventory << part     
          end 
         end 
         #activerecord-import (bulk import) 
         InventoryPart.import inventory 
        end 
       end 
      end   
     end 
     puts time 
    end

來源

2017-02-22 gemart

這不是針對一個好主意sidekiq進程依賴於web進程中的臨時文件。如果工作失敗並在下週重試，會發生什麼情況？如果您的Web和工作進程位於不同的機器或不同的容器中，會發生什麼情況？

您應該將CSV內容作爲參數推送或將文件移動到知名點以供工作人員選取。

來源

2017-02-22 17:28:49

你會推薦什麼？將文件保存到Amazon S3？我絕對不想將它們保存到我的應用程序中。我已將重試設置爲false，以便我可以查看錯誤，因爲此用例不一定需要重試。謝謝。 – gemart

「你應該把CSV內容作爲參數」 –

我認爲這裏的問題是csv文件是100k +行 – gemart

Rails + Sidekiq Csv導入錯誤

回答

相關問題