2013-04-26 42 views
0

Heyo,向數據庫中輸入抓取的數據

因此,我構建了一個工作刮板並將該文件添加到我的應用程序中。我現在試圖將刮刀中的信息放入我的數據庫中。我試圖使用find_or_create方法,但我不斷收到以下錯誤。

breads_scraper.rb:49:in `block in summary': uninitialized constant Scraper::Bread (NameError) 
from /Users/Cameron/.rvm/gems/ruby-1.9.3-p392/gems/nokogiri- 1.5.9/lib/nokogiri/xml/node_set.rb:239:in `block in each' 
from /Users/Cameron/.rvm/gems/ruby-1.9.3-p392/gems/nokogiri-1.5.9/lib/nokogiri/xml/node_set.rb:238:in `upto' 
from /Users/Cameron/.rvm/gems/ruby-1.9.3-p392/gems/nokogiri-1.5.9/lib/nokogiri/xml/node_set.rb:238:in `each' 
from breads_scraper.rb:24:in `map' 
from breads_scraper.rb:24:in `summary' 
from breads_scraper.rb:57:in `<class:Scraper>' 
from breads_scraper.rb:9:in `<main>' 

我的代碼如下所示。我的理論是我正在使用find_or_create,或者文件不知道如何到達麪包方法和控制器。

require 'rubygems' 
require 'nokogiri' 
require 'open-uri' 
require 'uri' 
require 'json' 

url = Nokogiri::HTML(open("http://en.wikipedia.org/wiki/List_of_breads")) 

class Scraper 

def initialize 
    @url = "http://en.wikipedia.org/wiki/List_of_breads" 
    @nodes = Nokogiri::HTML(open(@url)) 

end 

def summary 

    bread_data = @nodes 

    breads = bread_data.css('div.mw-content-ltr table.wikitable tr') 
    bread_data.search('sup').remove 

    bread_hashes = breads.map {|x| 

     if content = x.css('td')[0] 
     name = content.text 
     end 
     if content = x.css('td a.image').map {|link| link ['href']} 
     image =content[0] 
     end 
     if content = x.css('td')[2] 
     type = content.text 
     end 
     if content = x.css('td')[3] 
     country = content.text 
     end 
     if content = x.css('td')[4] 
     description =content.text 
     end 

    { 
     :name => name, 
     :image => image, 
     :type => type, 
     :country => country, 
     :description => description, 
    } 
    Bread.find_or_create(:title => name, :description => description, :image_url => image, :country_origin => country, :type => type) 

     } 

    end 


bready = Scraper.new 
bready.summary 
puts "atta boy" 
end 

謝謝!

+0

它看起來像你期待您的軌道環境可用這取決於你如何運行腳本。 – pguardiario 2013-04-26 01:19:17

+0

關於如何加載課程的任何建議?謝謝 – DynastySS 2013-04-26 01:33:57

+0

從環境設置的耙子任務。您也可以省略要求。 – pguardiario 2013-04-26 01:42:25

回答

2

從rake任務調用刮刀。

的lib /任務/ scraper.rake

namespace :app do 
    desc "Scrape breads" 
    task :scrape_breads => :environment do 
     Scraper.new.summary 
    end 
    end 

現在,您可以按以下方式運行rake任務:

rake app:scrape_breads 
+0

謝謝,這看起來很有幫助! – DynastySS 2013-04-26 02:14:12

0

它看起來像麪包類沒有加載。

+0

我有一種感覺是這個問題。我會誠實的,但我不知道如何加載它... – DynastySS 2013-04-26 01:22:38