在PIG中引用嵌套元組中的字段;

我一直堅持這幾個小時，我無法弄清楚我做錯了什麼。我有關係「分組」，與在PIG中引用嵌套元組中的字段;

grouped: {seedword: chararray,baggy: {outertup: (groupy: (seedword: chararray,coword: chararray))}}

架構之間的關係是什麼樣是一個示例：（自動，{（（汽車，汽車）），（（汽車，卡車））} ）

我需要生成一個種子字和一個元組。在我的例子中，我想

（汽車，（汽車，卡車））。

我曾嘗試：

FOREACH grouped GENERATE baggy::outertup.groupy.coword; 

FOREACH grouped GENERATE baggy.outertup.groupy.coword; 
FOREACH grouped GENERATE baggy.groupy.coword;

，沒有這些工作，並給我的錯誤信息說沒有這樣的領域。請幫忙！！

這裏的一些我的代碼：

keywords = LOAD 'merged' USING as (seedword:chararray, doc:chararray); 

---COUNT HOW MANY DOCUMENTS EACH WORD IS IN 
group_by_seedword = GROUP keywords BY $0; 

invert_index = FOREACH group_by_seedword GENERATE $0 as seedword:chararray, keywords.$1; 
word_doc_count= FOREACH invert_index GENERATE seedword, COUNT($1); 

-- map words to document 
words_in_doc= GROUP keywords BY doc; 
word_docs = FOREACH words_in_doc GENERATE group AS doc, keywords.seedword; 
--(document:(keyword, keyword, keyword...)) 

--map words to their cowords in doc 
temp_join = JOIN keywords BY doc,word_docs BY doc; 
--DUMP temp_join; 
cowords_by_doc = FOREACH temp_join GENERATE $0 as seedword:chararray, $3 as cowords; 

cowords_interm= FOREACH cowords_by_doc GENERATE seedword, FLATTEN(cowords); 
cowords = FILTER cowords_interm BY (seedword!=$1);---GETS RID OF SINGLE DOC WORD; 
temp_join_count1 = JOIN cowords BY $0, word_doc_count BY seedword; 

-- GETS WORDS THAT OCCURE BY THEMSELVES IN A SINGLE DOCUMENT 
G = JOIN cowords_interm BY $0 LEFT OUTER, cowords by $0; 
orph_word = FILTER G BY $2 is null; 
orph_word_count = FOREACH orph_word GENERATE $0,null, 0; 

temp_join_count= UNION temp_join_count1, orph_word_count; 

inter_frac = FOREACH temp_join_count GENERATE $0 as seedword:chararray, $1 as coword:chararray, 1.0/$3 as frac:double; 
inter_frac_combine = GROUP inter_frac BY (seedword, coword); 
inter_frac_sum = FOREACH inter_frac_combine GENERATE $0 , SUM(inter_frac.frac) as frac:double; 

filtered = FILTER inter_frac_sum BY ($1 >=$relatedness_ratio); 
grouped= GROUP filtered by $0.seedword; 
g = FOREACH grouped GENERATE group as seedword:chararray, filtered.$0; 
named = FOREACH g GENERATE $0 as seedword:chararray, $1 as baggy:bag{(outertup:tuple(groupy:tuple(seedword:chararray, coword:chararray)))};

你可以嘗試輸入文件應該是這樣的：

car doc1.txt 
auto doc1.txt 
bunny doc2.txt 
ball doc2.txt 
toy car doc2.txt 
random doc3.txt

平面doc3.txt

來源

2012-01-12 YuliaPro

請問您可以包括幾行你的輸入和你的豬腳本？如果我能夠將它插入豬身上並隨身攜帶來幫助你，那將是非常棒的。 – 2012-01-12 20:11:17

當然;它有點長，但： keywords = LOAD'test'as（seedword：chararray，doc：chararray）; group_by_seedword = GROUP關鍵字BY $ 0; – YuliaPro 2012-01-12 21:23:09

我將代碼和示例文件添加到原始問題中！謝謝。我開始認爲這裏有一個bug： FOREACH分組GENERATE seedword，baggy.groupy; 我也收到一個錯誤：無效的字段參考。參考字段[groupy]在模式中不存在：seedword：chararray，coward：char數組。（所以它似乎是忽略所有嵌套元組）。 – YuliaPro 2012-01-12 21:59:18

我也有類似的問題，我無法引用內部元組。我的解決方案是平整數據，然後進行一些更多的過濾和分組。乾杯 V

來源

2013-01-29 00:15:03 user2019983

在PIG中引用嵌套元組中的字段;

回答

相關問題