使用SAS中的哈希對象從數據中提取某些行

Question

我有兩個SAS數據表。所述第一有許多數百萬條記錄，並且每個記錄是確定與順序記錄的ID，如下所示：

Table A 

Rec Var1 Var2 ... VarX 
1 ... 
2 
3 

第二表指定從Table A行應該被分配的編碼變量：

Table B 

Code BegRec EndRec 
AA  1200  4370 
AX  7241  9488 
BY  12119  14763 

因此Table B的第一行意味着Table A中的任何數據在012和4370之間具有rec應該被分配代碼AA。

我知道如何用proc sql來實現這個，但我想看看這是如何完成的，對散列對象。

在SQL中，這只是：

proc sql; 
select b.code, a.* 
from tableA a, tableB b 
where b.begrec<=a.rec<=b.endrec; 
quit; 

我的實際數據包含了數百個GB的數據，所以我想盡可能高效地做處理。我的理解是，在這裏使用散列對象可能會有所幫助，但我一直無法弄清楚如何映射我正在做的事情以便使用這種方式。

Answer 1

哈希對象溶液（來自@Rob_Penridge借用數據輸入代碼）。

data big; 
     do rec = 1 to 20000; 
     output; 
     end; 
    run; 

    data lookup;  
     input Code $ BegRec EndRec; 
     datalines; 
     AA  1200  4370 
     AX  7241  9488 
     BY  12119  14763 
     ; 
    run; 


    data created; 
     format code $4.; 
     format begrec endrec best8.; 
     if _n_=1 then do; 
     declare hash h(dataset:'lookup'); 
     h.definekey('Code'); 
     h.definedata('code','begrec','endrec'); 
     h.definedone(); 
     call missing(code,begrec,endrec); 
     declare hiter iter('h'); 
     end; 

    set big; 
    iter.first(); 
     do until (rc^=0); 
     if begrec <= rec <= endrec then do; 
     code_dup=code; 
     end; 
     rc=iter.next(); 
    end; 
    keep rec code_dup; 
    run; 

Answer 2

我不確定哈希表甚至會是最有效的方法。我會用SELECT語句作爲條件邏輯將會很快可能解決這個問題，它仍然只需要通過數據1個解析：

select; 
    when (1200 <= _n_ <=4370) code = 'AA'; 
    ... 
    otherwise; 
end; 

假設你將需要多次運行這段代碼和數據可能每次您可能不想硬編碼select語句時更改。所以最好的解決方案會使用宏動態構建它。我有我使用這些種情況（包括在底部）實用宏：

1）創建數據

data big; 
    do i = 1 to 20000; 
    output; 
    end; 
run; 

data lookup;  
    input Code $ BegRec EndRec; 
    datalines; 
AA  1200  4370 
AX  7241  9488 
BY  12119  14763 
; 
run; 

2）小表的內容保存到宏變量。您也可以使用call symput或其他首選方法來做到這一點。此方法假定您的查找表中沒有太多的行。

%table_parse(iDs=lookup, iField=code , iPrefix=code); 
%table_parse(iDs=lookup, iField=begrec, iPrefix=begrec); 
%table_parse(iDs=lookup, iField=endrec, iPrefix=endrec); 

3）動態構建SELECT聲明。

%macro ds; 
    %local cnt; 

    data final; 
    set big; 

    select; 
     %do cnt=1 %to &code; 
     when (&&begrec&cnt <= _n_ <= &&endrec&cnt) code = "&&code&cnt"; 
     %end; 
     otherwise; 
    end; 

    run; 
%mend; 
%ds; 

這裏是實用的宏：

/***************************************************************************** 
** MACRO.TABLE_PARSE.SAS 
** 
** AS PER %LIST_PARSE BUT IT TAKES INPUT FROM A FIELD IN A TABLE. 
** STORE EACH OBSERVATION'S FIELD'S VALUE INTO IT'S OWN MACRO VARIABLE. 
** THE TOTAL NUMBER OF WORDS IN THE STRING IS ALSO SAVED IN A MACRO VARIABLE. 
** 
** THIS WAS CREATED BECAUSE %LIST_PARSE WOULD FALL OVER WITH VERY LONG INPUT 
** STRINGS. THIS WILL NOT. 
** 
** EACH VALUE IS STORED TO ITS OWN MACRO VARIABLE. THE NAMES 
** ARE IN THE FORMAT <PREFIX>1 .. <PREFIX>N. 
** 
** PARAMETERS: 
** iDS  : (LIB.DATASET) THE NAME OF THE DATASET TO USE. 
** iFIELD  : THE NAME OF THE FIELD WITHIN THE DATASET. 
** iPREFIX : THE PREFIX TO USE FOR STORING EACH WORD OF THE ISTRING TO 
**    ITS OWN MACRO VARIABLE (AND THE TOTAL NUMBER OF WORDS). 
** iDSOPTIONS : OPTIONAL. ANY DATSET OPTIONS YOU MAY WANT TO PASS IN 
**    SUCH AS A WHERE FILTER OR KEEP STATEMENT. 
** 
****************************************************************************** 
** HISTORY: 
** 1.0 MODIFIED: 01-FEB-2007 BY: ROBERT PENRIDGE 
** - CREATED. 
** 1.1 MODIFIED: 27-AUG-2010 BY: ROBERT PENRIDGE 
** - MODIFIED TO ALLOW UNMATCHED QUOTES ETC IN VALUES BEING RETURNED BY 
** CHARACTER FIELDS. 
** 1.2 MODIFIED: 30-AUG-2010 BY: ROBERT PENRIDGE 
** - MODIFIED TO ALLOW BLANK CHARACTER VALUES AND ALSO REMOVED TRAILING 
** SPACES INTRODUCED BY CHANGE 1.1. 
** 1.3 MODIFIED: 31-AUG-2010 BY: ROBERT PENRIDGE 
** - MODIFIED TO ALLOW PARENTHESES IN CHARACTER VALUES. 
** 1.4 MODIFIED: 31-AUG-2010 BY: ROBERT PENRIDGE 
** - ADDED SOME DEBUG VALUES TO DETERMINE WHY IT SOMETIMES LOCKS TABLES. 
*****************************************************************************/ 
%macro table_parse(iDs=, iField=, iDsOptions=, iPrefix=); 
    %local dsid pos rc cnt cell_value type; 

    %let cnt=0; 
    /* 
    ** OPEN THE TABLE (AND MAKE SURE IT EXISTS) 
    */ 
    %let dsid=%sysfunc(open(&iDs(&iDsOptions),i)); 
    %if &dsid eq 0 %then %do; 
    %put WARNING: MACRO.TABLE_PARSE.SAS: %sysfunc(sysmsg());  
    %end; 

    /* 
    ** GET THE POSITION OF THE FIELD (AND MAKE SURE IT EXISTS) 
    */ 
    %let pos=%sysfunc(varnum(&dsid,&iField)); 
    %if &pos eq 0 %then %do; 
    %put WARNING: MACRO.TABLE_PARSE.SAS: %sysfunc(sysmsg());  
    %end; 
    %else %do; 
    /* 
    ** DETERMINE THE TYPE OF THE FIELD 
    */ 
    %let type = %upcase(%sysfunc(vartype(&dsid,&pos))); 
    %end; 

    /* 
    ** READ THROUGH EACH OBSERVATION IN THE TABLE 
    */ 
    %let rc=%sysfunc(fetch(&dsid)); 
    %do %while (&rc eq 0); 
    %let cnt = %eval(&cnt + 1); 
    %if "&type" = "C" %then %do; 
     %let cell_value = %qsysfunc(getvarc(&dsid,&pos)); 
     %if "%trim(&cell_value)" ne "" %then %do; 
     %let cell_value = %qsysfunc(cats(%nrstr(&cell_value))); 
     %end; 
    %end; 
    %else %do; 
     %let cell_value = %sysfunc(getvarn(&dsid,&pos)); 
    %end; 

    %global &iPrefix.&cnt ; 
    %let &iPrefix.&cnt = &cell_value ; 

    %let rc=%sysfunc(fetch(&dsid)); 
    %end; 


    /* 
    ** CHECK FOR ABNORMAL TERMINATION OF LOOP 
    */ 
    %if &rc ne -1 %then %do; 
    %put WARNING: MACRO.TABLE_PARSE.SAS: %sysfunc(sysmsg());  
    %end; 


    /* 
    ** ENSURE THE TABLE IS CLOSED SUCCESSFULLY 
    */ 
    %let rc=%sysfunc(close(&dsid)); 
    %if &rc %then %do; 
    %put WARNING: MACRO.TABLE_PARSE.SAS: %sysfunc(sysmsg());  
    %end; 

    %global &iPrefix; 
    %let &iPrefix = &cnt ; 
%mend; 

調用這個宏的其它實例：

%table_parse(iDs=sashelp.class, iField=sex, iPrefix=myTable, iDsOptions=%str(where=(sex='F'))); 
%put &mytable &myTable1 &myTable2 &myTable3; *etc...; 

Answer 3

我會試圖使用直接訪問方法POINT =在這裏，它只會讀取所需的行號而不是整個數據集。 這是代碼，它使用與Rob的答案中相同的創建數據代碼。

data want; 
    set lookup; 
    do i=begrec to endrec; 
    set big point=i; 
    output; 
    end; 
    drop begrec endrec; 
    run; 

如果你有在大數據集已有的代碼列，你只是想更新從查找數據集中的值，那麼你可以做到這一點使用修改。

data big; 
    set lookup (rename=(code=code1)); 
    do i=begrec to endrec; 
    modify big point=i; 
    code=code1; 
    replace; 
    end; 
    run; 

Answer 4

這是我的解決方案，使用proc format。這也是在內存中完成的，就像一個哈希表，但需要更少的結構代碼才能工作。

（數據輸入代碼也從@Rob_Penridge借來的。）

data big; 
    do rec = 1 to 20000; 
    output; 
    end; 
run; 

data lookup;  
    input Code $ BegRec EndRec; 
    datalines; 
    ZZ   0  20 
    JJ  40  60 
    AA  1200  4370 
    AX  7241  9488 
    BY  12119  14763 
    ; 
run; 

data lookup_f; 
    set lookup;  
    rename 
     BegRec = start 
     EndRec = end 
     Code = label; 

    retain fmtname 'CodeRecFormat'; 
run; 

proc format library = work cntlin=lookup_f; run; 


data big_formatted; 
    format rec CodeRecFormat.; 
    format rec2 8.; 
    length code $5.; 

    set big;  

    code = putn(rec, "CodeRecFormat."); 
    rec2 = rec; 
run;