Home › Forums › Main Forums › SAS Forum › Teach you write SAS macro: How to split data by every 1000 records?
-
Teach you write SAS macro: How to split data by every 1000 records?
-
Question: I have a big data set with 100,000 records, I want to break it into 100 datasets by every 1000 records. How to do it?
Step 1: Fulfill the task with standard code
Data Z1 Z2 Z3.... Z100;
Set AAA;
If _N_=1-1000 then output Z1; --- I=1
If _N_=1001-2000 then output Z2; --- I=2
If _N_=2001-3000 then output Z3; --- I=3
..........
If _N_=99001-100000 then output Z100 --- I=100
Run;Step 2: Convert Macro Codes
%macro cover();
%local I;
data
%do I=1 %to 100;
BBB&I
%end;
;
set AAA;
%do I=1 %to 100;
if _n_/1000<=&I and _n_/1000>&I-1 then output BBB&I;
%end;
run;
%mend;
%cover;Improved Method I:
%macro subgroup;
%local I;
%do I=1 %to 100;
data Z&I;
set AAA;
if _N_ LE &I*1000 and _N_ GT (&I-1)*1000 then output;
else delete;
run;
%end;
%mend;
%subgroup;Improved Method II:
This is much better than Method I in efficiency, because SAS only reads in 1000 records each loop, other than reading in all the observations as in Method I. Options mprint mlogic;%macro subgroup;
%local I;
%do I=1 %to 100;
%let A=%eval((&I-1)*1000+1);
%let B=%eval(&I*1000);
%put A=*&A* B=*&B*;data Z&I;
set AAA (firstobs=&A obs=&B);
run;
%end;
%mend;
%subgroup;Improved Method III:
Data BBB;
Set AAA;
Order=_N_;
run;
%macro subgroup;
%local I;
%do I=1 %to 100;
data Z&I (drop=Order);
set BBB;
where Order LE &I*1000 and Order GT (&I-1)*1000;
run;
%end;
%mend;
%subgroup;WHERE statement will do subsetting at compile time. This method is also more efficient than Method I. Bingo!
Log in to reply.