Permutation Tests (and Sampling Without Replacement) Orders of Magnitude Faster Using SASŪ

by John Douglas ("J.D.") Opdyke.

Abstract: Six permutation test algorithms coded in SASŪ are compared. The fastest ("OPDN"), which uses no modules beyond Base SASŪ, achieves speed increases orders of magnitude faster than the relevant "built-in" SASŪ procedures (over 215x faster than Proc SurveySelect, over 350x faster than NPAR1WAY (which crashes on datasets less than a tenth the size OPDN can handle), and over 720x faster than Proc Multtest). OPDN combines SASŪ's fast sequential access, its most efficient array manipulation and memory usage, and a simple draw-by-draw procedure to very quickly and efficiently perform simple random sampling without replacement (SRSWOR). The particular draw-by-draw method used allows for the repeated creation of many SRSWOR permutation samples without requiring any additional storage or memory space. Based on these results, there appear to be no faster or more scalable permutation test or SRSWOR algorithms in SASŪ.

Key Words: Permutation, SAS, Scalable, Without Replacement, Sampling, SRSWOR

Author:
J.D. Opdyke, JDOpdyke@DataMineIt.com

Editor: Richard Graf, rgraf@sunstroke.sdsu.edu

READING THE ARTICLE: You can read the article in portable document (.pdf) format (211381 bytes.)

NOTE: The content of this article is the intellectual property of the authors, who retains all rights to future publication.

This page has been accessed 2874 times since JANUARY 25, 2011.


Return to the InterStat Home Page.