Downloads

Data-driven validation rules:
Custom data validation without custom programming
SESUG 2003, St. Petersburg, Florida

One of the most expensive and time-consuming aspects of data management is the programming of validation rules (also known as “edit checks”) to locate data values that may be incorrect. Implementing these kinds of checks through custom code is standard practice in many organizations, despite the high cost of creating and validating custom programs. A data-driven approach – especially one that handles multivariate consistency checks – can radically decrease the time and expense involved in creating custom validation rules. This paper describes a data-driven system for creating and applying validation rules to locate questionable data values in SAS datasets.

• Paper (PDF document)
• Source code for DataChecker system (Zip file)

Sounding the Trumpet: Effective Failure Notification
SESUG 2004, Nashville, Tennessee

A typical data management system is a complex collection of manual and computerized procedures. When something does go wrong, it is often essential to catch the failure quickly to avoid a cascade of costly downstream problems. In a system that employs SAS programs, one way to find out about system failures is to scan the SAS log window after a program executes. Manual scanning of the log may be all that’s needed for a simple program, but as a system grows in complexity, manual scanning may cease to be a reliable or efficient technique for noticing problems. Fortunately, there are alternatives to manual scanning. This paper describes programmatic techniques that are useful for capturing failures in SAS programs and alerting those who need to know about them.

• Paper (PDF document)