The U.S. Government increasingly uses statistical analysis and data mining to identify fraud and abuse in the healthcare market. A key tool in the Government’s arsenal is RAT-STATS software, which is used to sample and quantify improper claims, which then become the basis for damages and monetary penalties. As a result, providers, compliance officers and legal professionals must understand the methodologies used by the Government to perform its analysis. Understanding potential weaknesses in the Government’s analysis is also imperative to a sound defense. This post provides guidance on both fronts…
First, it offers an understanding of RAT-STATS’ capabilities to help firms better implement their own proactive statistical analysis. Second, it articulates the multitude of steps and decisions required to be used in conjunction with the software. It provides an outline of those steps and offers a reactive framework to refute Government analysis when it may be sub-standard.
RAT-STATS Overview
RAT-STATS is a free statistical software package developed by the U.S. government and provided to assist vendors, compliance professionals and auditors with statistical analysis. In turn, the government often relies on the software when investigating enforcement or fraud matters, and RAT-STATS is the primary statistical tool for the OIG’s Office of Audit Services.[1] Among other things, the software provides an interface for assisting the user in (1) determining the appropriate size of a sample to achieve a desired confidence and precision, (2) selecting random samples and (3) evaluating results of a sample in order to estimate overpayment amounts. While a user guide for RAT-STATS exists, the 394-page guide and 245-page companion manual are overwhelmingly technical and they often dispirit those who most need to understand them. This section does not offer a step-by-step guide for using RAT-STATS, however, it is intended for those who require a strategic understanding of RAT-STATS’ purpose, capabilities and shortcomings. This post will demystify RAT-STATS and clearly articulate what the software can and cannot do.
Hypothetical RAT-STATS Example
To facilitate the discussion of RAT-STATS, consider a hypothetical scenario in which the government has filed suit against HealthAid, a fictitious national nursing home network. The government’s complaint alleges a significant portion of HealthAid’s 92,000 mental health treatments were medically unnecessary. The government’s analysis, performed using RAT-STATS, alleged over 70 percent of treatments during the past four years were unnecessary, resulting in alleged damages to the government of $88 million.
RAT-STATS Functionality
RAT-STATS is designed to assist in selecting and analyzing random samples as well as interpreting the results. While many consider RAT-STATS to be a “black-box,” the reality is remarkably less complex. Consider your standard calculator, which performs discrete functions (add, subtract, etc.). RAT-STATS is no different. RAT-STATS’ performs three primary functions:
-
Determine sample size (i.e. how many data points should be examined)
The purpose of this function is to select a sample of sufficient size to yield useful results. With limited input, RAT-STATS calculates the number of instances required to infer characteristics of the full population in a meaningful manner. Notably, this calculation requires the user to first specify the desired level of uncertainty (i.e. confidence and precision) acceptable in the resulting conclusions. In our example, this would denote a subset of HealthAid’s 92,000 treatments. Assuming the government requires 95 percent confidence with 6 percentage points of precision, RAT-STATS would calculate a required simple random sample size of 1,085 treatments.[2]
-
Generate random numbers to select the sample
The purpose of this function is to help ensure the selection of a random sample based upon a predetermined sample size. RAT-STATS generates a set of random numbers used to select the specific sampling units from the total population. In our example, these random numbers would dictate exactly which of the 92,000 treatments should be selected, at random, to construct the sample of 1,085 treatments.
-
Infer (i.e. extrapolate) the performance of the total population, based on the results from the sample
After selecting and analyzing the sample, this function helps to infer how the sample results relate to the total population (i.e. frequency, size, etc.). For HealthAid, the government performs this function after analyzing the sample of 1,085 treatments, to demonstrate how the overpayment and/or frequency of unnecessary treatments translate to the total population.
As you can see, RAT-STATS’ functionality is reasonably simple to understand and rather limited in its scope. In fact, these functions can be performed quite easily without using RAT-STATS. However, with the government’s frequent use of the software and its familiarity with the outputs, professionals and providers should be conversant in the software’s role and capabilities.
Considerations for Using RAT-STATS
As described, it is worthwhile to think about RAT-STATS like a calculator with three distinct functions. But don’t be misled: using a calculator does not guarantee conclusions will be reliable, and using RAT-STATS is no different. A broader framework is required to render reliable conclusions.
Overreliance on RAT-STATS, without a grasp of the underlying data or circumstances, can lead to inaccurate or even invalid statistical conclusions. This is due to a variety of assumptions and decisions that exist outside of the software itself. These decisions comprise a broader statistical framework, and they include a variety of pitfalls that may result in sub-standard analysis. Therefore, it is imperative to consider the totality of decisions and analysis involved in statistical sampling, and the role software can play in such analysis. Forensus’ RAT-STATS Advisor provides additional posts articulatung a standardized 8-step framework for designing, planning and executing sampling analysis, along with a discussion of RAT-STATS and its role in conducting analysis.
[1] https://oig.hhs.gov/compliance/compliance-guidance/index.asp, last accessed on September 26, 2016.
[2] Certain government procedures and protocols for statistical sampling may specify other degrees of confidence and precision, however for the purposes of our example, we have selected 95 percent confidence with 6 percentage points of precision.
© Forensus Group, LLC | 2017