If one wants to know “how many guns were sold?” in X year or Y month, there is only one source that is commonly accepted at this point, the FBI National Instant Criminal Background Check System (NICS). However, the raw data from the NICS comes with many caveats. The biggest are 1) each background check may be for more than one gun (or even no guns if the buyer backs out), 2) not all states use NICS checks exclusively or allow alternative forms of background checks for gun sales (such as CCW licenses), 3) private sales are not included in the NICS for most states, and 4) and the use of NICS by state law enforcement for checking CCW licenses has often been conflated with firearms sales (this problem has been mitigated in more recent years though).
Guns were in stock, but the state background check system had a 3-5 day wait.
I first became interested in measuring gun sales in 2011, when I started working on a project to see what social factors seemed to influenced gun demand. That project was eventually published in 2017 (with my friend Marty Kolsa). But it took more than 5 years to publish what was a relatively straightforward study because it literally took us years to obtain data from the FBI that allowed us to parse out NICS background checks purpose codes (such as concealed carrying application NICS checks). Back in 2011, the FBI only made NICS publicly available for the TOTAL number of NICS checks. We actually wrote a Freedom of Information Request to the FBI to obtain the purpose codes, and after more than a year the FBI finally sent us the spreadsheets… as scans of printouts… in a PDF file… on a CD… that was sent via snail mail. We then had to manually enter in every state’s NICS numbers reading off the fuzzy digital printouts. Thankfully, things have improved. The FBI now provides NICS data in clean PDFs easily converted into spreadsheets. If someone has access to Adobe (~30 bucks a month) they can make their own monthly data in a few hours, bing-bang-boom (I know python folks can do this too, but I don’t have that skill).
Yet should one just use the NICS data? Several organizations use the NICS data to estimate firearm demand and sales and each makes their own tweaks to clean the data. There are three main organizations I’m aware of: 1) Small Arms Analytics (SAA) 2) the National Shooting Sports Foundation (NSSF) and 3) The Trace.
During 2020, Small Arms Analytics has put out monthly press releases using their method of cleaning NICS data to parse out trends in gun sales. And they seem to be offering data products for sale using NICS data as well as the Federal Firearms Licensee listings from the BATFE (the FFL listings is particularly curious to me, because that data is publicly accessible, I don’t know why one would spend money on this, but live and let live). Here is how SAA describe their most expensive data product, a database of “U.S. total firearms demand (by state, by firearms type; subscription, updated monthly)“
Its clear the raw data for this product are from the FBI NICS, which has the handgun and long gun NICS checks categories delineated. It seems the SAA folks are combining and adjusting the other NICS data as they see fit (which is fine) but beyond that, its hard to tell what they are doing and how its better than just using the FBI NICS data on one’s own. SAA is also seemingly trying to keep their data as proprietary as possible, I would be curious to know if it can be replicated.
The National Shooting Sports Foundation (NSSF) also uses NICS data in a similar way, and uses it to provide industry guidance to its members. The NSSF has been at this for a long time, they were providing their “adjusted” NICS numbers that tried to parse sales from background checks for CCW licenses and other uses back when I wrote our FOIA for our 2017 paper in 2011 (we didn’t have deep enough pockets to buy the NSSF data, so we wrote the FOIA and waited). The NSSF is pretty clear in describing how they use their data, as they explain in this press release criticizing The Trace (who I will talk about shortly). Again, like SAA, the NSSF is pretty clearly using the FBI NICS data and tweaking it as they see appropriate and adding unique state data as they see fit.
The last organization I’ll talk about regularly providing NICS data is The Trace. They cite the SAA as a guiding influence on their approach, and to their credit seem to be transparent about their methodology. Note too, that although the NSSF says the Trace doesn’t account for permit checks, the Trace claims they do.
Finally, there is the DIY approach which I have attempted. In my 2019 Social Science Research article on CCW laws and gun sales, I took an approach to parse out handgun from long gun sales that relied on trying to find states that didn’t report NICS data well and use their own data. The short way to describe this is to take Iowa as an example. Iowa requires a permit to purchase a handgun rather than an ordinary NICS check, but these permits aren’t reflected in the NICS data. I wrote a FOIA request to the Iowa State Police to see how many pistol purchase permits were issued in a given year to figure out how many guns were purchased (this is something the NSSF does too). But, again, the biggest share of the data came from the FBI NICS data. Beyond a handful of states where I supplemented/amended their data using state records, the NICS is pretty much it when you can subtract out the permit checks from the purchase checks.
So, SAA, NSSF, and The Trace all use the NICS data. SAA and NSSF seem to have proprietary methodologies and offer their data to anyone willing to purchase it. The Trace details their methodology and makes all their data open access. As a social scientist, I have to say I like the openness and plausible reproducibility of the Trace data. Although I am not sure I would encourage moving away from just simply using the NICS data straight from the FBI. It might be biased in some ways, but we KNOW what those biases are and where they exists (this might be better than dealing with unknown biases that might emerge from tweaks made by researchers that may not be clearly documented).
I might start using the Trace and comparing with the raw FBI NICS data in the future to see how much of a difference it makes. It helps that the Trace Data are open access, which makes such a comparison possible. Still, I’m still not sure what advantage any one source (SAA, NSSF, The Trace, or just raw NICS) has over the others or why one might want to purchase one over the other. If anything, I think I am partial to the FBI NICS raw data (i.e. just drop all the NICS checks that aren’t in the headers for handgun, long gun, multiple, other, and private sales). If for no other reason than it is simple, reproducible, and free. When dealing with social science data, and especially with firearms data, I think there is value in erring toward those traits. Reproducibility, specifically, seems good given the fact that firearms research has dealt with reproducibility issues in the past.
When it comes right down to it, it seems like we’re all just playing with the NICS data. I’ve yet to see anyone come up with a better foundation for measuring gun sales than the NICS numbers. Unfortunately, it seems unlikely that any one source can be demonstrated to be “the best” until it is know how many NICS checks are accurately capturing guns sales and a comparison is made across all of the states. Still, all of these approaches are better than guesswork or reliance on anecdotal accounts of business. Perhaps, for the moment this is a nerdy version of the cola wars. Are you a Coke or Pepsi person? Are you a raw NICS or NSSF person?
(sarcasm here) When it comes to gathering data, I am also a member of the Red Green DIY school of thought. You could purchase clean data and have the job done, but then you pay for something you could do yourself and you don’t know if a bad job was done. Whereas if you do it yourself you not only save money, you know for sure a bad job was done.
Trent Steidley 
2 thoughts on “Are all gun sales data the same?”
Comments are closed.