Understanding the NICJR report

Given the frequency in which the NICJR report is cited in City Hall and Seattle political circles, you would be forgiven if you thought that copies of it were flying off the shelf. But reading the report (affectionately referred to as “Nick Junior”) is like teenage sex: a lot of people are talking about it, but almost no one is doing it. And yet it has taken a central role in the debate over the nature and pace of changes to 911 response in our city, so it’s worth knowing what it says — and how skeptical we should be of its conclusions.

As a component of Mayor Durkan’s 2020 “Executive Order to Reimagine Policing and Community Safety in Seattle“, SPD contracted with the National Institute for Criminal Justice Reform (NICJR) to “conduct an analysis of SPD calls for service for the period 2017-2019” to answer four questions:

  1. What are the characteristics of calls for service to which Seattle Police respond?
  2. What are the primary initiation sources for calls for service to which Seattle Police respond?
  3. How much time do officers spend responding to called for service?
  4. Which types of calls for service should be responded to by a non-SPD alternative?

The report has pages of tables and charts in answer to the first three questions, but let’s be honest here, the whole point of the report is question #4: an attempt to quantify how much of what SPD does in response to 911 calls could be handed off to civilian-based organizations, some combination of government agencies and community-based organizations.

Click to access attachment-3-seattle-calls-for-service-analysis-report-with-appendices-nicjr-june-2021-final.pdf

There is a short list of talking points from the report’s findings that are on auto-repeat in City Hall:

  • Nearly 80% of SPD’s 911 calls are non-criminal in nature, while only about 6% relate to felonies;
  • 49% of 911 calls to SPD should be handled by organizations other than SPD;
  • SPD has admitted that 12% of its 911 calls could immediately be offloaded to alternative responses.

The truth, as you may imagine, is more complicated and nuanced. So let’s dig in.

SPD gave NICJR three years of police dispatch call data (2017-2019), which seems to be the same dataset the department has posted on its website. NICJR presented a high-level breakdown of calls for service by their source of origin: 911, alarms, police officers noticing something and calling it in (called “on-view”), and people calling SPD’s non-emergency number. They also presented several cuts on the most common call types, looking at overall statistics as well as from each of the sources of origin, seasonal shifts, day of week, time of day, precinct, and a few other categories. It’s all in the report, and on its own none of it is particularly enlightening. Then they start to get to the heart of the matter: they break out the calls by “crime category”: misdemeanors, nonviolent felonies, violent felonies, and non-criminal matters. They found that almost 80% of the calls were non-criminal in nature.

They also looked at which call types led to the most arrests, and what percentage of those call types ultimately led to arrests.

NICJR has a standard four-tier model that it uses to decide whether a particular call for service should be reponded to by SPD, a “community emergency response network” (CERN), or both:

Tier 1: CERN dispatched only
· Event type: Non-Criminal
Tier 2: CERN lead, with officers present
· Event type: Misdemeanor with low potential of violence
· If CERN arrives on scene and determines there is low potential for violence and an arrest is unnecessary or unlikely, officers leave.
Tier 3: Officers lead, with CERN present
· Event type: Non-Violent Felony or an arrest is likely
· If officers arrive on scene and determine there is no need for an arrest or an arrest is unlikely and violence is unlikely, officers step back and CERN takes the lead.
Type 4: Officers only
· Event type: Serious Violent Felony or high likelihood of arrest

Based on the crime category and the likelihood of an arrest, they assigned every call type (SPD uses about 350 different call types to categorize its calls for service) to one of the four levels. This led to their conclusion that up to 49% of SPD’s calls could be handled without SPD sworn officers’ involvement, and another 24% should be led by a CERN with SPD officers in secondary, supporting roles.

Looking deeper at the tier assignments, they decided that 100% of the top ten officer “on-view” call types and 80% of the 911 and non-emergency call types were in “Tier 1”, which does not require an SPD officer to respond.

The NICJR report concludes that the city should move forward to design alternative response systems for the 73% of calls where a CERN can lead the response.

To begin, there are several issues with the data that NICJR used for this analysis, as detailed in the report:

  • the call data doesn’t differentiate between sworn and non-sworn response;
  • not all units are included, such as the civilian-based Crisis Response Unit;
  • there is a bit of noise as changes in SPD’s call-type taxonomy happen over time;
  • if an arrest happens after the end of a service call (perhaps even days later), that is not linked back to the originating call;
  • this gives us a breakdown of the calls, but not a breakdown of where SPD officers are spending their time, since different call types require a greater or fewer number of officers and a greater or fewer number of officer hours to resolve.

These are important issues not to be ignored, but on their own they don’t undermine the study. However, there is one more, much larger, issue that does call into question the validity of the whole analysis. Each call has two call types associated with it: the one assigned by the 911 dispatcher at origination, and the “final disposition” assigned at the resolution of the call for service. The NICJR team used the final disposition call type for its analysis, because it more accurately describes what actually happened during the call. But this divorces the analysis from reality, because 911 dispatchers don’t have the benefit of hindsight: they don’t make decisions about whether to send out sworn officers or civilian alternatives based upon the ultimate disposition of the call; they have to make it based on the limited information they have when the call comes in. This highlights a critical assumption being made by NICJR: that 911 dispatchers can accurately predict the final disposition of a call at the time it comes in. There are two levels where this is problematic: first, a quick look at the call data shows that a large percentage of the calls change call type between the initial assignment and the final disposition; and second, for call types such as “prowler- trespass” where 1,866 of 40,661 calls resulted in an arrest, dispatchers may not be able to distinguish between when an officer is and is not needed.

To illustrate the first problem, here is a snapshot of all of the calls in the first quarter of 2017 with a final disposition of “Disturbance – other”, which is in the top 10 for On-view, 911, and non-emergency calls and listed as “Tier 1” for all three. You can see the wide variability in originating call type:

Click to access disturbance-calls-Q1-2017.pdf

And here are all of the “Prowler – Trespass” calls; not only do they also have significant variation between initial call type and final disposition, but (in reference to the second problem above) there’s nothing apparent that clearly distinguishes the calls that result in arrests from the rest (though there may be other information that dispatchers have that might help).

Click to access prowler-trespass-q1-2017.pdf

Another example: NICJR categorized alarm calls (including banks, ATMS, schools, and businesses) as “tier 1,” meaning that they would only receive a civilian response at first. While we can grant that many of those calls are false alarms, 911 dispatchers have little ability to predict which ones are false and which are real.

This leads us to the two key questions we need to be able to answer in order to create a real alternative response system:

1. Can dispatchers accurately predict, given the information they have at the time of the call, whether an armed response will be required?

2. What happens when they are wrong?

Answering the first question requires a type of analysis well-known to mathematicians and computer scientists, called “Bayesian inference.” It’s used to build models for a wide variety of diagnostic systems, from healthcare to car repairs. Given information about a set of outcomes (e.g. diseases) and the characteristics we are likely to observe (e.g. symptoms), Bayesian analysis lets us flip that around and predict the outcomes based on the observations. In this case, with a collection of data on the criminal-type calls and the ones that led to arrests (or otherwise required an armed officer), can we accurately predict them from an arbitrary set of information given to a dispatcher?  There is no doubt that a Bayesian model could be built; however, until it’s tried, there’s no way to know whether it will be accurate enough to be useful in making dispatch decisions. It may turn out to be accurate enough for only a subset of call types; that is still useful enough to become a component of a solution, even if it means in the short term only some calls can be diverted to alternate responses.

The second question — what happens when the dispatch decision is wrong — is tougher. SPD was worried enough about that question that it asked a noted academic criminologist, Professor Geoffrey Alpert from the University of South Carolina, to review the NICJR report and provide his thoughts. Alpert gave two pieces of feedback: first, he agreed with NICJR that the tiered-response model is the right approach. Second, he said the analysis and assignment of call types to the four tiers is overly simplistic because it leaves out the notion of risk: what happens when the wrong response is sent to a call.

Alpert recommended that SPD look at parallel work that went into the creation of the Cambridge Crime Harm Index, an effort to improve crime reporting. It recognizes that simple tallies of the number of arrests for crimes doesn’t give a useful picture of crime trends, especially when a petty crime and a more serious, violent crime are counted equally. The Cambridge Harm Index creates a weighted count, assigning to each type of crime a weight corresponding to its seriousness; it uses as a proxy for seriousness the minimum sentencing length for someone convicted of the crime. So in a city where violent felonies are decreasing and misdemeanors are increasing, the Harm Index will likely go down recognizing that the overall burden of crime on the community has decreased.

Alpert is suggesting that a similar index could be used to quantify the risk of harm associated with a call, and also potentially the risk of harm if the wrong response is sent. It would conceivably capture the harm caused by a crime in progress where armed officers were not sent in response and couldn’t intervene to stop it; the potential danger to a civilian responder caught up in a violent incident; and the increased risk of a violent conflict that harms suspects or bystanders if an armed police officer is sent into a situation where his or her presence will escalate tensions.

In its written response to the NICJR report, the executive branch’s inter-departmental team looking at alternatives to policing claim that they are following Alpert’s recommendations and starting to research an appropriate “harm model” that could be incorporated into a tiered dispatch process. It’s important to emphasize, however, that this is breaking new ground: there are no models to copy on how to build such a harm index for this context.

In the meantime, the interdepartmental team has acknowledged that about 12% of SPD’s 911 calls, related to “person-down” calls and lower-priority wellness checks, “can and should be explored for alternative responses starting now.”

Based on an initial cooperative analysis with SFD and CSCC, SPD agrees that 12 percent of calls, can and should be explored for alternative responses starting now. These are calls where we know the risk of harm is very low. This reduction does not equate to – and is not reflective of – the percent of service hours SPD currently spends on calls that could ostensibly be offloaded. These calls include person-down calls and priority three welfare checks which accounted for over 23,000 service hours between 2017 and 2019.

This led to the proposal for a “Triage One” pilot organization that Mayor Durkan included in her budget. The team would be stood up and plugged into the 911 dispatch process over the course of 2022, though it would remain a limited-scale pilot and would not be able to offload the full 12% of calls envisioned until it is proven out and city-wide expansion is funded.

To meet the immediate need for alternative responses, SPD, SFD, and CSCC are proposing a new pilot aimed at helping 9-1-1 triage non-criminal calls with no imminent health concerns. These triage teams will be a civilian forward alternative to fire or police, housed within the Mobile Integrated Health (MIH) program. Triage teams will respond directly to wellness check calls as identified by 9-1-1 dispatchers through collaboration with SFD and SPD. We will work with community organizations to hire people who bring not only expertise in outreach and behavioral health, but also lived experience and a tangible connection to the communities they will serve. They will be equipped with radios to request a police or EMS response as needed. It is expected that SPD will be requested only for criminal situations or to assist with potential violence or active suicidality. On the back end, the teams will be provided with a case manager able to follow up on client referrals and service connections and reduce the chance that they are called in the future.
This triage response team pilot is the first step of many to truly reimagine public safety in Seattle. SPD and The Mayor’s Office are committed to continuing this work to increase the number of available alternative responses and reduce the number and types of calls that are unnecessarily responded to by SPD patrol officers- those with no immediate safety or health risk.

This complicated picture looks very different from the simple talking points on the NICJR report we hear. If 911 dispatchers are clairvoyant, they could send civilian alternative responses to the appropriate set of 911 calls in place of SPD officers; unfortunately they aren’t, and they can’t — yet. And as of now the city has no model for the risk of harm it may be creating for responders and Seattle residents by trying to offload some calls. There is well-founded optimism for what lies ahead, but we’re not there yet. In the meantime, however, there is also great impatience to start offloading SPD calls now, both for the optics of showing meaningful progress on elected officials’ promises to downsize SPD’s role, and also to help address the staffing shortage in the police department.

The good news is that it looks like the budget for the Triage One team is surviving the Council’s budget process, as is SPD’s technology investments to start to build a harm index. So in the next year, we may see the first step in offloading that 12% of calls from SPD, and some groundwork laid for a much larger restructuring of 911 response in Seattle.

I hope you found this article valuable. If you did, please take a moment to make a contribution to support my ongoing work. Thanks!


  1. Two points you might be interested in Kevin:

    First, there is one police alternative that has existed for a long time, in Eugene Oregon called CAHOOTS. It has been studied extensively and people have discussed copying it. But guess what? CAHOOTS almost never deals with crimes. Check out page 4 of this link – a huge chunk of these calls are people calling and asking for a ride. It ends up being only 5-8% of EPD calls:


    Second, any sort of “risk-benefit” model that is done honestly is going to come out suggesting police still be sent to most things. Why? Because SPD virtually never uses force, and as these stats show, generally also avoids making an arrest. SPD’s use of force data shows 2,067 uses of force in 2020, which had many more uses of force due to riots/protests. That’s about 0.5% of all SPD calls for service. There were only 14 uses of deadly force – 0.0035% of all calls for service. What is the risk of an unarmed social worker being severely injured or killed (or failing to prevent someone else from being killed or injured) responding to car prowlers, trespass calls, and suspcious people? We don’t know. But it would have to be insanely low to do better than SPD.

    1. I’m well aware of CAHOOTS — Councilmember Lewis is a big fan and talks about it incessantly.

      While the stats show that SPD officers rarely make arrests, there are still valid reasons why you might want to send someone else. First, sworn offices are in short supply across the nation, and tend to be on the high end of the wage scale compared to other city employees. Second, there are cases where the mere presence of an armed officer can escalate a difficult situation, especially when there is distrust of the police as there often is in BIPOC communities.

      Part of the research SPD is undertaking is to try to answer some of those questions about what the risk is to an unarmed social worker.

  2. Kevin,

    I’ve appreciated your wordy and nerdy posts for quite awhile now, this one included.

    One aspect of this that I wish was included in the discussion, but hasn’t been, is man-hours per call. While the number of man-hours required to handle a given call won’t effect who is responding to said call, it will effect budgets and target staffing levels of the various agencies involved.

    For example, a typical trespass/prowler call takes about half a man-hour (15 minutes for 2 officers), while a DUI arrest takes about 8 man-hours (2 hours for the backing officer and 6 hours for the primary). DV arrest calls similarly take much more man-hours than their relative percentage of the number of calls (4 officers on-scene for a half hour each, plus another hour for the transport officer and another 2 hours for the primary officer for 5 man-hours total).

    The table showing the number of calls of a particular type would look a lot different and tell a much different story if it showed hours worked per call type instead of simply the number of occurrences.

    Thanks for all you do!

    1. John, that is definitely an issue, and one that I brought up very briefly. If you read the inter-departmental team’s response to the NICJR report, they discuss that at some length. I didn’t write more about the issue because there aren’t a lot of conclusions to draw yet, as they need more data and analysis. But yes, the bottom line is that diverting 12% of the calls away from SPD won’t necessarily lead to a 12% reduction in SPD’s needed person-hours. It’s also complicated by the fact that SPD is so short-staffed right now, that the first round of diversions probably won’t result in any reductions in SPD person-hours.

      There is a little bit of discussion of person-hours in the NICJR report too, with a smidgen of data. It also has some data about the average number of officers that respond to a particular type of call. There are structural forces at work too, though, like the fact that the consent decree essentially mandated that a minimum of 2 officers respond to a call; officers never respond alone. That makes it a bit more difficult to determine the potential impact of changes to 911 response.

  3. Thanks as always for the incisive commentary. I just wanted to add something that really bothered me when I read this report. The authors make a bold headline point of saying that 80% of the calls SPD go to are non criminal. Yet if you actually break down those calls individually you see that a great many of them are in fact crimes. Everything from car prowls to burglaries. The labeling is really misleading.

    Now it’s still possible to say those calls don’t need a sworn officer to respond, and I think that’s a valid topic to discuss. However, that is raising a pretty different question. Criminal investigations have always been a central part of policing, and indeed a lot of reformists are calling for recentering policing on that mission. Saying that these calls don’t need a police response raises a big question of who will be going instead. Will these people be conducting criminal investigations? It just seems that the authors largely pretend this question doesn’t exist.

    That brings me to my last point. The NICJR is an activist organization. Their entire stated goal is to reduce and or eliminate traditional policing. Perfectly fine for the city leadership to talk to activists, but to me it raises some pretty serious questions about how much we should simply rely on their findings. They have a vested interest in making the strongest case possible for minimizing police responses, and that’s what they seem to be doing here, even at some cost to the truth.

  4. I’m glad to hear there is some discussion of person-hours. I worry that SCC will use an incomplete analysis to justify further cuts to SPD’s budget.

Comments are closed.