Annual SPD public safety survey provides interesting insights, if you dig enough

Last month the Seattle Police Department’s annual public safety survey report was released. It is a timely reflection back on a tumultuous year for the community’s relationship with the police, as we approach the first anniversary of the death of George Floyd at the hands of Minneapolis police to be followed by months of violent clashes between protesters and SPD on the streets of Seattle and other cities.

The 147-page report provides the results of a survey taken in October and November 2020 of over 11,000 Seattle residents. The results are detailed; the analysis is not. Let’s take a look at what the report says, and then look at what else we can learn from it.

The survey has been conducted annually by Seattle University researchers on behalf of SPD for several years now. There are two parts to the survey that respondents were asked to complete. First, they were asked about specific issues: they could identify specific public safety and security concerns that they had, and they could provide written comments on public safety and security in Seattle and in their neighborhood.  Second, they were asked to provide rankings from 0 – 100 on a number of questions around five themes:

  • police legitimacy: “an acceptance of the rules, laws, and precepts that define the police role in society, and a willingness to grant deference to police as a consequence of the belief that they are the authorized representatives who dutifully carry out the rules and laws that make society function smoothly.” Survey respondents were asked to rate questions such as “Seattle police officers are honest” and “I have confidence in Seattle police officers.”
  • social cohesion: how tight-knit they believe their community is in various dimensions, such as whether it’s a good area to raise children or whether people are generally friendly.
  • informal social control: whether neighbors are likely to intervene when they witness crimes being committed or other bad things happening.
  • social disorganization: the extent to which there is perceived physical and social disorder in the neighborhood. Respondents were asked how often they witness a number of issues, including vandalism, fights on the street, drug sales, buildings with broken windows, graffiti, and litter.
  • fear of crime: respondents were asked to rate how often they worry about a number of possible events, including someone breaking into your home or vehicle to steal from you, or being physically attacked or sexually assaulted.

These five themes are pulled directly from the research literature around policing, crime, and public safety, and tie to hypotheses around what drives both fear of crime and actual crime — as well as residents’ perceptions of the police. The classic “broken windows theory” holds that signs of social and physical disorganization (often simply called “disorder”) are triggers to criminals that an area is attractive for committing crimes, as well as triggers that lead residents to fear crime more. On the other hand, critics of broken windows theory argue that disorder is only a cue when the “collective efficacy” of a community has broken down, where collective efficacy is a combination of social cohesion and informal social control; in other words, tight-knit communities where neighbors support each other and intervene when things go wrong have low crime even in the presence of high physical disorder. Here is SCC Insight’s previous article on understanding “broken windows” theory; it’s a helpful refresher before diving into the public safety survey since the concepts surveyed are deeply connected to it.

The report provides the top themes and concerns for the city as a whole, for each of the five SPD precincts, and for each neighborhood in the city.

For the city as a whole, the biggest surprise this year — especially when compared to previous years — is that issues around legitimacy and trust of SPD did not pop to the top of the list, while police capacity (specifically, the lack of it) and “city politics” related to funding and managing the police did. One might have expected a referendum on SPD given the events of the past year; if anything, it’s the reverse: city officials’ perceived mismanagement of SPD is apparently much more top of mind. This is equally reflected in the city-wide averages for the five themes listed above, in which police legitimacy dropped only slightly:

When reading the numbers keep in mind that for the first three, police legitimacy, social cohesion, and informal social control, a higher number is better, and for social disorganization and fear of crime, a lower number is better.  Also, the researchers polled separately on fear of crime during the day and at night, and combined them to create a single “fear of crime” metric.

Looking at the longer-term trends, it’s important to note that all of the metrics clump into the middle of the range (which may in part be an effect of the way they are calculated as an average of several questions), and none of them have seen dramatic movement over time. Police legitimacy has dropped from the low 60’s to the high 50’s over the past two years; social cohesion has dropped a bit, while informal social control dropped a bit more; social disorganization has steadily but slowly increased since 2016 (though is still low); and fear of crime is basically unchanged over the past five years.

The city-wide numbers hide a tremendous amount of variation within neighborhoods. It’s unclear why the survey bothers to report out statistics by police precinct, when the underlying data shows large variability within each precinct. For example, here is the rankings by neighborhood on police legitimacy, color-coded by precinct:

Similarly, here are the numbers for social disorganization, where there is even more variation:

With this much variation within each precinct, the precinct-level data not only doesn’t tell us anything useful about what is going on, but in fact makes it harder for us to see any underlying patterns. So from this point on in our analysis, we’re simply going to ignore the precinct boundaries and deal directly with the neighborhood-level data.

A housekeeping note is in order before we proceed further: not all neighborhoods in the city responded to the survey at equal rates. There are over 1,300 responses from Capitol Hill and almost 600 from Belltown, but only 13 from Pigeon Point and 19 from Miller Park. The researchers appropriately weighted the total survey responses to match the city’s overall demographics, but when we look at the neighborhood-level data itself, the ones with very low response rates simply can’t be trusted to be statistically representative of the residents of those neighborhoods — especially when we keep in mind that opt-in surveys such as this one tend to bias toward the negative because people with strong feelings opt-in at higher rates. As a result, I’ve dropped from the neighborhood-specific analysis the neighborhoods with fewer than 50 responses. That still leaves 46 neighborhoods to look at, but you might not find a specific one that you’re looking for if there weren’t enough responses. I am happy to give all of the data I scraped out of the report, including the neighborhoods with low response rates, to anyone who would like to crunch the numbers themselves. The full survey report also contains the data for every neighborhood where there were survey responses.

Looking at the bar charts above, one gets the nagging feeling that there is a pattern to the data but that we’re not looking at it the right way to see it (spoiler: there is, and we’re not). Where we see a range of results, we want to look for two things:  correlations between the different metrics, and clusters of neighborhoods that responded the same way (regardless of where they sit geographically).  To do that, we’re going to look at scatter-plots, which allow us to look at two dimensions of data; and bubble-plots, which show us three dimensions.

Within the realm of policing and public safety in Seattle, here are three questions that we can ask that the survey didn’t provide answers for:

  1. What correlates with police legitimacy?
  2. What correlates with fear of crime?
  3. What’s happening in Seattle’s neighborhoods where the percentage of Black residents is higher?

For the first two questions, it’s important for us to distinguish “correlation” with “causation.” Where we find connections between two metrics, the data isn’t going to tell us whether one is causing the other, whether it’s coincidence, or whether both are being driven by a third, independent factor. Those questions will require future research to fully understand.

I added the third question specifically because the disproportionate impact of biased and violent policing on Black communities in Seattle has been a near-constant topic since last summer, and yet it’s unclear who speaks for the beliefs and experiences of Seattle’s Black community — or for that matter to what extent those beliefs and experiences attributed to the Black community are widely held. The survey gives us an opportunity to let Seattle’s Black communities speak for themselves in large numbers.  To that end, I gathered from the City of Seattle’s web site the statistics on the percentage of each neighborhood’s population that is Black, and added that to the survey’s data set; it was challenging since the neighborhood boundaries in the survey don’t precisely match the city’s demographic information, and there were a few more neighborhoods where I couldn’t identify trustworthy statistics and had to omit them from the parts of the analysis below related to the Black community. But there are still plenty of data points for us to use for our analysis.

Police legitimacy

The first question is: what correlates with police legitimacy in Seattle? This is likely to provide a messy answer because there are clearly many factors that might factor into one’s viewpoint on SPD, such as personal history of interactions with the police, fear of crime and the perceived need for a police presence, national and local news reports of police misconduct, as well as news reports of heroic actions by officers. Let’s start with a straightforward one: fear of crime. Here’s a scatter-plot of the survey data comparing how neighborhoods rated police legitimacy with their perceived fear of crime:

(A note on this and the other graphs below: I did something here that I usually rail against, in that I used a non-zero baseline (the x-axis goes from 45 to 80 instead of 0 to 80, and the y-axis runs from 40 to 70). As we previously discussed, nearly all the rankings, even at the neighborhood level, are compressed into a fairly narrow range. I’m essentially zooming in on the center of the chart, which has an effect of magnifying differences potentially more than they are actually meaningful. I feel more comfortable doing it here because we’re not looking at trends, or even really trying to measure differences: we’re looking for correlations and clusters. But it is a practice that should be used sparingly and disclosed prominently when it is — as I’m doing here.)

The first and most obvious thing we notice about this graph is that there are no data points in the upper-left corner: high fear of crime, low police legitimacy. This could mean two things: if you hold SPD in low regard, then you don’t fear crime; and/or if you fear crime, then you think more highly of the police. We can also think about what the other three corners might mean for the clusters of neighborhoods in each:

  • bottom left, low fear of crime, low police legitimacy: neighborhoods essentially saying “we’ve got this on our own.”
  • bottom right, low fear, high legitimacy: “law and order” neighborhoods that feel safe to the residents but who might traditionally respect the police and other authorities. It’s not surprising to see some of Seattle’s richer (and whiter) neighborhoods here, such as Magnolia.
  • top right, high fear of crime, high police legitimacy: neighborhoods that perceive a need for the police. Some of the neighborhoods that have recently seen increases in a mix of visible homelessness and crime show up here, including SODO, Georgetown, and downtown.

We need to keep in mind that “fear of crime” is not the same as actual crime rates, so neighbors’ perception of overall crime (or the threat of it) may drive feelings toward police independent of what the crime statistics say. We should also note that in this chart there is a very large cluster of neighborhoods in the middle, and it’s not clear what that means. Overall, there isn’t a strong correlation between fear of crime and police legitimacy, though there are specific circumstances where it seems to be a factor.

How about informal social control? This is a graph with two very strong signals:

The first one is the line from the bottom-left to the top right: that tells us that there is a strong correlation between informal social control and police legitimacy. But it’s also counter-intuitive: you might think that the neighborhoods with strong informal social controls would be comfortable enough with their own ability to manage public safety that they were more willing to have a critical view of the police. We could hypothesize all sorts of reasons why this correlation exists, but the bottom line is that it needs more research.

The second strong signal in this graph is the cluster of neighborhoods along the bottom of the chart with very low ratings for informal social control. These are all neighborhoods with very pronounced and visible homeless populations. They cover the spectrum of police legitimacy. So while informal social control often ties very closely to police legitimacy, it appears that there are overriding factors.

Next let’s look at social cohesion, the other half of “collective efficacy.”

It’s hard to see much of anything here — the data points don’t line up, and there aren’t clear clusters. Pretty clearly social cohesion of a neighborhood doesn’t correlate with police legitimacy.

Finally, let’s look at social disorganization. Here we’re going to switch to a bubble chart, which will help us to clearly see the three clusters. The larger the bubble, the greater the percentage of the population that is Black.

In the bottom-right corner, we see a cluster of rich white neighborhoods. In the top-right, we again see a cluster of neighborhoods with pronounced, visible homeless populations. And in the bottom-left we see two overlapping clusters: the city’s Black neighborhoods, and some of the city’s rich but self-styled “woke” neighborhoods such as Fremont, Wallingford, Roosevelt/Ravenna, Greenwood, the U District, and Phinney Ridge — with Capitol Hill right on the edge of the cluster.

As we saw with fear of crime, there are no neighborhoods in the top-left corner: high social disorganization and low police legitimacy don’t mix. But overall this chart is more about clusters than correlations: social disorganization and police legitimacy can be related, but there seem to be overriding factors at play.

But let’s not miss the other big insight from this chart:  all of the neighborhoods with the largest Black communities rated themselves as low social disorganization and low police legitimacy. We didn’t see that kind of consistency with white neighborhoods, or rich neighborhoods. So let’s jump to our third question and look at the full picture of what the survey tells us about Seattle’s Black communities and how they see themselves.

Black communities

As we can see from the chart below, and as we talked about above, all of the Seattle neighborhoods with substantial Black communities rated low on police legitimacy. Or at least on the low end of the Seattle scale: the lowest is Judkins Park/North Beacon Hill at 46.8 out of 100.

As for the elements of “collective efficacy,” the Black communities all rated themselves high on social cohesion, but less consistently (or perhaps simply more in the middle) on informal social control.

They all self-rated on the low end of the scale for social disorganization, as mentioned above.

And they also rate low on fear of crime.

At first glance, it looks like all the other neighborhoods cover the entire spectrum of ratings. However, on three of them — social cohesion, social disorganization, and fear of crime — you can see that the neighborhoods with the large and highly visible homeless communities are outliers: remove them from the graph, and the range of responses for the neighborhoods with larger Black communities is not that different from the range from the rest of the neighborhoods in the city. And therein lies another important observation: none of the neighborhoods with the largest Black populations are the neighborhoods where we hear the most about problems with the homeless population.

This raises several important questions that need further exploration:

  • Why is the proportion of the population that is Black such a strong predictor of social cohesion, social disorganization, and fear of crime?
  • Why is there no overlap between the neighborhoods with significant Black populations and the neighborhoods where there is the greatest conflicts with homeless encampments?
  • Are there similar patterns for Asian and Hispanic/Latino/Latina demographics, or is there something unique about Seattle’s Black communities?

Fear of crime

And that brings us to our final question: what correlates with fear of crime?

(aside: even though the survey splits out daytime and nighttime crime, the responses on the two are extremely correlated; people just rank fear of nighttime crime a little higher. So we’re going to ignore the distinction and just look at the combined metric.)

Let’s start by looking at social disorganization, and to no one’s surprise, it is strongly correlated to fear of crime.

This goes to the heart of “broken windows” theory: people associate crime with disorder.  But at the same time, this is a really enlightening chart about Seattle: once again, the top-right quadrant contains the neighborhoods where we hear the most about issues with homeless encampments. If you remove those from the charts, all the rest of the neighborhoods are below 50 on both fear of crime and social disorganization.

When we look at informal social control, we see the inverse correlation we might expect: the more informal social control, the less fear of crime. But again, the city’s Black communities seem to be forging their own path; informal social control is clearly not the driving force behind their low fear of crime.

On the other hand, social cohesion is a very strong predictor of fear of crime (inversely correlated) across the entire city. But once again, if you remove the neighborhoods that are most associated with problematic homeless encampments (in the top left), what remains is a cluster in a very narrow range: above 50 on social cohesion, below 50 on fear of crime.


So what did we learn? There are a few big lessons:

  1. We should be ignoring precinct boundaries. They tell us nothing interesting or useful. And to the extent that SPD is using precincts to differentiate its approach to policing, it is making a mistake. SPD publishes micro-community policing plans, which are supposedly based in part on the neighborhood responses in the annual survey, but that might be too granular — there are clearly clusters of neighborhoods, not necessarily adjacent to each other, that have similar views on community safety and SPD could be working with them as larger units.
  2. Future surveys should overlay race and ethnicity data on top of the data they are currently collecting. It certainly shows that those communities are uniquely situated in their views on public safety; SPD needs to integrate that information, but the city government as a whole needs to be considering that as it looks at community-based investments outside of policing — because crime is apparently not central to how Seattle’s Black communities think about public safety.
  3. On the other hand, homelessness is clearly having a profound effect on how neighborhoods view public safety, and particularly those neighborhoods where we see and hear the most about problematic homeless encampments. Now we need to be careful about cause and effect here: is fear of crime higher because of problematic homeless encampments, or are we hearing more complaints about homeless encampments in the places where the fear of crime is higher?  Or is it a vicious cycle where they feed each other?
  4. There are distinct clusters of neighborhoods in Seattle with very different views on police legitimacy, for their own reasons. That means that there is no one model for improving it. At the same time, police legitimacy has not plummeted in the last year, as many had predicted, and it’s notable that overall in Seattle police capacity and the influence of city politics on the police department raised more concerns than police misconduct.
  5. The neighborhoods with the largest Black communities are not the ones where we hear the most about problems with homeless encampments. We need to understand why that is the case.
  6. The survey report provides a bit of validation for “broken windows” theory, in that it shows that people associate disorder with crime. The report also provides a partial validation of the opposing theory that “collective efficacy” is more important than perceived disorder: increased social cohesion was strongly tied to low fear of crime, but informal social controls were less correlated.  But there are two important caveats to this conclusion. First, actual crime and fear of crime are not the same, and perceived disorder can drive up fear of crime even when the reality is quite different. Second, even if the general “broken windows” theory holds true that people associate disorder with crime, the reflexive theory of “broken windows policing,” that cracking down on small affronts to order such as graffiti and panhandling will lower overall crime rates, has not generally held to be true. Yet belief in it has led to epidemics of biased and racist over-policing over the past three decades. The community safety report is neither a recommendation nor an endorsement of broken windows policing tactics in Seattle.

I hope you found this article valuable. If you did, please take a moment to make a contribution to support my ongoing work. Thanks!