Earlier this month King County released its annual “Point in Time” count of homeless persons in the county, as required by the federal Department of Housing and Urban Development (HUD) as a condition of receiving grant funding.
Last year, SCC Insight reported on the significant methodology issues with the count that rendered its year-over-year trend data unusable. This year those same issues exist, plus some new ones. However, that doesn’t mean there’s nothing to learn from the new report; we just need to dig harder.
The Point in Time count is a collection of surveys used to try to get a complete picture of the county’s homeless population at a single point in time, including:
- a count of individuals living unsheltered on a single night (this year it was January 24, 2020), conducted by sending volunteers and paid “guides” with lived experience of being homeless out into every part of the county in the early hours of the morning to find and count homeless people;
- a count of youth and young adults living unsheltered, conducted by sending volunteers and guides out midday (also on January 24). Historically, youth have been undercounted because they are often adept at “hiding in plain sight” and crashing on friends’ couches or floors overnight, so the survey method has been adapted to better find and count them;
- a count of homeless individuals and families who are sheltered that night in one of the many facilities offering shelter of various forms in the county. 85% of the data is pulled directly from the county’s central HMIS data system, for the agencies that use it; the other 15% of the data comes from agencies that don’t participate in the HMIS system; their agency staff are surveyed directly to obtain the count data.
- A post-count survey, conducted in the month following the one-night count, to gather more in-depth data on demographics, causes of homelessness, needs and issues.
In last year’s survey, there were two glaring signs that the methodology was flawed. First, the number of people (volunteers and guides) sent out to look for unsheltered homeless people changed from year to year — and the 2017-2019 data shows (not surprisingly) that the number of homeless people found roughly correlates with the number of people looking.
Second, there were numerous wild swings in the post-count survey results that strongly suggest sampling bias in who was actually surveyed. For instance:
This year a new company, VN Research, took over the contract to implement the survey on behalf of the county. Fortunately, it realized the importance of maintaining consistency in methodology, and so it sought to tweak things rather than make wholesale changes (though it recommended some larger changes for the oversight board to consider). However, circumstances also conspired against it: unlike the previous three years, in the early hours of January 24th, a heavy rainstorm hit the Seattle area, dropping 1.14 inches of rain. Likely because of this, the number of volunteers who showed up to count that night was significantly lower than in past years. VN also believes that the heavy rainfall made it harder to find unsheltered homeless individuals, and may have shifted more people on that night into shelters. In addition, earlier that week there was a shooting incident in downtown Seattle and a major homeless encampment cleanup effort that may have discouraged some volunteers from participating and/or caused more unsheltered individuals to move out of the immediate area. As a result, the number of counters dropped from 602 in 2019 to 335 this year.
VN Research also made a few methodology changes that probably affected the count results. For unsheltered counts, instead of asking volunteers and guides to cover all the roads in a given sector of the county, it selected routes for them where it thought the probability was highest that homeless individuals would be found. Also, it worked to improve cooperation with homeless service provider agencies in the county to obtain better and more complete information.
Together, this means that there is no reliable year-over-year trend data on the number of unsheltered individuals in King County — and thus no reliable trend data on the total number of homeless individuals. So when VN Research shows us this chart:
They are really saying that there were at least that many homeless individuals in King County on the night the count was done. Their claim that 11,751 is an estimate of the entire homeless population is not to believed, nor is their claim that the population increased by 5% from the previous year. We really have no idea what the total size of the homeless population is, for any of these years. It could be substantially larger, and for 2020 it very likely was. If they had at least kept the methodology and the size of the counting crew consistent, we could assume that the undercount was consistent across years. But they didn’t (though they were at least partially the victim of circumstances) so we have no idea what the trend really is.
The post-count survey also continues to be problematic. The good news is that they collected a large number of surveys (832), which improves reliability. Also, they pull much of the information directly from the HMIS system, which improves accuracy. The bad news is that they are reliant on homeless service providers to conduct the surveys, and they have no means to ensure consistency across providers as to who is being asked to take the survey, so the results are still at high risk for sampling bias. In fact there continues to be evidence of such bias. This year, the total number of surveys collected was much lower (VN says in the report that many of the service providers were unaware of the survey and focused their efforts on the one-night count). That increases the overall margin of error, as well as the magnitude of any sampling bias introduced. VN claims that for population-wide demographics, the margin of error is +/- 3.3% at a 95% confidence level. For subgroups, however, the margin of error will be much greater (VN Research doesn’t estimated the margin of error for subgroups in its report). Getting 832 survey responses from 11,751 people is, on one hand, really great and if the sample was truly random it would have excellent statistical power.
On the other hand, there is still plenty of room for sampling bias, and we can see that there are still wild swings for several of the demographics that suggest there is indeed bias. For instance, Native Americans: 3% of the counted population in 2018, and 15% this year.
We know a bit of the story here: it was widely believed that Native Americans have been dramatically undercounted for years, and in 2019 an effort was started to ensure that the Point in Time count did better at finding and counting them by collaborating with agencies that provide services to that community. In part, it was believed that the “multiple races” category included a large number of Native Americans. The effort is clearly paying off: we can see that the Native American count is way up (now 15% of the total) and the “multiple races” category has dropped accordingly.
There are several other categories, however, where the swings don’t have clear explanations. A prominent one is age, where the “under 18” population jumped from 1% to 19% in a single year, the 18-24 group dropped from 18% to 9%, and the 41-50 group dropped from 27% to 15%. The 41-50 group continues to oscillate dramatically year after year.
The under-18 numbers tie in with a dramatic increase in individuals in family households this year:
Again, this suggests changes in sampling, rather than dramatic changes in the underlying demographic.
When looking at the local where individuals were staying that night, we see a big drop in “outside” and a commensurate increase in people in abandoned buildings. That could simply be the heavy rain that night. It doesn’t mean that the numbers are wrong, but it means that we can’t rely on trend information. The number of people living in vehicles jumped back up again, though still well below the 2018 count; again, it should make us suspicious of the sampling methodology.
Other big swings: the number of people for whom this was their first time experiencing homelessness,
and the number of individuals fleeing domestic violence:
In a year (2019) where very little changed in the way the city responds to homelessness, it’s difficult to believe that there could be this many underlying demographic changes. And remember, the count and survey were performed in January and February, before COVID-19 struck. The overall accuracy may in fact be improving, but we can’t rely on that until the numbers stabilize across years.
But before we discard the report entirely, there is one group represented here that we can learn from: the sheltered population. Remember: they are counted using a completely different process than unsheltered individuals, a process that relies on the county’s HMIS system to provide data on 85% of the individuals, with the rest of the data coming from the providers operating the shelters that don’t participate in the HMIS system. There are no volunteer counters, and the list of providers is well-known. So year over year this is a data set that is much more consistent and reliable. It doesn’t give us big insights into the homeless population as a whole — we can’t assume that the people willing to enter shelters have the same demographics as the people living unsheltered — but we can learn something about who is in the shelter system and ther needs, and potentially gain some insights as to the efficacy of the county’s efforts to shelter the homeless. The Point in Time report gives the raw numbers for the sheltered population, but doesn’t do any analysis, so it’s time to pull out that copy of Excel and go to work.
The total number of homeless individuals sheltered in King County dipped a bit in 2018, but has risen back up to the 2017 levels, and as we know from city and county reports the shelter system is mostly full to capacity.
But the interesting story is who is getting into the shelters. After two years of decline, the number of family households in shelters increased a bit again this year. The number of unaccompanied youth and young adults saw a more substantial jump up this year as well. But the big story is the dramatic increase since 2018 of the number of chronically homeless individuals in shelters — nearly doubling. That is almost certainly very good news: the chronically homeless population is the most challenging to serve, and if more of them are coming inside, then it portends well for the county’s ability to address their needs and help them to get stabilized.
But since the total number of individuals in shelters is approximately flat, this raises an important question: if these groups went up, who went down? Who is there fewer of in shelters? The data in the Point in Time report doesn’t break out the size of the sheltered population for all subgroups that have been the focus of attention, such as veterans and those experiencing homelessness for the first time. But it does give us other demographic information, such as race/ethnicity, age and gender:
Unfortunately, it doesn’t give us a clear answer: there are small increases and decreases, but nothing that correlates with the big increase in chronically homeless in the past two years. That suggests (but far from confirms) a couple of hypotheses: perhaps the sheltered population has remained the same but now more of them are categorized as chronically homeless; or the decrease has been among individuals who don’t fall into a particular subgroup that is a special focus of attention for homeless service providers. But it also tells us that the increase in chronically homeless has not introduced new racial/ethnic biases — though by the same token, nor has it addressed any of the existing ones.
Finally, broken out by region of the county we can see some trends about where additional shelter capacity is being created and/or utilized:
The eastern part of King County is sheltering slightly fewer individuals, as is the southwest area. The northeast is building back up after a two-year drop-off, and the southeast corner is growing its share. But the biggest player by far, to no surprise, it still Seattle, providing over two-thirds of the shelter. Seattle is now back above its 2017 total as well. It will be interesting to see what impact the new regional homeless authority has on these numbers over the coming years.
What are the big take-aways?
- You still shouldn’t believe in the accuracy of the total homeless count, and especially not the quoted 5% increase in 2020. There were at least 11,751 homeless individuals in King County in January, and probably many more than that.
- The survey as a whole still suffers from methodology issues and sampling bias, though the numbers for the sheltered population are more reliable.
- The sheltered homeless population in Seattle seems to have shifted demonstrably over the past two years. Now one third of the sheltered individuals are chronically homeless.
- Outside of Seattle, there continues to be wide variation in how communities are stepping up to provide for the homeless population.
I hope you found this article valuable. If you did, please take a moment to make a contribution to support my ongoing work. Thanks!