Research Design,

Data Collection,

and Analysis Plan

The Family Options 12-Year Study

U.S. Department of Housing and Urban Development | Office of Policy Development and Research

Research Design, Data Collection, and

Analysis Plan

The Family Options 12-Year Study

ed to:

U.S. Department of Housing and Urban

Development

Office of Policy Development & Research

Submitted to:

U.S. Department of Housing and Urban Development

Office of Policy Development & Research

Prepared by Abt Associates:

Scott R. Brown

Lauren Dunton

Daniel Gubits

Debra McInnis

Brenda Rodriguez

Marybeth Shinn

Michelle Wood

May 4, 2022

C O N T E N T S

CONTENTS

1. Introduction ........................................................................................................................... 1

1.1 Background on the Family Options Study ...................................................................... 1

1.2 Organization of Document .............................................................................................. 2

2. Previous Findings from the Family Options Study ............................................................ 3

2.1 Key Findings from the 20- and 37-Month Analyses ....................................................... 3

Program Use After a Stay in Shelter ....................................................................... 3

Outcomes for Families Assigned to Usual Care ...................................................... 3

Insights from 80 In-Depth Interviews with Families ................................................. 3

Findings from the Impact Analysis Comparing the Randomized Groups ................ 4

Costs of the Interventions ........................................................................................ 5

2.2 Creating a Research Platform for Learning About Family Homelessness ..................... 5

2.3 Rationale for 12-Year Followup on Family Options Study .............................................. 6

3. Research Questions for the 12-Year Study ........................................................................ 7

3.1 Literature Review and Rationale .................................................................................... 8

Research Question 1: Outcomes Over 12 Years for Families Not

Randomized to SUB ................................................................................................ 8

Research Questions 2a/c: Impact of SUB on Housing Stability and Family

Well-Being ............................................................................................................. 10

Research Question 2b: Impact of SUB on Children as They Age ......................... 12

Research Question 3: COVID-19, Housing Stability, and Health .......................... 15

3.2 Data Sources ............................................................................................................... 15

4. Administrative Data Sources ............................................................................................. 17

4.1 Currently Funded Administrative Data Sources ........................................................... 17

IMS/PIC and TRACS ............................................................................................. 18

NDNH .................................................................................................................... 18

4.2 Optional Administrative Data Sources .......................................................................... 18

IMS/PIC and TRACS Data on Adult Children ........................................................ 19

Additional NDNH Data Collection .......................................................................... 20

National Student Clearinghouse Postsecondary Education Data ......................... 22

Research Data Assistance Center for Medicaid Claims ........................................ 23

National Center for Health Statistics for Mortality Data ......................................... 24

5. Primary Data Collection ...................................................................................................... 26

5.1 Adult Head of Household Survey ................................................................................. 26

Survey Topics ........................................................................................................ 26

Parent on Child Modules ....................................................................................... 28

Sample .................................................................................................................. 28

Informed Consent .................................................................................................. 29

5.2 Child Survey (Children Ages 10-17) ............................................................................. 29

Survey Topics ........................................................................................................ 29

Sample .................................................................................................................. 30

Informed Consent .................................................................................................. 30

Family Options 12-Year Study RD/DCAP

May 4, 2022 ▌ii

C O N T E N T S

5.3 Adult Child Survey (Ages 18-30) .................................................................................. 30

Survey Topics ........................................................................................................ 31

Sample .................................................................................................................. 31

Informed Consent .................................................................................................. 31

6. Survey Administration ........................................................................................................ 32

6.1 Participant Tracking and Re-Engagement ................................................................... 32

6.2 Timing of Survey Activities ........................................................................................... 33

Administering the Adult Head of Household Survey.............................................. 34

Administering the Child and Adult Child Surveys .................................................. 35

Incentives .............................................................................................................. 35

6.3 Monitoring Survey Data Collection ............................................................................... 35

6.4 Data Collection Metrics ................................................................................................ 36

7. Analysis ............................................................................................................................... 37

7.1 Impact Comparisons .................................................................................................... 38

7.2 Outcomes and Samples ............................................................................................... 40

7.3 Descriptive and Impact Analyses ................................................................................. 46

7.4 Moderation of Impacts (Subgroups) ............................................................................. 47

7.5 Sample Sizes and Statistical Power ............................................................................. 49

7.6 Cost and Non-Experimental Analysis ........................................................................... 49

Cost Analyses ....................................................................................................... 50

Non-Experimental Analyses .................................................................................. 51

7.7 Approach for Developing a Unified Dataset ................................................................. 51

References ................................................................................................................................. 55

Appendix A. Family Options Study as a Platform for Research ........................................... 59

Appendix B. Administrative Data Elements ........................................................................... 67

Family Options 12-Year Study RD/DCAP

May 4, 2022 ▌iii

C O N T E N T S

List of Exhibits

Exhibit 1-1. Summary of 20- and 37-Month Impact Results ......................................................... 4

Exhibit 3-1. Research Questions, Hypotheses, Rationales, and Analyses ................................... 7

Exhibit 3-2. Data Sources, by Research Question and Funding Status ..................................... 16

Exhibit 4-1. Summary of Currently Funded Administrative Data Sources and Key Outcomes ... 17

Exhibit 4-2. Summary of Optional Administrative Data Sources and Potential Outcomes ......... 18

Exhibit 4-3. Timeline of Data Holds for Family Heads Only vs. Expansion to Include Adult

Children ................................................................................................................... 20

Exhibit 5-1. Summary of Adult Head of Household Survey Instrument ...................................... 26

Exhibit 5-2. Summary of Child Survey Instrument ...................................................................... 29

Exhibit 6-1. Re-Engagement and Survey Activities, OMB Requirements, and Timing ............... 32

Exhibit 6-2. Summary of Primary Data Collection ....................................................................... 33

Exhibit 6-3. Incentives for the Family Options 12-Year Study ..................................................... 35

Exhibit 6-4. Data Collection Process .......................................................................................... 37

Exhibit 7-1. Current Use and Any Use Since Baseline of Long-Term Rent Subsidies as of March

2019 and December 2021 Based on HUD IMS/PIC and TRACS Administrative

Data, by Assignment Condition and Select Comparison Groups ............................ 39

Exhibit 7-2. Outcomes Hypothesized to Be More Sensitive to Current vs. Cumulative

Rent Subsidy Use ................................................................................................... 40

Exhibit 7-3. Outcomes for Planned Analyses for Family Heads and Minor Children .................. 41

Exhibit 7-4. Outcomes for Planned Analyses with Adult Children ............................................... 45

Exhibit 7-5. Minimum Detectable Effects for Confirmatory Homelessness Outcome for SUB vs.

Al Other Conditions, By Response Rate ................................................................. 49

Exhibit 7-7. List of Planned Public Use Data Files and Updates to Files .................................... 52

Exhibit 7-8. List of Planned Restricted Access Data Files and Updates to Files ........................ 53

Exhibit 7-9. List of Planned PII Data Files and Updates to Files ................................................. 54

Family Options 12-Year Study RD/DCAP

May 4, 2022 ▌iv

1. Introduction

This document is the combined Research Design and Data Collection Plan and Analysis Plan for Contract GS-00F-252CA, Task Order #86614621F00014, the Family Options 12-Year Study.

The Family Options Study provides an unparalleled research platform for studying sheltered family homelessness. The 20- and 37-month impact analyses found that priority access to a long-term rent subsidy such as a housing choice voucher produced substantial benefits for families who had experienced homelessness. The 12-year analysis will measure the long-term impacts of access to a long-term rent subsidy to determine whether the benefits observed earlier continue or whether new findings emerge. The 12-year analysis will estimate returns to homelessness, other aspects of family well-being, and the life trajectories of children who have aged into adulthood during the 12 years since they experienced an episode of sheltered homelessness with their parents.

To support these impact estimates and other analyses, the Abt study team will collect and analyze new primary data plus extant administrative data. The primary data collection will focus on three groups of respondents:

1. Adult heads of households in the study families.

2. Children ages 10–17.

3. Adult children ages 18–30.

As currently planned, the administrative data will be from two sources:

1. HUD’s Inventory Management/Public Housing Information Center (IMS/PIC) and Tenant Rental Assistance Certification System (TRACS) for receipt of housing assistance.

2. National Directory of New Hires (NDNH) for quarterly wages and employment.

Under optional contract tasks, the study team would analyze additional administrative data, such as NDNH data for adult children, Medicaid data for health outcomes, National Student Clearinghouse (NSC) for postsecondary educational outcomes, and the National Center for Health Statistics (NCHS) for mortality data.

Another goal of the Family Options 12-Year Study is to expand the sample of study participants who have consented to provide their personally identifiable information (PII) to HUD, creating opportunities for future research. During primary data collection with adult heads of household, study interviewers will attempt to obtain this consent for the portion of the study sample who did not give consent previously or who gave consent only through December 2021.

1.1 Background on the Family Options Study

HUD sponsors the Family Options Study to develop evidence about which types of housing and services interventions work best for families who experience homelessness. The study compares the effects of three active interventions—a long-term rent subsidy, a short-term rent subsidy (rapid rehousing), and project-based transitional housing—with one another and with the usual care available to families who experience homelessness.

Family Options 12-Year Study RD/DCAP

May 4, 2022 ▌1

C H A P T E R 1 : I N T R O D U C T I O N

From September 2010 through January 2012, 2,282 families enrolled in the Family Options Study across 12 communities after spending at least 7 days in an emergency shelter. 1 After providing informed consent and completing a baseline survey, the families were randomly assigned to one of four groups:

1. SUB (subsidy): families received priority access to a long-term rent subsidy, typically a Housing Choice Voucher (HCV).

2. CBRR (community-based rapid rehousing): families received priority access to a short-term rent subsidy, lasting up to 18 months, in the form of community-based rapid rehousing assistance.

3. PBTH (project-based transitional housing): families received priority access to a temporary, service-intensive stay, lasting up to 24 months, in a project-based transitional housing program.

4. UC (usual care): families received access to usual care homeless and housing assistance but did not have priority access to any particular program.

In the first 3 years after random assignment, the study team attempted to contact all enrolled families approximately every 3 months, using a combination of phone calls, letters, and information from national change of address searches and other passive tracking that does not involve direct contact with families in the study. The team completed brief tracking surveys 6, 12, and 27 months after random assignment and conducted extensive followup surveys 20 and 37

months after random assignment. An additional attempt to contact families and administer a brief tracking survey 78 months after random assignment yielded completed tracking surveys with 49

percent of the study sample and viable locating information for an additional 26 percent.

These results laid the foundation for this 12-Year Study, suggesting it is feasible to collect additional primary data from the study sample.

1.2 Organization of Document

The remainder of this Research Design/Data Collection and Analysis Plan is organized as follows. Chapter 2 summarizes previous findings from the Family Options Study. Chapter 3

discusses the research questions and hypotheses that guide this 12-Year Study and the data sources. Chapter 4 discusses administrative data; and Chapter 5 describes the sample and presents the survey topics, and Chapter 6 discusses plans for collecting primary data. Chapter 7

details the study’s analysis plans and outcomes. Chapter 7 also presents plans for developing the study’s unified dataset.

1 The 12 communities participating in the study are Alameda County, California; Atlanta, Georgia; Baltimore, Maryland; Boston, Massachusetts; Bridgeport and New Haven, Connecticut; Denver, Colorado; Honolulu, Hawai’i; Kansas City, Missouri; Louisville, Kentucky; Minneapolis, Minnesota; Phoenix, Arizona; and Salt Lake City, Utah.

Family Options 12-Year Study RD/DCAP

May 4, 2022 ▌2

2. Previous Findings from the Family Options Study

This chapter summarizes key findings from previous phases of the Family Options Study.

2.1 Key Findings from the 20- and 37-Month Analyses

In addition to evidence about the relative impacts of alternative housing and services interventions, the Family Options Study offers lessons about the experiences of families who do not receive a special offer of assistance after a stay in shelter. In documenting the types of programs families used, the study provides insights about the homeless service system.

The experimental design of the Family Options Study provides a strong basis for conclusions about the relative impacts of the interventions on families’ housing stability and many other aspects of well-being at both 20 and 37 months after enrollment. The study examined outcomes in five domains: (1) housing stability, (2) family preservation, (3) adult well-being, (4) child well-being, and (5) self-sufficiency. The large and favorable impacts of priority access to a long-term rent subsidy in reducing homelessness support the view that, for most families, homelessness is a problem of housing affordability that can be remedied with long-term rent subsidies. In this section we summarize key findings from the 20- and 37-month analyses.

Program Use After a Stay in Shelter

The study team used all available sources of data from HUD’s IMS/PIC and TRACS records, local Homeless Management Information Systems, programs, and the families themselves to ascertain, to the best of our ability, where families stayed in each month for the first 3 years of the study. Use of long-term rent subsidies gradually converged for the four study groups, as some families initially assigned to CBRR, PBTH, or UC secured rent subsidies over time; but important contrasts remained at 37 months. Fully 88 percent of families offered a long-term subsidy used one at some time during the 3 years, as did more than one-third of each of the other three study groups. The mix of long-term subsidy types used varied by group. Families assigned to the SUB intervention were most likely to use HCVs (83 percent), whereas families assigned to the other interventions who secured long-term rent subsidies were about evenly split among HCVs, public housing, and permanent supportive housing.

Outcomes for Families Assigned to Usual Care

The study team found that families assigned to Usual Care received a variety of housing and services costing about $41,000 each in the first 37 months after enrolling in the study. Even so, these families had experienced continuing hardship. Almost two-fifths (39 percent) had stayed in a shelter in the prior year or reported experiencing homelessness or being doubled up in the prior 6 months. One in six (17 percent) were separated from at least one child. Levels of psychological distress, substance use, and domestic violence remained high, and more than half of families reported being food insecure. Children had attended 2.1 schools on average in the prior 3 years and had been absent about 5 percent of the time. Nearly two-fifths of families (37 percent) reported working for pay in the week before the 37-month survey, but annual incomes from all sources still averaged only $12,099—far too little to afford housing on the open market.

Insights from 80 In-Depth Interviews with Families

In-depth interviews with 80 families in four sites an average of 7 months after study enrollment helped the study team to understand the experiences of families from their own perspectives. The

Family Options 12-Year Study RD/DCAP

May 4, 2022 ▌3

C H A P T E R 2 : B A C K G R O U N D A N D R A T I O N A L E

study team learned that some families turned down offers of housing assistance to live near school, work, and sources of social support; to maintain family integrity; and to attain what they hoped would be greater permanence (Fisher et al., 2014). Families told the team how shelters, transitional housing programs, and doubled-up housing interfered with the families’ routines and rituals; and they described the creative ways that they coped (Mayberry et al., 2014; Mayberry, 2016).

The study team also learned how parents understood housing’s effect on their parenting and their children’s outcomes (Brown, 2021); and how poverty and hardship, before and after shelter entry, led to separations of parents from children (Shinn et al., 2015). And the study team learned that the kinds of doubled-up housing that families arrange after a period of homelessness are often difficult or even dangerous for both parents and children (Bush and Shinn, 2017).

Findings from the Impact Analysis Comparing the Randomized Groups

Exhibit 1-1 displays a summary of the key impact findings at 20 and 37 months after enrollment.

The 20- and 37-month findings demonstrate that having access to a long-term rent subsidy produces substantial benefits for families, at a cost of about 9 percent more than other types of assistance.

Exhibit 1-1. Summary of 20- and 37-Month Impact Results

SUB vs. UC

CBRR vs. UC

PBTH vs. UC

Outcomes

20 mos.

37 mos.

20 mos.

37 mos.

20 mos.

37 mos.

Housing stability

+ + + + + + □ □ □ □ □ □ + + □ □ + □

Family preservation

□ □

□ □

□ □

□ □

Adult well-being

□ □ + + □ □ + + □ □ □ □ □ □ □ □ □ □ □ □ □ □ □ □

Child wel -being

+ + □ □ + □ □ + □ + □ □ □ □ □ + □ □ □ □ □ □ □ □

Self-sufficiency

_ + □ □ + □ □ + + □ □ □ □ □ □ □ □ □

Legend: + beneficial effect. – detrimental effect. ● ambiguous effect.

Key: CBRR=community-based rapid rehousing. PBTH=project-based transitional housing. SUB=long-term rent subsidy. UC=usual care.

Specifically, the 20- and 37-month impact analyses found the following:

The offer of a long-term rent subsidy led to substantial reductions in homelessness and residential instability relative to usual care. At the 37-month followup survey, families who were offered a long-term rent subsidy were less than half as likely to report being homeless or doubled up in the prior 6 months and less than a quarter as likely to have stayed in an emergency shelter in the prior year. They had stayed in fewer different places.

The offer of a long-term rent subsidy had radiating and largely positive impacts in each of the other domains relative to usual care.

Adult well-being. Parents offered long-term rent subsidies reported less psychological distress, alcohol and substance use, and domestic violence. That is, problems that can sometimes lead to homelessness were reduced when families had access to stable affordable housing.

Family Options 12-Year Study RD/DCAP

May 4, 2022 ▌4

C H A P T E R 2 : B A C K G R O U N D A N D R A T I O N A L E

Child well-being. Children in families offered vouchers were less likely than those in families offered usual care to be separated from parents or to be in foster care at 20 months, although these effects were no longer detected at 37 months. In addition, at 37 months, parents and children who had been in shelter together were more likely to be separated. Because of the high rate of intimate partner violence that family heads had experienced, and the correlation between intimate partner violence and separations, the subsidies may have enabled some respondents to escape violent relationships.

Children in families offered vouchers had fewer school moves, better attendance (at 20 months), and fewer behavior problems as reported by parents (at 37 months) than did children in families offered usual care. Other analyses found that 3- to 4-year-old children and 13- to 17-year-old children in families who received vouchers were more likely to be in a higher functioning group across all outcome domains than were children in usual care. Results here were mediated by improvements in housing stability and housing quality, reductions in family stress, and strengthened family routines (Brown, 2021).

Food security and self-sufficiency. Families offered vouchers were more food secure than families offered usual care. Their incomes did not differ, but about 20 percent fewer families (a 5.6 percentage point difference) reported working for pay in the week before the 20-month survey. This difference in employment was no longer detected by the 37-month survey; but over the entire 37-month followup period, offers of a long-term rent subsidy reduced the proportion with any work for pay by 6 percentage points and reduced the average number of months worked by about 2 months relative to usual care. Outcomes of families who were offered transitional housing or rapid rehousing were largely equivalent to those of families who received usual care.

Costs of the Interventions

In the 37 months after enrollment, the cost of all program use for families offered vouchers exceeded that of families assigned to the other interventions by roughly 9 to 10 percent. On average, SUB programs cost about $1,200 per family per month, which is lower than the corresponding monthly costs for emergency shelter and PBTH programs but higher than the monthly cost for CBRR programs. During the 37-month period, however, SUB families used programs (usually permanent housing subsidies) to a much greater extent than did the families assigned to the other interventions. That is, the substantial gains in housing stability and other outcomes associated with assignment to the SUB intervention come at some additional cost.

2.2 Creating a Research Platform for Learning About Family Homelessness

The experimental research design and detailed, longitudinal information collected for the adults and children enrolled in the Family Options Study provide a strong platform for research on family homelessness. By documenting and archiving the restricted access data at the U.S. Census Bureau, making a public use file available on HUD’s website, 2 and sponsoring small grant programs, HUD has facilitated multiple research efforts. Appendix A summarizes these and other research efforts. The U.S. Department of Health and Human Services (HHS) also used data from the Family Options Study to further investigate the needs of families and children experiencing homelessness and the ways in which its programs, including benefits, employment, 2 The public use file can be accessed at https://www.huduser.gov/portal/family_options_study.html#impact-ir-tab.

Family Options 12-Year Study RD/DCAP

May 4, 2022 ▌5

C H A P T E R 2 : B A C K G R O U N D A N D R A T I O N A L E

and other services might meet those needs. HHS commissioned 10 research briefs that draw on the Family Options Study data. The research briefs examined use of public benefits programs, adolescent development, child separation, employment experiences, and other topics. This 12-Year Study expands the already robust longitudinal data for study families. Enrolling adult children and increasing the number of families who agree to share their PII will create additional opportunities for future research far beyond the analysis proposed for the study itself.

2.3 Rationale for 12-Year Followup on Family Options Study

Although other recent research has examined various aspects of family homelessness and interventions to serve families experiencing homelessness, none has been able to address the central question of the Family Options 12-Year Study: To what extent do the reductions in homelessness and favorable effects on other aspects of adult and child well-being continue over time? What new findings emerge? The 12-Year Study will explore answers to these questions.

By January 2022, some 37 percent of the children who were present with their parents in shelter at the time of random assignment had aged into adulthood. As a result, the 12-Year Study can contribute to our understanding of the extent to which family homelessness in childhood leads to homelessness among young adults. Of special interest is whether priority access to long-term rent subsidies in childhood reduces subsequent homelessness in young adults or affects other markers of young adult development.

Family Options 12-Year Study RD/DCAP

May 4, 2022 ▌6

3. Research Questions for the 12-Year Study

The Family Options 12-Year Study provides an unparalleled research platform for studying sheltered family homelessness. The study will address the following questions: 1. What are the long-term outcomes for families after a stay in shelter, in the absence of priority access to a long-term rent subsidy?

2. Relative to usual care and other types of assistance, what are the long-term impacts of priority access to a long-term rent subsidy on:

2a. Returns to homelessness and other forms of housing instability?

2b. Life trajectories of children as they age into adolescence and young adulthood?

2c. Other aspects of family well-being: family preservation, self-sufficiency, and adult well-being?

3. How were patterns of homelessness and housing instability affected by the COVID-19

pandemic? Did priority access to long-term subsidies have an impact on pandemic-related outcomes? Did current use of subsidies influence the ability of formerly homeless families to weather the pandemic?

For each of those research questions, exhibit 3-1 summarizes its associated hypotheses and rationale and the study’s analysis approaches. Following the exhibit we provide a more detailed rationale in the context of earlier findings from the Family Options Study, literature on family homelessness and interventions to address it, and related research on long-term impacts of instability and exposure to favorable neighborhoods on children.

Exhibit 3-1. Research Questions, Hypotheses, Rationales, and Analyses

Research Question

Hypothesis

Rationale

Analysis Approach

RQ1: Outcomes over Descriptive question:

These analyses describe the natural Descriptive analysis,

12 years for families • Families wil continue to stabilize history of homelessness for families tabulating 12-year not randomized to

and show improvements on all

who receive the usual interventions

outcomes and

SUB

outcomes over time

available in the homeless service

comparing them to 20-,

• They wil continue to obtain and

system (shelter, transitional housing, 37-, and 78-month

use rent subsidies

rapid rehousing) and establish the

outcomes

• Families assigned to PBTH, CBRR, counterfactual for later comparisons to Experimental impact and UC wil not differ beyond

families who received priority offers of analysis comparing

chance

long-term rent subsidies

CBRR versus UC and

Hypotheses suggest a continuation of PBTH versus UC

trends observed at 20 and 37 months

RQ2a: Impact of SUB Priority access to a long-term rent

We expect that early access to stable Experimental impact

on housing stability subsidy wil continue after 12 years to housing in the form of a long-term rent analysis have a positive impact on housing

subsidy puts families on a better

stability relative to usual care or other trajectory of housing stability over time.

types of assistance.

Although we expect the groups to

converge with respect to both use of

subsidies and residential stability, we

stil expect a positive impact of SUB

after 12 years

Family Options 12-Year Study RD/DCAP

May 4, 2022 ▌7

C H A P T E R 3 : R E S E A R C H Q U E S T I O N S

Research Question

Hypothesis

Rationale

Analysis Approach

RQ2b: Impact of SUB Long-term rent subsidies wil have a We expect that the increased Experimental impact

on children as they

positive impact on both adolescent

residential and school stability, food

analysis

age

children (ages 10-17) and adult

security, and stability of routines, along

children (18-30) in appropriate

with reduced parental distress,

developmental domains. For

induced by long-term rent subsidies

adolescents, these include school,

wil have an impact on children as they

behavior, substance use, optimism

age into adolescence and early

about the future, and social support. adulthood. Research suggests that the

For the adult children, these include

length of exposure to positive

housing stability, education, and work. environments is important for youth

Long-term subsidies will also have

outcomes, so we anticipate differences

positive impacts on predictors of

in outcomes 12 years after random

adolescent success such as school

assignment even where we did not see

moves and family routines

them earlier

RQ2c: Impact of SUB Some of the radiating benefits of the Child separations and foster care Experimental impact

on family well-being SUB intervention in the domains of

placements may accumulate over

analysis

family preservation, adult well-being, time. Adult well-being wil continue to and self-sufficiency wil be observed improve in the groups not given priority 12 years after random assignment.

access to long-term rent subsidies as

Some reduction in labor force

they become more stable, but we

participation wil stil be observed

expect some group differences to

persist due to the stability and security

of a long-term rent subsidy.

Reductions in economic distress and

food insecurity wil persist because

families’ needs are partly met by the

voucher; reductions in work effort are

expected to persist due to income and

substitution effects

RQ3: COVID-19,

Both priority access to a long-term rent Current use of a voucher in early 2020 Descriptive analysis of housing stability, and subsidy a decade earlier and current should protect against residential family experiences

health

use of a housing subsidy by any study instability, hardship, and distress in the during pandemic household in early 2020 wil protect

event of job loss. Priority access to a

against returns to homelessness and long-term rent subsidy a decade

Experimental impact

other forms of housing instability

earlier may affect outcomes by

analysis to examine

during the COVID-19 pandemic. They increasing the likelihood of (stil )

impacts of priority

will also lead to better health and

having a subsidy in 2020 and reducing access to a long-term

behavioral health outcomes and less the likelihood of doubling up

rent subsidy on

material hardship

experiences during the

COVID-19 pandemic

3.1 Literature Review and Rationale

Research Question 1: Outcomes Over 12 Years for Families Not Randomized to SUB

This research question sets up the counterfactual for the 12-year impact analysis that will compare the relative effects of priority access to a long-term rent subsidy to usual care, a short-term rent subsidy, and transitional housing. The descriptive analysis to address Research Question 1 will help policymakers understand the usual course of homelessness when families have access to the interventions that are typically available to them through the homeless service system. The typical interventions are emergency shelters; rapid rehousing; transitional housing; and over time, some access to HCVs, public housing, and permanent supportive housing.

Family Options 12-Year Study RD/DCAP

May 4, 2022 ▌8

C H A P T E R 3 : R E S E A R C H Q U E S T I O N S

Comparisons among families in the Family Options Study from the point of shelter entry to the 20-month and 37-month followup surveys show clear improvements in housing stability, income, employment, and adult well-being over time, even for families who received no special offer of assistance (Gubits et al., 2018). Housing stability and income continued to improve between 20 and 37 months, but there was little improvement in family separations or measures of food security and child well-being between the two followup points. (Not all measures were available at study enrollment.)

Other studies have also found improvements in adult and child well-being as families stabilized after a period in shelter. For example, a study of a Family Critical Time Intervention (FCTI) in Westchester County, NY, targeted 210 families in which the mother had a mental health diagnosis. Consistent with findings from the Family Options Study, mothers showed clinically meaningful reductions in mental distress over 15 months regardless of experimental treatment (Samuels et al., 2015). As was the case for their mothers, children of all ages showed reductions in psychological symptoms over time (Shinn et al., 2015). Rafferty et al. (2004) used school administrative data to examine changes in children’s school performance before, during, and after an episode of homelessness in New York City. Performance on standardized tests dropped during the period children were in homeless shelters but rebounded afterwards.

Studies in populations somewhat similar to the Family Options Study’s have also shown the relationship of housing stability to broader well-being outcomes. For example, the Fragile Families study focused on families in 20 urban areas, with a strong overrepresentation of nonmarital births. Three years after children were born, researchers found that housing instability, defined as moves in the past 2 years and housing disarray, were associated with maternal depression and generalized anxiety disorder even after controlling for other social factors. In this descriptive study, researchers found similar associations for incident cases of depression and generalized anxiety disorder, suggesting that the direction of causality ran from instability to mental health, rather than the reverse (Suglia et al., 2011). Similarly, the SHARE

study of women survivors of interpersonal violence found that the number of risks for housing instability (e.g., eviction notice, problems with the landlord, multiple moves) was associated with multiple mental health, quality of life, and work or school absences even after controlling for the level of danger in the abusive relationship. Instability was more important for these outcomes than were factors such as substance use (Rollins et al., 2012).

No previous study of families experiencing homelessness has followed families for more than 5

years. This 12-year followup study will provide critical long-term information about how families fare over time after experiencing an episode of homelessness. We hypothesize that families who are not offered immediate access to long-term rent subsidies will continue to stabilize and show improvements on all outcomes over time. A possible exception is family separations and foster care placements. Previous research has found continued separations and involvement in Child Protective Services for families after an episode of homelessness has ended. For example, one study in New York found that 18 percent of families who experienced homelessness received child welfare services in a 5-year period following their first shelter admission (Park et al., 2004). Other studies have also found increases in child welfare involvement after an episode of homelessness (Cowal et al., 2002; Rodriguez and Shinn, 2016).

In addition to examining well-being outcomes for families who did not receive priority access to a long-term rent subsidy, the study team will examine patterns of use of housing assistance over

Family Options 12-Year Study RD/DCAP

May 4, 2022 ▌9

C H A P T E R 3 : R E S E A R C H Q U E S T I O N S

time. The 37-month followup determined that more than one-third of families who were not offered long-term rent subsidies had managed to obtain one, although the mix of types of long-term subsidies was different from that in the group offered priority access to long-term rent subsidies that usually were Housing Choice Vouchers. We hypothesize that increasing numbers of families who did not get priority access to long-term rent subsidies will obtain such subsidies over time. We will also examine the extent to which families in all groups who receive long-term rent subsidies maintain them. We are not aware of any prior research that addresses this question among formerly homeless families. The answer is important for HUD’s understanding of the cost of housing interventions for families who have experienced homelessness.

A final aspect of Research Question 1 is a comparison of the three groups of families who were not initially offered priority access to long-term rent subsidies; that is, families in the PBTH, CBRR, and UC groups. At 20 months and at 37 months, there were few differences among these groups. We hypothesize that this will continue to hold true 12 years after random assignment, justifying combining all three groups for comparisons to the SUB group for the primary impact analysis of Research Questions 2 and 3.

Research Questions 2a/c: Impact of SUB on Housing Stability and Family Wel -Being

Research Question 2 extends the core question of the Family Options Study to the period 12

years after study enrollment. We focus first on adult housing outcomes (Question 2a) and other aspects of family well-being: family preservation, self-sufficiency, and adult well-being (Question 2c), and then on outcomes for children who have aged into adolescence and young adulthood (Question 2b).

Housing and Adult Well-Being

Since HUD launched the Family Options Study in 2010, multiple studies have been published from experimental and quasi-experimental evaluations of interventions for families experiencing homelessness. Collectively, these studies are well summarized by the title of one of them:

“Housing matters, services might: Findings from the high-needs families program evaluation”

(Rog et al., 2021). This study offered long-term rental subsidies combined with supportive services (permanent supportive housing) to families in Washington State who had histories of chronic homelessness and at least two co-occurring barriers (e.g., mental illness, substance use disorder, domestic violence, trauma from violence, or involvement with the criminal justice system and/or Child Protective Services). The supportive housing offered at least three core services to address family barriers. The quasi-experimental evaluation followed 662 families for 12 months. Families in supportive housing used more mental health outpatient and substance use outpatient services, made more emergency-department visits, and retained Temporary Assistance for Needy Families benefits to a greater extent than did propensity-score-matched families in public housing. The intervention did not affect housing stability or family well-being, including employment, criminal justice, or child welfare. However, families in both types of housing who were involved with Child Protective Services had higher rates of reunification than did a third matched group of families entering shelter (essentially a UC comparison group) (Rog et al., 2017).

The study of FCTI in Westchester County, where both mothers and children showed improved well-being over time, also tested experimental effects of a structured case-management approach to link families with services in their new communities. The control group received the usual

Family Options 12-Year Study RD/DCAP

May 4, 2022 ▌10

C H A P T E R 3 : R E S E A R C H Q U E S T I O N S

care, involving less-intensive case management. Families in both groups received access to affordable housing, but families in the FCTI group got it somewhat faster. Mothers in the two groups did not differ on mental distress over time (Samuels et al., 2015). A New York City intervention, Home to Stay, was billed as an evaluation of an intensive transitional services program. The evaluation compared families who received more-intensive case management and more access to housing subsidies versus the usual care families received in shelters. The families receiving the more-intensive case management left shelter faster and were less likely to return.

However, differences between groups were no longer significant after controlling for whether families exited shelter with a housing subsidy for employed individuals that had employment and minimum payment requirements versus to a housing subsidy for families in the child welfare system or who were receiving federal disability benefits that did not have these requirements (Levitt et al., 2013).

Although family homelessness appears to be on the rise in Europe (Baptista et al., 2017; O’Sullivan, 2017) and Australia (Johnson and Watson, 2017), experimental evaluations in other countries seldom involve families. The only experimental evaluation of interventions for families that has generated evidence to date comes from the Czech Republic: the Housing First intervention for families resulted in strong effects on both housing and health outcomes (Ripka, 2018). That study found that the ability to provide families with flexible cash assistance when needed was an important addition to affordable housing in promoting housing stability.

We have not found studies of housing interventions with substance use and intimate partner violence as outcomes, although these factors can sometimes lead to homelessness. Priority access to long-term rent subsidies reduced both these outcomes at 20 months, but only intimate partner violence at 37 months.

We hypothesize that priority access to a long-term rent subsidy will reduce homelessness and improve all other aspects of residential stability 12 years later. The 12-year followup will use

“return to homelessness” as the confirmatory outcome for the study, defined as whether the family experienced one night of homelessness or doubling up in the past 5 years. Although we anticipate that the intervention groups will continue to converge, we expect that early access to stable housing in the form of a long-term rent subsidy put families on a better trajectory of housing stability over time . We hypothesize that some of the radiating benefits in the adult well-being domain will continue to be observed at 12 years, although we expect groups to converge somewhat as families in all groups stabilize.

Housing and Family Preservation

There is a large overlap between families who stay in homeless shelters and families with involvement in child welfare services. Several studies have examined the role of housing interventions for families at the intersection of these two groups. For example, Collins et al.

(2020) conducted an evaluation with 163 families with a child in out-of-home placement who did not have stable housing. Control families received traditional child welfare services but no housing; treatment families received supportive housing with case management services in the form of an FCTI. After 24 months, the treatment families were less likely to use emergency shelter or have any other homeless system involvement. Two other experiments offered supportive housing to unstably housed families who were involved in the child welfare system.

Both showed strong improvements in housing stability and more mixed effects on child welfare outcomes (Fowler and Chavira, 2014; Pergamit et al., 2019).

Family Options 12-Year Study RD/DCAP

May 4, 2022 ▌11

C H A P T E R 3 : R E S E A R C H Q U E S T I O N S

At the 20-month followup in Family Options, we found that priority access to long-term rent subsidies reduced child separations from parents and foster care placements. By 37 months, those impacts had dissipated. Separations in any six-month period are relatively low in incidence, but over a longer term, separations and foster care placements may accumulate. Park et al. (2004) found that 18 percent of children experiencing homelessness in New York City received child welfare services in a five-year period after their first shelter admission. Therefore, we will extend the measurement of separations and foster care placements for the entire 12-year period since study enrollment. Continued residential instability in groups that did not receive priority access to long-term rent subsidies might lead to additional child welfare involvement over time. We will omit measurement of separation of partners from each other because it is an ambiguous outcome (separations were associated with domestic violence). We hypothesize that access to long-term rent subsidies will reduce separations of children from parents and foster care placements over the full 12-year followup period.

Self-Sufficiency

At 37 months, families with priority access to long-term rent subsidies had lower levels of economic stress and food insecurity than families assigned to usual care. They also had lower levels of work effort. Continued stability and reduced housing costs induced by the offer of long-term subsidies may have continued positive impacts on food security and economic stress in the long-term rent subsidy group, but we also expect continued improvements in the other groups as they become more stable, so that impacts may be smaller than observed previously.

The reduction in work effort among families who received priority access to long-term subsidies persisted through followup quarters 13 and 14 in administrative records. This finding is consistent with both theoretical predictions (work brings a lower return because rent is based on income; needs for income are reduced) and prior research (e.g., Jacob and Ludwig, 2012; Mills, 2006), so we expect it to continue.

We hypothesize that priority access to long-term rent subsidies will lead to reductions in family separations and foster care placements, improvements in adult behavioral health, reductions in food security and economic stress, and reductions in work effort 12 years after study enrollment, relative to families not given such priority access. This represents a continuation, perhaps at a reduced level, of impacts observed in previous analyses. In the case of adults, we have little reason to expect “sleeper effects,” that is, new impacts that were not observed earlier. Thus, we will confine the outcomes we examine to those for which we found impacts at 20 or 37 months (including the negative effect on employment).

Research Question 2b: Impact of SUB on Children as They Age

Research Question 2 also concerns the impact of priority access to a long-term rent subsidy on the life trajectories of children as they age into adolescence and early adulthood. We will consider two groups of children: adolescents (ages 10-17 at the time of the 12-year survey) and children who have aged into adulthood, adult children (ages 18 to 30). For household heads (Questions 2a and 2c), we will confine our research questions to the continuation of outcomes we observed in previous analyses. For adolescents and young adults, we will consider the developmental tasks appropriate to the age group and examine outcomes that may or may not have shown impacts earlier. The research literature suggests that we may find sleeper effects among children who have aged into young adulthood.

Family Options 12-Year Study RD/DCAP

May 4, 2022 ▌12

C H A P T E R 3 : R E S E A R C H Q U E S T I O N S

Adolescents Younger than Age 18

We observed impacts of long-term subsidies on adolescents in the 20- and 37-month followup analyses (Brown, 2021; Gubits et al., 2015; Gubits et al., 2016) and expect to see parallel effects for children who have aged into this group at the 12-year time point. Indeed, because the children who have aged into the group have had longer periods of stability induced by long-term rent subsidies, we may expect stronger effects than found in earlier analyses.

For adolescents, the key indicator that signals a successful transition to adulthood is school. We will examine adolescents’ school attendance, effort, performance, and disciplinary problems, from the adolescents’ self-reported information and from parental report. In earlier analyses, we found impacts of the long-term rent subsidies on parents’ reports of children’s behavior, with both reductions in behavior problems and increases in prosocial behavior. Other studies have also found short-term effects of housing interventions on children’s behavior. For example, in the FCTI study, children showed scattered experimental impacts relative to controls when their parents did not. Impacts of FCTI included a reduction of internalizing and externalizing problems for children ages 1½ to 5, a decline in externalizing problems for adolescents ages 11-16, and a decline in self-reported school problems among children ages 6 to 16 over a 24-month followup period. These impacts were in addition to the reductions in symptoms in both treatment and control groups over time (Shinn et al., 2015). However, no studies have examined the long-term effects of housing interventions on children in families who have experienced homelessness.

In the 12-year followup survey, we will ask both parents and the adolescents themselves about these areas. We also will examine use of tobacco and other substances and positive indicators such as social support and optimism about the future. Further, we will examine some key predictors of adolescent outcomes. Household routines and their converse, disorder and chaos, were important predictors of youth outcomes at 37 months (Brown, 2021). We will continue to ask parents about these issues and ask adolescents about parental monitoring of their behavior.

We will ask adults about the number of schools their children have attended since random assignment, because of the robust association in the literature between both residential and school moves and academic outcomes (e.g., Haveman et al., 1991; Swanson and Schneider, 1999). We found experimental impacts of long-term rent subsidies on school moves in earlier analyses; in the 12-year followup, we will extend the analysis to cover the full followup period.

We hypothesize that priority access to long-term rent subsidies will have long-term positive impacts on adolescents’ well-being across all these measures. These positive impacts arise from exposure to more stable living arrangements, improved food security, and reduced parental distress induced by random assignment to a long-term housing subsidy, as well as improvements in the additional predictors of successful transition to adulthood specified in this section.

Examining the causal pathways is beyond the scope of this contract, but the data will be available for secondary analysis.

Young Adults Ages 18-30

By January 2022, some 37 percent of the children who were present with their parents in shelter at the time of random assignment had aged into adulthood. As a result, the 12-Year Study can contribute to our understanding of the extent to which family homelessness in childhood leads to homelessness among young adults. The 12-year impact analysis will provide evidence about

Family Options 12-Year Study RD/DCAP

May 4, 2022 ▌13

C H A P T E R 3 : R E S E A R C H Q U E S T I O N S

whether access to long-term rent subsidies in childhood reduces subsequent homelessness in young adults or affects other markers of young adult development. This represents an important new area of investigation. If offers of long-term housing subsidies to families lead to better outcomes for their children as those children age into adulthood, that finding would have important implications for policy.

Children as well as adults are affected by instability, and many of these effects are manifested in early adulthood. A study of a nationally representative sample of youth in the National

Longitudinal Study of Adolescent to Adult Health found that residential changes in adolescence were associated with increased depression, criminal activity, and smoking in young adulthood.

Further, children living in multigenerational homes (some of whom may have met the Family Option Study’s definition of doubling up with another household because the family could not find or afford a place of their own) were more likely to be arrested than were children living in single-generational homes. Interestingly, changes in family structure did matter (Fowler et al., 2015). Other studies have found that residential moves during school reduce the probability of high school graduation (Haveman et al., 1991) and increase the likelihood of dropping out (Swanson and Schneider, 1999).

Harvey (2020) used data from the National Longitudinal Survey of Youth 1979 (NLSY79) and Child and Young Adult cohort to examine the cumulative effects of years spent in doubled-up arrangements (for this population, defined by who is present, not the inability to afford a place).

Being doubled up with non-kin adults or extended family other than grandparents led to adverse educational and health outcomes in young adulthood. Bush and Shinn (2017) found that, in many cases, being doubled up after being homeless is not good for children.

The Voices of Youth Count (VoYC) survey (Morton et al., 2017) found that among young adults ages 18-25 who experienced homelessness (broadly defined to include couch surfing), “the majority had experiences of homelessness or housing instability that started in childhood or adolescence.” VoYC also found that, among young people ages 13-25, nearly a quarter who had unaccompanied experiences of homelessness had earlier experiences of homelessness with their families. That study did not examine anything about the earlier experience of family homelessness, nor did it examine the prevalence of homelessness among young adults who had prior homeless experiences with their families, but it seems unlikely to be negligible. As of now, no studies have examined the long-term outcomes of child and adolescent well-being in families who previously experienced homelessness. However, the Moving to Opportunity (MTO) demonstration has produced relevant findings about neighborhood effects on low-SES (Chetty et al., 2016). Though all families in MTO had access to long-term rent subsidies, participating families have been followed for a long time, with their children aging into adolescence and adulthood. Short-term results showed that experimentally induced moves to low-poverty neighborhoods improved family safety and multiple aspects of adult well-being (although not economic outcomes); effects for children were mixed. The long-term impacts were more pronounced on children who were age 18 or younger at study enrollment and age 21 or older in 2012. Children who moved to a low-poverty neighborhood before they turned age 13 had higher earnings, higher rates of college attendance, and lower rates of single parenthood. However, those who moved as adolescents ages 13 or older had slightly unfavorable effects. Further, benefits of moving to low-poverty areas increased linearly with length of exposure to the better environments.

Family Options 12-Year Study RD/DCAP

May 4, 2022 ▌14

C H A P T E R 3 : R E S E A R C H Q U E S T I O N S

The MTO results, and parallel associations of adult outcomes with length of exposure to neighborhoods with lower poverty found in census data (Chetty and Hendren, 2018), suggest that priority access to a long-term rent subsidy may have impacts on children’s long-term well-being that could exceed those observed at 37 months. To the extent that long-term rent subsidies set families on a better trajectory across multiple domains, the length of exposure to that changed trajectory may be important, with children who were younger at study enrollment benefiting more and those who were already adolescents benefiting less. Other experimental studies of employment-based welfare and antipoverty policies (summarized by Huston [2017]) suggest that young children benefit more than their older siblings.

Overall, we hypothesize that the greater residential and school stability and food security, reduced domestic violence, and improvements in parents’ behavioral health induced by access to long-term rent subsidies will have favorable impacts on children. However, we anticipate that these effects may vary by age. Children who are younger than age 18 at the time of the 12-year followup survey will have experienced the greater stability induced by long-term rental subsidies for a substantial period, in some cases their entire lives, assuming that this stability continues over time. The youngest among them were infants and toddlers at the time of greatest experimental contrast between groups, and because of this and the large effects of environments on very young children (e.g., Huston, 2017), they may show the largest effects. Young adults ages 18-24 will have experienced these experimental impacts from age 6 to age 12 until they left the parental home: these young adults are comparable to the MTO and census samples that moved before age 13, and we may expect large impacts. Young adults ages 25 and older will have experienced the intervention for a shorter time, on average, before leaving the parental home, and we may expect smaller impacts.

Research Question 3: COVID-19, Housing Stability, and Health

Research Question 3 asks how patterns of homelessness and housing instability were affected by the COVID-19 pandemic. The question also examines whether priority access to long-term rent subsidies helped families to weather the pandemic better, compared with the usual care and the other interventions. An interesting question is whether the advantages of a long-term rent subsidy for weathering the pandemic depend on the family currently using the subsidy, as the subsidy would be likely to protect households from falling behind on rent and experiencing food insecurity and economic and psychological distress in the event of job loss. Experimental impacts could also arise because subsidies might also protect households from doubling up and overcrowding that could exacerbate the spread of COVID-19. We hypothesize associations between current use of rent subsidies and better housing and health outcomes during the pandemic. Because we expect priority access to housing subsidies at the time of shelter entry to have continuing effects on differential use of subsidies and on current housing status, we expect that the SUB intervention may also have protective effects a decade later.

3.2 Data Sources

Exhibit 3-2 below shows the data sources the study team will use to address each of the research questions.

Family Options 12-Year Study RD/DCAP

May 4, 2022 ▌15

Exhibit 3-2. Data Sources, by Research Question and Funding Status

Funded Data Sources

Unfunded Data Sources

Adult

Research Data

Adult

HUD Admin

Expanded

National

Assistance

Research Question

Head of

Child

Data

NDNH Data

NCHS

Household

Survey

Child

(IMS/PIC,

NDNH

(Adult

Student

Center

(Mortality Data)

Survey

Web Survey

TRACS)

Children)

Clearinghouse

(Medicaid

claims)

RQ1: Outcomes over 12

years for families not



















randomized to SUB

RQ2a: Impact of SUB on

housing stability





RQ2b: Impact of SUB on

children as they age

a









RQ2c: Impact of SUB on

family well-being

















RQ3: COVID-19, housing

stability, and health











Key: IMS/PIC=Inventory Management/Public Housing Information Center. NCHS=National Center for Health Statistics. NDNH=National Directory of New Hires.

SUB=long-term housing subsidy. TRACS=Tenant Rental Assistance Certification System.

a Parent on child module of the survey.

Abt Associates

Family Options 12-Year Study RD/DCAP

May 4, 2022 ▌16

4. Administrative Data Sources

This chapter discusses currently funded administrative data collection and additional administrative data collection sources that are not currently funded. Although we will obtain most of the primary study outcomes from survey data, the proposed administrative data sources have key advantages:

• The administrative data sources discussed below provide national coverage of the full study sample. 3 Though the study began in 12 local sites, participants were living in nearly all states as of the 78-month tracking survey. Coverage on outcomes from national administrative sources is not subject to survey non-response.

• Administrative data can typically be captured at a more granular level (e.g., 5 years of quarterly earnings) that would take too long or cost too much to collect with a survey.

• Administrative data sources can capture outcomes that are more difficult to assess in surveys (e.g., historical outcomes subject to recall bias). For example, families may be aware of receiving housing assistance but unable to accurately explain or recall its source (e.g., federal program or local housing assistance program).

4.1 Currently Funded Administrative Data Sources

We plan to draw from two sources: (1) HUD’s Inventory Management/Public Housing Information Center (IMS/PIC) and Tenant Rental Assistance Certification System (TRACS) administrative data and (2) National Directory of New Hires (NDNH) data on quarterly wages and employment. Exhibit 4-1 summarizes these administrative data sources and key outcomes.

We assume that written informed consent obtained from heads of households at the time of study enrollment supports the ongoing collection of data from these two sources. Authorization to collect data from these sources was explicitly detailed in the original consent obtained from study participants at enrollment.

Exhibit 4-1. Summary of Currently Funded Administrative Data Sources and Key Outcomes Data Source

Description

Key Outcomes / Uses

Inventory Management/Public Provides national quarterly data on use of HUD- • Ever used housing subsidy Housing Information Center

funded long-term rent subsidy programs. Includes • Currently using housing subsidy (%) (IMS/PIC)

type of assistance received, duration of

• Number of months of subsidy

Tenant Rental Assistance

assistance, and address. Participants can be

receipt

Certification System (TRACS)

matched on Social Security number or on name + • Monthly program use rates

date of birth. Data can be included in archiving

• Residential history

National Directory of New Hires Provides national quarterly data on earnings,

• Earnings in past year ($), by year

(NDNH)

employment, and Unemployment Insurance from • Any employment in past year, by aggregation of state Unemployment Insurance

year

reporting. Participants matched based on Social • Number of quarters employed in Security number. Data cannot be included in

past year, by year

archiving

3 As discussed subsequently, some of the optional data sources are national but may not cover the full sample due to varying consent requirements (e.g., whether verbal consent is acceptable) and the breadth of consent of release of information for administrative data collection provided at varying points of the study.

Abt Associates

Family Options 12-Year Study RD/DCAP

May 4, 2022 ▌17

C H A P T E R 4 : A D M I N I S T R A T I V E D A T A S O U R C E S

IMS/PIC and TRACS

Administrative data from IMS/PIC and TRACS will continue to provide an important source of information on study families’ receipt of housing assistance through HUD’s HCV, public housing, and multifamily assisted housing programs. For the 78-month tracking study, the study team obtained extracts of these data covering the period through Q1 2019 (March 2019). For the 12-Year Study, we will collect additional extracts of IMS/PIC and TRACS data to measure receipt of HUD housing assistance and the duration of rental assistance for the full 12-year followup period. For study families who are receiving rental assistance, these data also provide contact information that supports location tracking to inform the adult survey administration.

(Appendix B details the IMS/PIC and TRACS data elements.)

NDNH

Employment and earnings outcomes will continue to be measured through NDNH, a national database maintained by the Office of Child Support Enforcement (OCSE) within HHS. We anticipate that HUD will continue collecting NDNH data on adult heads of household through its Memorandum of Understanding (MOU) with OCSE. (Appendix B also presents the NDNH data elements.)

4.2 Optional Administrative Data Sources

Additional administrative data sources might be informative but are outside the study’s current scope. These should be considered for contract expansion and include the following: (1) HUD

IMS/PIC and TRACS data collection for adult children, (2) expanded NDNH data collection for adult heads of household and adult children, (3) NSC data on postsecondary educational outcomes for adult heads of household and adult children, (4) Research Data Assistance Center (ResDAC) for Medicaid/Medicare data on health conditions, and (5) National Center for Health Statistics (NCHS) for mortality data.

Exhibit 4-2. Summary of Optional Administrative Data Sources and Potential Outcomes Data Source

Description

Potential Outcomes / Uses

Inventory

Source provides national quarterly data on use • Ever used housing subsidy

Management/Public

of HUD-funded long-term rent subsidy

• Currently using housing subsidy (%)

Housing Information

programs. Includes type of assistance received, • Number of months of subsidy receipt Center and Tenant Rental duration of assistance, and address.

• Monthly program use rates

Assistance Certification

Participants can be matched on Social Security • Residential history

System (for adult children) number or on name + date of birth. Data can be included in archiving

National Directory of New Source provides national quarterly data on

• Adult child earnings in past year ($)

Hires (additional data

earnings and employment from aggregation of • Adult child employment in past year collection)

state and federal Unemployment Insurance

• Adult child number of quarters employed in

reporting. Propose expanding sample to

past year

include adult children who consent to study

participation. Expansion of variables in pass-

through file could support other research aims,

such as cost analyses. Participants matched

based on Social Security number. Data cannot

be included in archiving

Family Options 12-Year Study RD/DCAP

May 4, 2022 ▌18

C H A P T E R 4 : A D M I N I S T R A T I V E D A T A S O U R C E S

Data Source

Description

Potential Outcomes / Uses

National Student

Source provides national data on

• Ever enrolled in postsecondary education

Clearinghouse

postsecondary enrollment in 2-year, 4-year,

• Degree attainment (any, 2-year, 4-year)

(postsecondary education) and vocational certificate programs, covering • Time to degree completion

>96 percent of degree-granting institutions.

Matches con be conducted using Social

Security number or both name + date of birth.

Requires ink or electronic consent. Data cannot

be included in archiving

Research Data Assistance We propose accessing national Medicaid

• Inpatient admissions (per 100 months)

Center (national Medicaid claims data through the ResDAC data

• Home health visits (per 100 months)

claims)

warehouse. These data can be used to

• Physician office visits (per month)

understand how receipt of housing assistance • Outpatient hospital services (events per may affect medical service use patterns. The

month)

majority of our sample reported Medicaid

• Prescription drug events (per month)

receipt in followup surveys, so we likely have • Total Medicaid fee-for-service payments high sample coverage

• Chronic or potentially disabling conditions

(including psychiatric and substance use

One limitation of this source is a three-year lag

disorders)

in data availability. Abt’s prior experience is that

no consent is required for this source (data

anonymized). ResDAC Medicaid claims data

cannot be included in archiving

National Center for Health We proposed accessing data on date and

• Percentage deceased

Statistics (mortality)

cause of death through a Data Use Agreement • Cause of death

between HUD and NCHS. Fewer than 2

percent of total sample were deceased on the

basis of study tracking as of the 78-month

survey, but the COVID-19 pandemic may have

accelerated mortality

NCHS data only available for sample members

that provided consent to release PI to HUD

(with a match conducted based on Social

Security number or on name + date of birth)

IMS/PIC and TRACS Data on Adult Children

Administrative data from IMS/PIC and TRACS on adult children could provide an important source of information on their receipt of housing assistance through HUD’s HCV, public housing, and multifamily assisted housing programs. One important question is the extent to which intervention differences in adult child well-being and self-sufficiency outcomes may be associated with ongoing receipt of housing assistance—whether from continuing to reside with the family head receiving housing assistance or as their own head of household. Abt would request data from the IMS/PIC and TRACS data systems for the adult children. We would plan to use the same finder file template and request the same variables as for the adult head of household match. We would want to identify any match where the adult child was a head of household, co-head of household, or household member and their age was greater than or equal to 18 years at the time of the effective date of action. The consent we obtain from the adult children during study enrollment will allow us to match their information to IMS/PIC and TRACS data.

Family Options 12-Year Study RD/DCAP

May 4, 2022 ▌19

C H A P T E R 4 : A D M I N I S T R A T I V E D A T A S O U R C E S

Additional NDNH Data Collection

HUD has an MOU with OCSE that could be modified to allow for the additional data collection opportunities described below.

First, the sample could be expanded to include adult children who provide consent to participate in the Family Options Study and provide their Social Security number (SSN). The consent and SSNs we obtain from the adult children during study enrollment permit us to match their information to NDNH records. The value of adding adult children to the study sample is that some prior research on housing assistance indicates potential intervention effects of long-term rent subsidies on earnings of young adults, primarily among those whose families began receiving housing assistance during their early childhood (e.g., Andersson et al., 2016).

Obtaining NDNH earnings and employment information for young adults who enroll in the study might enable longitudinal assessment of intervention effects on these outcomes (if sufficient numbers of young adults enroll).

Only the most recent eight quarters of NDNH data are available for individuals unless data hold requests are submitted to OCSE under an MOU that permits retaining data for research purposes.

Delays in submitting data hold requests then can result in lost data as older quarters are purged from the system and then are no longer available to be held for research purposes. Thus, if we proceed with this option, we would recommend more frequent data holds during the study enrollment period to preserve as much historical data as possible (relative to the current annual data hold schedule for family heads shown in exhibit 4-3). Thus, we would want to match young adults to NDNH soon after they consent to study participation to ensure as much of their historical data as possible is held. We would recommend that the MOU specify quarterly data hold requests during the adult child survey data collection period (currently October 2022 to December 2023). After completing study enrollment, we can transition to annual data holds on the same schedule as adult heads of household, as there would no longer be a risk of losing historical data.

Exhibit 4-3. Timeline of Data Holds for Family Heads Only vs. Expansion to Include Adult Children

Data Hold Dates Current Family Head Data Hold Schedule (Annual)

Recommended Adult Child Data Hold

Schedule (NEW)

Nov 2021

Yes, New data for Q3 2019 to Q3 2021

n/a

mid-Jun 2022

Yes, New data for Q4 2021 to Q2 2022

n/a

October 2022: Study Enrollment Begins

mid-Dec 2022

No

Yes, Q1 2021 to Q4 2022

mid-Mar 2023

Yes, New data for Q2 2022 to Q1 2023

Yes, Q2 2021 to Q1 2023

mid-Jun 2023

No

Yes, Q3 2021 to Q2 2023

mid-Sep 2023

No

Yes, Q4 2021 to Q3 2023

mid-Dec 2023

No

Yes, Q1 2022 to Q4 2023

December 2023: Study Enrollment Ends (Projected)

mid-Mar 2024

Yes, New data for Q2 2023 to Q1 2024

Yes, New data for Q2 2023 to Q1 2024

mid-Mar 2025

Yes, New data for Q2 2024 to Q1 2025

Yes, New data for Q2 2024 to Q1 2025

We always request the most recent eight quarters of data. Exhibit 4-3 indicates the new quarters of data being added to the data file. For family heads, the most recent data hold included data up

Family Options 12-Year Study RD/DCAP

May 4, 2022 ▌20

C H A P T E R 4 : A D M I N I S T R A T I V E D A T A S O U R C E S

through Q3 2021. For adult children, the timing of enrollment and addition to the NDNH sample matters because only the most recent eight quarters of data will be available. That is, an adult child who enrolls in October 2022 would have data going back to Q1 2021 (from the December 2022 quarterly hold) whereas a later enrollee in May 2023 would have data going back only to Q3 2021 (from the June 2023 quarterly hold).

Second, information on study participants included in the pass-through file could be expanded to include additional baseline covariates, survey outcomes, and program use data. The “pass through variables” are variables that are provided by data users and are merged onto NDNH

records. These variables allow for various types of analysis. To comply with privacy protections in place at the time, the original NDNH MOU between HUD and OCSE permitted only six pass-through variables, which were the bare minimum needed to estimate pairwise intervention impacts on employment and earnings for each intervention relative to UC. The variables we passed through were (1) random assignment group; (2) random assignment groups for which the family appeared eligible at random assignment; (3) calendar quarter of random assignment; (4) site; (5) eligibility indicator (to distinguish families dropped from the final analysis sample); and (6) head of household indicator (to distinguish adult head of household from spouses or partners).

From our experience with other recent federal studies that use NDNH data (e.g., Health Profession Opportunity Grants Program evaluations, Transitional Living Program Youth Outcomes Study), OCSE is willing to allow a larger number of variables on the pass-through file, including both additional baseline covariates and outcomes collected through other methods.

Adding variables has expanded the kinds of research questions that can be answered with NDNH

data. For example, whether employment and earnings outcomes vary by participant characteristics (i.e., difference by race, ethnicity, or disability status) could be explored.

Including survey outcomes such as housing stability could permit examination of whether changes in employment and earnings outcomes are associated with other outcomes of interest.

Including program use data could permit exploration of whether spells of program use are correlated with employment or earnings outcomes. Inclusion of program use and survey data would also support cost analyses, if funded. NDNH earnings and employment data could substantially enhance the quality of imputed (predicted) housing stability and program use datasets we might use in the cost analysis (discussed in Section 7.6).

Third, New Hire and Unemployment Insurance data files could be collected in addition to the Quarterly Wage file currently requested in the MOU. NDNH data consist of three data files—

Quarterly Wage, New Hire, and Unemployment Insurance. The original MOU for the Family Options Study only includes the Quarterly Wage file, which provides earnings by employer for each job held during the period. The New Hire file contains information on all newly hired employees, as reported by employers to each State Directory of New Hires (federal agencies report directly to NDNH), with the key variable being the date of hire. The Unemployment Insurance file includes information on individuals who applied for or received UI, the benefit amount, and the date the claim was processed.

At the time of the original MOU, the study team prioritized the earnings data for use in employment and earnings outcomes. In recent projects, OCSE has treated requests for one of the specific data files similarly to requests for all three. Adding the New Hire and Unemployment Insurance data files would not incur any additional cost to HUD. Though there are currently no

Family Options 12-Year Study RD/DCAP

May 4, 2022 ▌21

C H A P T E R 4 : A D M I N I S T R A T I V E D A T A S O U R C E S

specific study research questions that require these data sources, research questions could emerge in the future that could make use of these data for additional analyses.

As an example, although the interventions’ impact on Unemployment Insurance is not currently a study outcome, Unemployment Insurance data could be used descriptively to provide context for earnings patterns observed during the pandemic. For part of this period, Unemployment Insurance benefits were enhanced and extended. Understanding the portion of the sample receiving those benefits and how those benefits compared to prior levels of earnings could provide context to spells of unemployment or lower earnings observed in the Quarterly Wage file during the pandemic.

National Student Clearinghouse Postsecondary Education Data

NSC data is a cost-effective national source of postsecondary educational outcomes for the family head and adult children included in the study. Postsecondary enrollment, degree attainment, degree type (e.g., two- vs. four-year) and time to degree completion could be assessed for all adult children who consent to participate in the study and for the adult heads of household for whom we have obtained electronic or written informed consent.

Our survey instruments capture information on highest educational attainment and degree completion among survey respondents as of the time of survey completion. Key advantages of the NSC as a data source to supplement survey data collection are (1) tracking progress on educational attainment over time (particularly for adult children who may be more likely to be currently enrolled in programs at the time they are surveyed); (2) addressing potential survey non-response (among the family heads who have provided written or electronic content); and (3) collecting more detailed information on institutional characteristics and degree attainment than included in the survey instruments (e.g., enrollment in two-year vs. four-year institutions, graduation date, type of degree obtained).

The NSC is a national database on postsecondary enrollment based on information contributed by participating universities. It currently covers more than 97 percent of students enrolled in two-or four-year programs as well as certificate programs in those participating institutions, with coverage going back to 2005. This data source could be used to assess outcomes such as postsecondary enrollment and degree attainment for adult children who consent to participate in the study and for the adult heads of household for whom we have obtained electronic or written informed consent. For reference, a recent study of homeless and runaway youth who participated in Transitional Living Programs found 28 percent of youth ( n=365, ages 16-21 at study enrollment) had ever been enrolled in a postsecondary program during a four-year period (Mahathey et al., 2021). This study also observed similar current enrollment rates between NSC

administrative data and youth self-report data from a survey conducted at study entry.

To obtain data, Abt would execute an MOU with the NSC to conduct the data match. NSC

matches can be conducted using SSNs and/or a combination of name and date of birth. After the MOU is established, Abt would upload a finder file with applicable participants’ SSN, name, and date of birth to NSC’s secure portal. A flag in the file returned indicates whether the match was made based on SSN or name and date of birth.

We recommend reassessing the potential value of NSC data collection in March 2023 at the planned 20-week survey data collection check-in. At this time, we will have informed

Family Options 12-Year Study RD/DCAP

May 4, 2022 ▌22

C H A P T E R 4 : A D M I N I S T R A T I V E D A T A S O U R C E S

projections of the final number of adult children likely to enroll in the study and updated projections on adult heads of household that could be included in the sample based on whether they have provided written or electronic consent. If HUD elects to proceed with NSC data collection at this time, beginning the MOU process no later than June 2023 should provide sufficient time to establish the MOU by December 2023 and to receive data for analysis by March 2024. We would submit a memo summarizing our analysis in late 2024.

Research Data Assistance Center for Medicaid Claims

Collecting Medicaid claims data would allow us to test hypotheses relating stable housing to families’ health diagnoses and healthcare utilization. One hypothesis is that rates of behavioral health and substance use disorder diagnoses will be low overall, but even lower in the SUB

group. Publicly available algorithms can be used to identify certain chronic health conditions, mental health and substance use disorders, or potentially disabling conditions based on diagnoses on Medicaid FFS claims and managed care encounter records. These algorithms could be used to create diagnosis outcomes. A second hypothesis is that health utilization patterns should reflect greater use of physician office visits and lower use of inpatient admissions over time, facilitated by greater housing stability improving access to routine care. Utilization outcomes could include inpatient admissions, home health visits, physician office visits, outpatient hospital services, prescription drug events, and total Medicaid fee-for-service (FFS) payments.

Medicaid claims and encounter data on individual utilization can be accessed at the federal level, without the need for negotiating agreements at the state level. However, the cost of obtaining data is substantial, so sample coverage and utility of the data merit close consideration. Sample coverage in the Medicaid data is likely to be high. At the 37-month followup, the study found that more than 80 percent of study families reported having publicly funded health insurance, so we expect to be able to match most study families to Medicaid records based on PII (primarily SSNs). Claims data also typically do not require consent to access, so the data could be obtained for the full study sample.

However, the Medicaid claims data have two important limitations. The first is a relatively narrow window of data availability. The Transformed Medicaid Statistical Information System (T-MSIS) was created in 2016. Data prior to that time are available through the now-obsolete Medicaid Analytic eXtract (MAX) system but require considerable effort to link and process to create consistency with current data standards and format. On the other end, there is an 11- to 12-month lag in the availability of the T-MSIS data (preliminary 2021 data will be released around December 2022). Thus, data would likely be available only from 2016 through 2020 or 2021, depending on when the MOU is executed.

A second limitation is that intervention impacts derived from claims data require careful consideration of the context of the sample to be interpretable. Claims data do not provide specific information about the severity of patients’ illnesses or patients’ functional or cognitive status—

only use of services and diagnosis. Changes in healthcare service utilization can reflect both change in need for services (e.g., deterioration in health) and changes in access to care (e.g., insurance status, ability to attend appointments). The interventions studied target housing stability, which could plausibly affect both health (e.g., reduced exposure to housing-based health risks) and access to healthcare services. At study enrollment, Family Options Study family heads were relatively young (median age of 29) and healthy (21 percent reported a disability that limited work), with the majority of families reporting Medicaid use (60 percent). Findings would

Family Options 12-Year Study RD/DCAP

May 4, 2022 ▌23

C H A P T E R 4 : A D M I N I S T R A T I V E D A T A S O U R C E S

need to be considered in conjunction with survey data on mental and behavioral health and access to insurance to understand whether differences observed between interventions groups could be attributable to underlying differences in health relative to access to services.

If HUD exercises this optional data collection, Abt would obtain the specified data from the ResDAC at the University of Minnesota. Abt would need to purchase the data files from ResDAC to use the data. The ResDAC data link records across state Medicaid programs and for dual eligibility for Medicaid and Medicare. 4 Abt staff are experienced using these data and know the required steps of entering into a Data Use Agreement with ResDAC, providing a finder file for linkage, and receiving and cleaning records received. We would need to pursue a Data Use Agreement with ResDAC in 2022 to ensure we have time to execute the agreement and obtain data in time for analysis and reporting. If this option is exercised, we will also submit a memo summarizing our analysis in late 2024. This summary memo will document the data we collected, the time period covered, the data covered, and our findings. We will also incorporate the analyses into the study’s final report, should HUD choose to exercise that optional task.

National Center for Health Statistics for Mortality Data

Entering into a Data Use Agreement with the National Center for Health Statistics (NCHS) at the Centers for Disease Control and Prevention could provide information on mortality and the cause of death. Though the agreement likely would not be executed in time to prospectively inform study tracking efforts, it could be used to better understand how many families could potentially have been surveyed. That is, administrative data may identify some families who are deceased that were not identified as such though prior tracking efforts. Additional deceased families identified then could be removed from the analysis sample and from computation of survey response rates.

It is also possible that mortality may be considered as a study outcome as time passes. The average family head was relatively young at baseline (median of age 29). Mortality was too rare to explore as a potential study impact in the 37-month followup. As of the 78-month tracking survey, less than 2 percent of the total family head sample (1.8 percent) were deceased, which is still rare enough to make potential intervention effects difficult to detect reliably. However, the COVID-19 pandemic may have accelerated mortality in the study sample. Despite their relative youth, parents in the study sample may have had high rates of risk factors such as asthma and obesity. Thus, it may be worth measuring whether study interventions affected mortality, subject to sample limitations discussed below.

The key drawback to NCHS data for the present study is that it would be limited to families who have provided release of PII information to HUD. NCHS requires either SSNs or name and date of birth to conduct matches, and the Data Use Agreement would need to be made between HUD

and NCHS. As a result, the data match would provide contemporaneous information only on this 4 We note that Medicare data are also available through ResDAC, but we do not recommend collecting Medicare data. Our assessment is that Medicare data are not likely to add substantial benefit relative to the cost of obtaining the data and the intensive efforts needed to process and analyze the data. A key concern is that a low proportion of families are likely be included in Medicare data. Few family heads will have aged into eligibility, as the sample is relatively young (median of age 29 at baseline, with only 8 percent age 45 and older). Family heads who are receiving Social Security Disability Insurance (SSDI) should be eligible for Medicare, but only 9 percent of family heads self-reported SSDI receipt at 37 months. There would be a similar lag in data availability.

Family Options 12-Year Study RD/DCAP

May 4, 2022 ▌24

C H A P T E R 4 : A D M I N I S T R A T I V E D A T A S O U R C E S

sub-sample of participants. New consents for release of PII to HUD obtained in the present study could only be obtained from non-deceased participants who (by definition) would not be present in the mortality database. An analysis of mortality then would need to combine the administrative and survey tracking data on mortality and consider how to address potential bias from persons not included in the consent to release PII sample and not located through tracking being excluded from the administrative data match. However, over a longer period of time, NCHS data may be valuable for monitoring mortality rates among an expanded consent-to-release PII sample.

Family Options 12-Year Study RD/DCAP

May 4, 2022 ▌25

5. Primary Data Collection

This section discusses the three primary data collection activities for the Family Options 12-Year Study: adult head of household survey, child survey (ages 10-17), and adult children survey (ages 18-30). Our approach to the Parent on Child Module of the adult head of household survey, the child survey, and the adult child survey reflects our priority on collecting detailed information from a finite number of children per family (up to six in the Parent on Child Module, up to three in the child survey, and up to three adult children for the web survey). For each of the three surveys, we discuss the survey topics, the survey sample, and the informed consent processes.

5.1 Adult Head of Household Survey

The 12-year adult head of household survey is adapted from the 20- and 37-month survey instruments. Consistent with the earlier adult surveys, the 12-year adult head of household survey will measure outcomes in the five study domains: (1) housing stability, (2) family preservation, (3) adult well-being, (4) child well-being, and (5) self-sufficiency. We reviewed the previous instrument using an equitable evaluation lens to consider whether any of the existing questions had the potential for racial bias and made some revisions that are detailed below.

Survey Topics

Exhibit 5-1 displays the topics we will include in the 12-year adult head of household survey. It summarizes changes from the 37-month adult head of household survey, including items that we removed, added, modified, or simplified for this round of data collection.

Exhibit 5-1. Summary of Adult Head of Household Survey Instrument

36-Month Adult Head of

Household Survey Modules

Summary of Changes to Module Items

Module A: Housing Stability and Added: items on whether respondent ever experienced an eviction; housing program use in History

past year (adapted from 78-month tracking survey); homelessness and/or doubling up in Domain: Housing Stability

past 5 years; homelessness or doubled up in past 5 years; housing assistance since the pandemic

Module B: Family Composition Removed: items about partner separation and reunification of children with parents and Preservation

Added: question about whether any child experienced a separation or foster care placement Domain: Family Preservation

since enrollment, disability status for all household members

Simplified: information about all family members living with the adult head of household

Module C: Housing Quality and Removed: items about takeup of the assistance to which the family was randomly assigned, Affordability

items on whether respondent receives HUD housing assistance for current housing

Domain: Housing Stability

Added: new items on housing crowding (do household members ever sleep elsewhere or in rooms other than bedrooms because there are not enough bedrooms)

Modified: expanded item on number of places stayed from 6 months to 1 year, revised items on housing quality

Module D: Employment Income, Modified: timeframe for employment items to ask about period since beginning of COVID-19

Self-Sufficiency, and Hardship

pandemic (March 2020); and timeframe for items about school or training to ask about period Domain: Self-Sufficiency

since random assignment

Module E: Adult Well-Being

Removed: items on posttraumatic stress disorder, items from HOPE scale Domain: Adult Well-Being

Modified: item on intimate partner violence to ask about past year instead of past 6 months Abt Associates

Family Options 12-Year Study RD/DCAP

May 4, 2022 ▌26

C H A P T E R 5 : P R I M A R Y D A T A C O L L E C T I O N

36-Month Adult Head of

Household Survey Modules

Summary of Changes to Module Items

Module F: Parent on Minor Child Removed: items related to childcare and younger children because focal children are now Domain: Child Well-Being

older; some parenting items related to effort and challenges in raising children Added: item on chronic absenteeism in past school year.

Modified: expanded item on adult child(ren)’s school and training from last data collection to entire study period; expanded the time period for number of schools child(ren) attended from last data collection point to entire study period and ask by type of school (e.g., elementary/middle/high school); expanded the lookback period for items on child behavior problems and suspension/expulsion from 6 months to 1 year; expanded timeframe for items on arrests and police involvement from 6 months to entire study period

Simplified: reduced the number of health questions because we did not find intervention impacts in prior research on closing gaps in health disparities between low-income families and those experiencing homelessness (Grant et al., 2007)

Module G: Service Receipt

Removed: entire module

New Module G: Parent on Adult Added: item for adult children about history of incarceration, educational attainment, family Child

formation, parenthood, timing of parenthood, and housing stability.

Domain: Child Well-Being

Modified: expanded timeframe for items on arrests and police involvement from 6 months to entire study period

New Module H: COVID-19

Added: items on employment, remote schooling, household finances, and housing instability Experiences (NEW)

during the pandemic

Module I: Respondent and Focal Combined: with module below

Child Contact Information

Module J: Contact Information Modified: collect contact information for adult respondent, focal child(ren) 16 or older, adult child(ren), and two secondary contacts

Housing stability domain. We will adjust the recall period for questions about returns to homelessness to cover the most recent 6 months and the 5 years before the survey. We have also eliminated questions about whether the adult head of household receives HUD housing assistance for their current housing, because we can measure receipt of HUD assistance with the HUD administrative data that we will collect for the full study sample for the 12-year followup period. Respondents are not always reliable reporters about their receipt of rental assistance.

Family preservation domain. We will inquire about separations of children from parents and foster care placements for the entire 12-year followup period. These are salient events that are unlikely to be forgotten. We will continue to collect information on household composition. We propose eliminating questions on partner separation and reunification of children. Previous analyses on separation of partners who were with the family in shelter and reunification of children who were separated from the family at the time of enrollment were based on a small sample of families. Also, marital dissolution or continuation is not a central policy focus.

Adult well-being domain. We will assess mental health and alcohol and substance use and dependence. Because we have no reason to expect sleeper effects for adults, we propose eliminating outcomes (such as the posttraumatic stress disorder index)5 for which there were no detectable effects at earlier analyses. In contrast, we will assess health at 12 years despite the 5 Posttraumatic stress disorder was not one of the study’s key outcomes.

Family Options 12-Year Study RD/DCAP

May 4, 2022 ▌27

C H A P T E R 5 : P R I M A R Y D A T A C O L L E C T I O N

absence of earlier impacts, because health outcomes are highly relevant in the context of COVID-19.

Self-sufficiency domain. We will retain our previous approach to measurement. We will eliminate questions about service receipt because that information was relevant only in the first 3

years.

COVID-19 impacts. We are including a new module for adult heads of household about their experiences during the COVID-19 pandemic, including job disruptions, disruptions in children’s schooling and consequent learning losses and effects on parental employment, and housing disruptions.

Parent on Child Modules

In the 37-month data collection, we asked parents to report on focal children ages 3-20, with an additional short battery of questions added for focal children ages 18 and older. For the 12-Year Study, we will ask parents to report on up to six children in one of the two Parent on Child Modules:

• Module F for up to three minor children now ages 10-17:

− Up to two focal children selected previously, now ages 10-17.

− Up to one, two, or three newly selected focal children (either present at baseline or born less than 10 months after random assignment) now ages 10-17. 6

• Module G for up to three adult children now ages 18-30.

Module F, the Parent on Minor Child Module, contains some modifications from the Parent on Child Module used in the 37-month survey. Module G, a Parent on Adult Child Module, is a brief version of the Young-Adult Parent on Child Module used in the 37-month survey. Module G contains questions about educational attainment (high school graduation, postsecondary enrollment and graduation, degrees earned); employment; incarceration; family formation, parenthood, timing of parenthood; and housing stability.

Sample

Abt will attempt to complete an adult head of household survey with all 2,241 adult heads of household in the study. 7 Abt will begin conducting the adult head of household survey by phone.

If it is not possible to conduct the survey by phone, and if public health guidelines regarding social distancing permit in-person interviewing, interviewers may conduct the surveys in person when feasible.

6 The number of newly selected focal children who are ages 10-17 will be determined by how many previously selected focal children there are in this age range. The number of newly selected focal children ages 10-17 will be up to [three minus the number of previously selected focal children in this age range]. For example, if there is one previously selected focal child in the 10-17 age range, then we will attempt to collect up to two new focal children in this age range. If there are no previously selected focal children ages 10-17, we will attempt to collect up to three new focal children in this age range.

7 Because 41 of the 2,282 heads of household in the Family Options Study were confirmed deceased as of the 78-month tracking survey, the starting sample for the 12-year head of household survey is 2,241 adults.

Family Options 12-Year Study RD/DCAP

May 4, 2022 ▌28

C H A P T E R 5 : P R I M A R Y D A T A C O L L E C T I O N

Informed Consent

Before beginning the interview, the study interviewer will verbally obtain the respondent’s consent to participate in the survey.

5.2 Child Survey (Children Ages 10-17)

In conjunction with the adult head of household survey, Abt will conduct a survey with up to two children ages 10-17. For families who have at least one previously selected focal child under age 18, the one or two previously selected minor focal children will be in the target sample for the child survey. 8 For families who do not have a previously selected focal child under age 18, Abt will select up to one minor focal child who either was present in shelter at baseline or was born less than 10 months after baseline.

Survey Topics

Similar to the adult head of household survey, the 12-year child survey builds from the 37-month child instrument. We reviewed the previous instrument using an equitable evaluation lens to consider whether any of the existing questions had the potential for racial bias and made some revisions that are detailed below. Exhibit 5-2 displays the topics included in the 12-year child survey. It summarizes proposed changes from the 37-month child survey, including items that we propose to remove, add, modify, or simplify for this round of data collection.

Exhibit 5-2. Summary of Child Survey Instrument

36-Month Child Survey

Modules

Summary of Changes to Module Items

Domain: Child Well-Being

Module A: Prosocial Behavior Removed: Trait Anxiety Scale for Children (T-Anxiety Scale) Added: the self-administered Strengths and Dificulties Questionnaire, which includes items on internalizing and externalizing problems and prosocial behavior

Module B: Fears

Removed: entire module

Module C: Life Events

No changes

Module D: Youth Risky Behavior Removed: CDC Youth Risk Behavior survey items on tobacco, alcohol, and drug use Added: items on smoking, other tobacco use (including vaping), alcohol, and marijuana, including use before school or work. These questions are slightly modified from the National Longitudinal Survey of Youth and CDC’s National Youth Tobacco Survey

Module E: School Experiences Modified: questions on school attitudes to better understand school outcomes Module F: Parental Monitoring Removed: items on child’s relationship with parents and Involvement

Added: items about child’s interactions with parents and parent’s knowledge of child’s activities

8 The age of children at the start of the fielding period (expected to be November 1, 2022), rather than the age of children on the day of survey response, will determine whether children are considered minor children or adult children for the study data collection efforts. This will allow us to develop predetermined target samples for each survey. Determining ages on a single day for all families means that some children who are 17 years old at the start of the fielding period will have turned 18 years old when we survey their parent and when they respond to the child survey. We consider this an acceptable tradeoff for the simplification in process that adopting this approach will afford us.

Family Options 12-Year Study RD/DCAP

May 4, 2022 ▌29

C H A P T E R 5 : P R I M A R Y D A T A C O L L E C T I O N

36-Month Child Survey

Modules

Summary of Changes to Module Items

Domain: Child Well-Being

Module G: Social Support and Removed: items from Child HOPE scale Community Engagement

Added: items about child’s future plans after high school and items to measure child’s social support, life satisfaction, and community participation

Key: CDC=Centers for Disease Control and Protection.

We will examine adolescents’ school attendance, effort, performance, and disciplinary problems, by self-report as well as parental report. In earlier analyses, we found impacts of the long-term rent subsidies on children’s behavior, as reported by parents, observing both reductions in behavior problems and increases in prosocial behavior. At 12 years, we will ask these questions of both parents and the adolescents themselves, substituting the child version of the Strengths and Difficulties Questionnaire used previously only by parents for the anxiety and goal-oriented thinking questions used previously with children.

Substance use can derail children’s development and thus continues to be an important topic in the survey. We will ask a streamlined set of questions adapted from the National Longitudinal

Survey of Youth that focus on tobacco and broad categories of other substances. We will retain a measure of life events that was associated with other child outcomes in earlier analyses. We will omit the previous measure of “fears” out of concern that it could trigger trauma for children.

We will learn about the adolescent’s future plans, life satisfaction, and sources of adult support through new questions from the Monitoring the Future survey. We will ask questions about community engagement using questions from the National Survey on Drug Use and Health. We also will ask adolescents about parental monitoring, substituting questions from Monitoring the Future for the previous questions from the Strong African American Families study that did not have good psychometric properties and were not predictive of outcomes in previous analyses.

Sample

We will conduct surveys with children ages 10-17:

• Up to two focal children, who were selected as focal children in the 20- or 37-month followup survey, a total of 1,843 children.

• For families who do not have a previously selected focal child in the 10-17 age range, up to one additional child who was present in shelter at baseline or born less than 10 months after random assignment, a total of 377 children.

Informed Consent

Like the prior rounds of data collection, for the child survey we will first acquire parent permission from the adult respondent to administer the child survey to the selected focal children in the responder’s household. Prior to beginning the child survey, the interviewer will also read the child assent form to the child and obtain the child’s assent to proceed.

5.3 Adult Child Survey (Ages 18-30)

For adult children who consent to enroll in the study, Abt will administer a brief, web-based survey.

Family Options 12-Year Study RD/DCAP

May 4, 2022 ▌30

C H A P T E R 5 : P R I M A R Y D A T A C O L L E C T I O N

Survey Topics

In the web-based adult child survey, we will collect information on the following topics:

• Housing stability and experiences of homelessness on their own without their parents.

• Educational attainment: high school graduation or GED, postsecondary education, degrees, training programs; anticipation of future educational attainment.

• Employment and employment disruptions due to COVID-19 pandemic.

• Food security.

• Psychological distress.

• Substance use.

• Intimate partner violence.

• Arrests.

• Family formation; parenting and its timing.

• Access to health care, insurance.

Sample

The sample of adult children is children with their parents in shelter at study enrollment who have now aged into adulthood (ages 18-30). We estimate a total sample of 1,831 of these adult children. These adult children were exposed to the effects of the intervention while living with their family. As part of the adult head of household tracking and survey efforts, we will ask the parents for contact information for up to three adult children. We will then conduct an enrollment call, contacting the adult children by phone to describe the study, confirm their email address to administer the web survey, and obtain their SSN. If the parent does not respond to the 11-year tracking survey (see Chapter 6 for description of this survey) or the adult head of household survey, or does not provide contact information for adult children, we will not attempt to track or obtain consent from their adult children.

Informed Consent

The Abt interviewers will attempt to obtain informed consent and to administer the web survey with adult children whose parents provide contact information during either the 11-year tracking survey or the adult head of household survey. Because these children are now older than age 18, they need to provide their own informed consent to participate in the study.

After we obtain their contact information from their parents, we will conduct an enrollment call with the adult child. During this enrollment call, we will also ask them to provide their SSN and complete the Information Release Form to provide their permission to match their SSNs to administrative data sources (e.g., NDNH) for the current study and for future research endeavors.

Family Options 12-Year Study RD/DCAP

May 4, 2022 ▌31

6. Survey Administration

This section details our plans for administering the primary data collection with adult heads of household, children ages 10-17, and adult children ages 18-30. We describe participant tracking and then the timing of the survey activities, mode, and plans for monitoring and reporting survey results. Our ability to begin data collection activities is contingent upon receiving Office of Management and Budget (OMB) approval.

6.1 Participant Tracking and Re-Engagement

Accurate locating information is crucial to achieving high survey response rates. At the time that the 12-year data collection begins, 5 years will have passed since our last contact with study participants in March 2018. We will conduct a mix of passive and active tracking activities to obtain updated contact information. Passive tracking does not require OMB approval and can begin as soon as we obtain updated extracts from HUD administrative data. Active tracking can occur as soon as a month after receiving OMB approval.

Exhibit 6-1 displays the tracking and re-engagement activities, dependence on OMB approval, timing, and purpose. Our approach draws on multiple contact attempts with varying degrees of intensity and flexible response options.

Exhibit 6-1. Re-Engagement and Survey Activities, OMB Requirements, and Timing Re-Engagement

Contingent on

Activity

OMB Approval

Purpose

Timing

Passive Tracking

HUD PIC/TRACS

No

Provides updated address data for study participants January 2022, with first administrative data

who are stil receiving HUD-funded housing assistance data extract

Proprietary database

No

Provides updated address and phone information, may Initial search 4 months

search

also indicate deceased

after award; ongoing

during data collection

period

Active Tracking

Welcome back/re-

Yes

Alerts participants that the study is resuming. Provides As soon as OMB

engagement letter with

an opportunity to remind the participants of the study approves tracking

Participant Contact

and the upcoming survey data col ection. Participant activities, ideally 6 months Update Form

update contact form collects updated contact

before the start of adult

information to ensure that we have the most current

survey data collection

data available before data collection. We wil send the

re-engagement letter and Participant Contact Update

Form to the adult heads of household to the last

address we have for the respondent. We wil also send

these materials via email for all households for whom

we have an email address

Tracking cal (the “11-

Yes

Tracking call allows interviewers to connect with study Approximately 1-2 months year tracking survey”)

participants and re-engage them while obtaining

after sending the welcome

updated contact information including contact

back/re-engagement letter

information for the adult children. We wil use a similar with Participant Contact tracking survey instrument to that used in the 78-Update Form

month tracking study

Abt Associates

Family Options 12-Year Study RD/DCAP

May 4, 2022 ▌32

C H A P T E R 6 : S U R V E Y A D M I N I S T R A T I O N

During the 78-month data collection, we obtained consent from 44 percent of the total sample for HUD to use participants’ PII to match to other administrative datasets indefinitely. For the 1,272

sample families who have not given consent to share their PII with HUD indefinitely, we will attempt to gain this consent during this tracking process. By completing the Information Release Form, the adult head of household agrees to allow HUD to continue to use their PII, primarily SSNs, to match their information with other research datasets.

If we are not able to locate a respondent to complete the 11-year tracking survey but are able to contact them to participate in the survey, we will request they complete the Information Release Form at that time.

6.2 Timing of Survey Activities

We anticipate obtaining OMB approval for survey data collection by October 2022. We will begin survey data collection approximately four weeks after receiving OMB approval.

Abt will conduct the survey data collection in two phases. Phase 1 is the first 20 weeks of data collection with the full sample of adult heads of household, focal children ages 10-17, and adult children ages 18-30. The interviewers will conduct the child survey after the adult head of household survey, by phone or in person. We assume that we can begin Phase 1 of data collection by the end of November 2022.

Based on the results of the first 20 weeks of data collection, HUD will determine whether to continue to Phase 2 data collection with the full sample, continue to Phase 2 data collection with a partial sample, or discontinue data collection.

If HUD chooses to conduct both Phase 1 and Phase 2 data collection, then our data collection plan involves a 12-month field period for completing the adult head of household and child surveys (Months 13-24 after contract award). 9 With this schedule, the time elapsed between random assignment and the start of the 12-year field period could range from 11 years and 2

months for the families who enrolled in the study at the end of the random assignment period to 12 years and 7 months for the families who enrolled at the beginning of the random assignment period.

Exhibit 6-2 summarizes the information discussed in previous sections about the primary data collection activities. The exhibit shows the expected sample sizes, target response rates, incentives, and interview length for each component of the primary data collection.

Exhibit 6-2. Summary of Primary Data Col ection

Data Col ection Component

Sample Size

Sample Size

(Families)

(Individuals)

Incentive

Interview

Length

Adult Head of Household

2,241

2,241

$50

60 min.

Parent on Child Module

Adult Children (ages 18-30)

1,107

1,831

part of adult head of household

Minor Children (ages 10-17)

1,740

2,858

survey

9 We will complete the adult child data collection three. Seems like consistency makes sense—but I don’t care either way. months after the adult head of household survey is completed.

Family Options 12-Year Study RD/DCAP

May 4, 2022 ▌33

C H A P T E R 6 : S U R V E Y A D M I N I S T R A T I O N

Data Col ection Component

Sample Size

Sample Size

(Families)

(Individuals)

Incentive

Interview

Length

Adult Child Survey (ages 18-30)

1,107

1,831

$35

15 min.

Child Survey (ages 10-17)

1,740

2,220

$25

30 min.

Notes: Adult children includes up to three adult children, prioritizing any previously selected focal children and then selecting among any remaining adult children who were with the family in the emergency shelter at baseline. For the Parent on Child Module, minor children includes up to three children per family, including up to two previously selected focal children. Remaining slots for the Parent on Child Module would be fil ed by not previously selected children with the family in the emergency shelter at baseline and/or not previously selected biological children born less than 10 months after baseline. For the child survey, in families with previously selected minor focal children, minor children includes only those previously selected focal children. In families with no previously selected minor children, new focal children wil be selected. The child survey sample wil include up to one not previously selected minor child either present with the family in the emergency shelter at baseline or born less than 10

months after baseline.

We will use a mixed mode (phone to field) methodology, with local interviewers responsible for all survey data collection. 10 Local interviewers will conduct the following activities to maximize the responses early in the field period.

Administering the Adult Head of Household Survey

Weeks 1-8. During the first eight weeks of data collection, interviewers will attempt to conduct interviews by telephone. Interviewers will use email and text messaging to confirm or set appointments. They will also call secondary contacts to obtain updated contact information for respondents.

Weeks 9-12. If needed, interviewers will begin in-person interviews. The interviewers will conduct in-person interviews only if public health guidelines requiring social distancing have been relaxed. Otherwise, we will continue interview attempts with telephone, email, and text messages only. If in-person interviewing is possible, interviewers will make in-person contact attempts at the address listed for the respondent first and then at the addresses for any alternate contacts. 11

Weeks 12-20. Interviewers will continue to attempt interviews with all non-complete cases; that is, cases where they are still attempting to locate the participant or where they have located but not yet set an appointment. If HUD decides to discontinue data collection after discussions beginning during the 20th week of data collection, the interviewers will complete any previously scheduled interviews and complete all remaining data collection work by May 2023.

10 Though 93 percent of the 78-month respondents reported having internet access, most had access through a tablet or phone. We considered the option of offering an online version of the adult head of household survey. The length and complexity of the survey makes it ill-suited for web administration. Further, a long web survey could be a burden for respondents, as it would increase their monthly cellular data usage.

11 The 20- and 37-month followup surveys also showed that study families moved a lot. By the time of the 20-month followup interview, study families who were initially recruited in 12 states were already dispersed across 42 different states. For the 12-Year Study, the study team is prepared to track families and interview them by phone wherever they are living. We will conduct in-person interviews when necessary and when possible to do so.

Family Options 12-Year Study RD/DCAP

May 4, 2022 ▌34

C H A P T E R 6 : S U R V E Y A D M I N I S T R A T I O N

Administering the Child and Adult Child Surveys

Abt will conduct the child survey data collection with focal children ages 10-17 at the same time as the adult head of household survey.

Though we will begin collecting contact information for the adult children during the tracking call with the adult head of household, we cannot begin enrolling adult children in the study until we obtain OMB approval. We will ask parents for contact information for their adult children and that the parents inform their adult children about the study. As soon as we receive OMB

approval, Abt will begin contacting the adult children to enroll them into the study and to direct those who enroll to the web survey. We will conduct the adult child enrollment concurrently with the adult head of household survey. For adult heads of household who do not complete the tracking call, we will request contact information for the adult children at the time of the adult head of household survey.

After obtaining the contact information for the adult children:

Enrollment call. A field interviewer will contact the adult child using the telephone number provided by their parent to conduct study enrollment. During this enrollment call, the interviewer will briefly explain the study, confirm their contact information, and ask for their SSN. The respondent will also be asked to complete the Information Release Form.

Web survey. Using the contact information provided by the parent and adult child participant, we will email adult children who agree to participate a link to complete a brief survey, including an electronic consent form. We will use text messaging and telephone calls to remind participants to complete the survey.

We will conduct adult child enrollment on an ongoing basis in conjunction with the adult head of household survey.

Incentives

Abt will use incentives to help minimize non-

Exhibit 6-3. Incentives for the Family

Options 12-Year Study

response (and any resulting bias) and offset the

Head of Household Survey

respondents’ costs of participation. Exhibit 6-3

summarizes the planned incentives for tracking

Welcome back/re-engagement letter and update

$15

and survey data collection. The planned

Tracking cal

$20

incentive structure is the same as the 37-month

Adult head of household survey

$50

data collection for the tracking mailing, adult

Child Data Collection (Ages 10-17)

head of household survey, and child survey. We Child survey

$25

propose to increase the incentive for the tracking

Adult Child Data Collection (Ages 18-30)

call to $20 because we have added a new request Adult child enrollment call

$15

for contact information for adult children to that Adult child web survey

$25

call. The incentive for the study of adult children

is $40: $15 provided after the enrollment call and $25 after the adult child completes the web survey.

6.3 Monitoring Survey Data Collection

The study team will monitor data collection closely. During data collection, the study team will submit weekly sample disposition updates to document the number of completed interviews by

Family Options 12-Year Study RD/DCAP

May 4, 2022 ▌35

C H A P T E R 6 : S U R V E Y A D M I N I S T R A T I O N

site, respondent, and intervention group, along with details about the status of locating. The weekly updates will include details about the disposition of each case, such as whether interviewers are attempting to locate the respondent, have made contact, or have an interview scheduled. Interviewers will record detailed notes on each attempt to contact study participants.

Throughout the data collection period, field managers will regularly review all non-completed cases and interviewer notes on contact history, and then work with the survey managers and local interviewers to design strategies for locating each respondent. Two full-time field managers will supervise the day-to-day activities of the field interviewers, including routinely conducting additional searches for study participants for whom we do not have good contact information, using tools such as Accurint searches and the contact history of the case. This approach will allow the field managers to conduct an in-depth review of the attempts completed and then work with the field interviewers to create a tailored strategy to work each case.

We will monitor the data collection progress using the same indicators and reporting developed for the prior phases of data collection. Internally, we will convene weekly production report meetings to review and discuss data collection progress, production rates, locating challenges, sample priorities, and production goals. We will also meet with HUD weekly for the duration of the data collection period to review the production reports, share insights from the field, and discuss any changes in approach if needed. These systems will allow us to track survey completion for the adult head of household, children, and adult children data collection efforts by site, enrollment cohort, intervention group, and randomization set group to ensure comparable completion rates. 12

6.4 Data Collection Metrics

Based on the results of the first 20 weeks of data collection, HUD will determine whether and how to proceed with Phase 2 of data collection. At 20 weeks, the study team will discuss with HUD the number of heads of household who completed the survey, the total number of households that were successfully contacted, and the number of heads of household who we have been able to locate. We will also consider the baseline equivalency for impact estimates and differential attrition among the different intervention groups.

If the adult head of household response rate is 25 percent or greater and locating results indicate the response rate is likely to increase, Abt will begin Phase 2 data collection with the full adult sample. If the response rate is between 20.1 and 24.9 percent and locating results indicate the response rate is likely to increase, Abt will continue adult data collection only with the adults who are part of the SUB vs. UC evaluation sample. If the adult response rate is less than 20

percent after 20 weeks of data collection and locating results indicate it is unlikely that the response rate will increase substantially, Abt will end data collection and will not conduct Phase 2.

If HUD approves Phase 2 data collection, either with the full sample of adult heads of household or with the heads of household in the SUB vs. UC evaluation sample, data collection will continue for an additional 26 weeks. Altogether, if HUD approves Phase 2, all adult head of 12 “Randomization set” refers to the pairwise comparisons for the adult head of household.

Family Options 12-Year Study RD/DCAP

May 4, 2022 ▌36

C H A P T E R 6 : S U R V E Y A D M I N I S T R A T I O N

household data collection will be completed by the end of October 2023. Exhibit 6-4 illustrates the timeline for the data collection process.

Exhibit 6-4. Data Collection Process

Phase 1 Data Col ection- Weeks 1-26

Phase 2 Data Col ection- Weeks 27-52

WEEK 20 Response Rate

Data Collection with the Full Sample Continues

25% or higher

Optional Task 5.3(a) Phase 2 of the Adult Data

Adult Data Col ection

Collection

Data Collection with a Partial Sample Continues

• Task 5.1 Phase 1 Head

20% - 24.9%

Optional Task 5.3(b) Phase 2 of the Adult Data

of Household Data

Collection

Collection

Below 20%

End of Data Collection

Optional Data Collection with the Full Sample

Continues

Optional Task 6.2(a) Survey of Adult Children

Child Data Col ection

25% or higher

(ages 18-29)

Optional Task 6.3(a) Child Survey Data Collection

• Task 6.1 Survey of

(ages 8-17)

Adult Children

(ages 18-29)- Phase 1

Data Collection with a Partial Sample Continues

Optional Task 6.2(b) Survey of Adult Children

• Task 6.3 Child Survey

20% - 24.9%

(ages 18-29)

Data Collection

Optional Task 6.3(b) Child Survey Data Collection

(ages 8-17)- Phase 1

(ages 8-17)

Below 20%

End of Data Collection

Family Options 12-Year Study RD/DCAP

May 4, 2022 ▌37

7. Analysis

This section discusses our approach for the Family Options 12-Year Study data analysis. The research questions and hypotheses for the analysis are presented in exhibit 2-1. Much of this analysis is contingent on HUD exercising the option of Phase 2 data collection with the full adult head of household sample and analysis of that survey data. If HUD does not exercise the option, then we plan to conduct analyses only of the PIC/TRACS and NDNH administrative data.

7.1 Impact Comparisons

The main focus of the analysis is to understand the extent to which beneficial effects of priority access to long-term rent subsidies are still evident after 12 years. In previous reports, we focused on the SUB versus UC comparison to examine the effects of that priority access. In the 12-year analysis, we plan to shift primary attention to a comparison that includes the CBRR and PBTH

groups in the counterfactual to SUB, rather than continuing to highlight the SUB versus UC

pairwise comparison. The main advantage of focusing on a SUB versus UC + CBRR + PBTH

comparison (subsequently referred to as the “SUB versus all other conditions” comparison), rather than SUB versus UC only, is that it provides a larger sample, increasing the precision of the impact estimates. 13 Conceptually, this compares the difference between immediate access to long-term rent subsidies (which are not ordinarily immediately available to families in homeless shelters) to the mix of interventions that are available in most communities (whether offered immediate or usual access to these interventions through the study) accompanied by usual access to long-term rent subsidies.

Although the SUB versus all other conditions comparison has not been previously examined, we analyzed SUB versus CBRR + PBTH in the 20- and 37-month followup reports. Those analyses produced results similar to the SUB versus UC comparison: relatively large favorable effects of SUB. That the differences in the outcomes of the UC, CBRR, and PBTH families at 37 months were relatively modest opens the door to pooling these groups in the 12-year analysis.

Additional justification for the decision to pool the other conditions comes from the observation that CBRR and PBTH families have used long-term rent subsidies at similar rates as the UC

families. 14 Exhibit 7-1 shows rates of current use of any long-term rent subsidy as of March 2019

and December 2021 (the most current data available) and whether families had used a long-term rent subsidy after baseline by these time points. The rates of subsidy use at these time points are generally similar for CBRR, PBTH, and UC groups. The finding of similar subsidy use plus the lack of observable impacts at the 37-month followup between these other conditions leads us to the conclusion that it is reasonable to combine these conditions as a counterfactual to SUB.

The SUB versus all other conditions comparison may provide a better opportunity to observe the effect of SUB relative to SUB versus UC because of both a larger sample size and a larger 13 This impact comparison would not include all randomly assigned families. Instead, it would include only those families who had SUB in their randomization set (i.e., a SUB slot was available, and the family indicated they would be eligible for SUB) at the time of random assignment.

14 The decision about whether to pool CBRR, PBTH, and UC should rest on whether these conditions have provided similar levels of access to long-term rent subsidies. We interpret the finding of similar levels of subsidy use as evidence that the conditions have, in fact, provided similar level of access to subsidies.

Abt Associates

Family Options 12-Year Study RD/DCAP

May 4, 2022 ▌38

C H A P T E R 7 : A N A L Y S I S

contrast in the use of long-term subsidies. Because long-term subsidy use is lower among the CBRR and PBTH groups than among the UC group, there is a larger contrast in subsidy use in the SUB versus all other conditions comparison relative to the SUB versus UC comparison. As of December 2021, the contrast in current subsidy use for SUB versus all other conditions is 15

percentage points, compared to 11 percentage points for SUB versus UC. Likewise, the contrast in the proportion that ever used a subsidy since study enrollment for SUB versus all other conditions (41 percentage points) is larger than that for SUB versus UC (38 percentage points).

This implies that we might expect to see somewhat larger effects for SUB versus all other conditions relative to SUB versus UC only, if effects are present at the 12-year time point.

Exhibit 7-1. Current Use and Any Use Since Baseline of Long-Term Rent Subsidies as of March 2019 and December 2021 Based on HUD IMS/PIC and TRACS Administrative Data, by Assignment Condition and Select Comparison Groups

Currently Using Any Long-Term Rent

Ever Used Any Long-Term Rent

Subsidy as of:

Subsidy Since Study Enrollment as of:

Comparison Sample

Number

Assigned

March 2019

December 2021

March 2019

December 2021

N

%

N

%

N

%

N

%

By Group Assignment

SUB

599

286

47.7

251

41.9

498

83.1

506

84.5

CBRR

569

141

24.8

145

25.6

222

39.0

241

42.4

PBTH

368

574

20.1

76

20.7

122

33.2

144

39.1

UC

746

220

26.0

220

29.5

297

39.8

338

45.3

By Comparison Group

SUB vs. All Other

Conditions

SUB

599

286

47.7

251

41.9

498

83.1

506

84.5

Al Other Conditions

1,162

291

25.0

313

26.9

460

39.6

509

43.8

Difference

22.7

15.0

43.6

40.7

SUB vs. UC

SUB

599

286

47.7

251

41.9

498

83.1

506

84.5

UC

540

148

27.4

165

30.6

226

41.9

250

46.3

Difference

20.3

11.3

41.3

38.2

Notes: For comparability, we use the number of family heads originally assigned (n=2,282). In practice, the analytic sample wil be based on non-deceased family heads, which is a somewhat smaller group (n=2,241 as of 78-month survey). The “ever used” numbers reported previously in the 20- and 37-month reports were based on survey responders. Because we use all family heads originally assigned here, percentages may be lower than those previously reported because the denominator dif ers. Sample size for UC dif ers in the By Group Assignment and By Comparison Group panels because the former includes all families assigned to UC whereas the latter includes only families who were eligible for both SUB and UC at the time of random assignment (e.g., UC families in Atlanta and Baltimore, which did not offer SUB, would not be included in the SUB vs. UC comparison). For the same reason, the sample size for the primary pooled comparison is lower than the sum of number assigned to CBRR, PBTH, and UC in the By Group Assignment panel, as the primary pooled comparison includes only families eligible for assignment to SUB and at least one other condition.

Source: IMS/PIC and TRACS administrative data, 2010-2021.

What might these differences in long-term rent subsidy imply for impact magnitudes? We note that exhibit 7-1 shows a narrowing in current use rates between SUB and the other intervention groups, compared with a difference in use of roughly 50 percentage points in December 2013, when the 37-month followup was conducted. For many outcomes, we expect that what matters for impact is the current (contemporaneous) difference in long-term rent subsidy use. For example, improved housing stability may be directly tied to recent or current rent subsidy use.

Family Options 12-Year Study RD/DCAP

May 4, 2022 ▌39

C H A P T E R 7 : A N A L Y S I S

For these outcomes, we would expect smaller effects from SUB at the 12-year point than those found at the 20- and 37-month points. For other outcomes, particularly for child outcomes, it may be that it is the difference in cumulative use of (or “exposure” to) long-term rent subsidies that matters. For example, cumulative housing stability reducing the accumulation of school moves over time may support later favorable outcomes, even if families are not currently using rent subsidies. Cumulative effects could also arise from an intervention during more sensitive periods of development. For example, reduced behavior problems later in life could stem from cumulative effects of reduced exposure to housing instability in early childhood. Prior research (Andersson et al., 2016) also indicates that differences in cumulative exposure may help account for differences in earnings and incarceration outcomes in young adulthood. For those outcomes, we might expect larger effects from SUB than those found at earlier time points.

Exhibit 7-2 below provides an example of some outcomes we hypothesize may be more sensitive to contemporaneous use and others that may be more sensitive to cumulative use.

Exhibit 7-2. Outcomes Hypothesized to Be More Sensitive to Current vs. Cumulative Rent Subsidy Use

Current Subsidy Use

Cumulative Subsidy Use

• Housing instability: e.g., At least one night

• Child educational outcomes: Grades

homeless or doubled up during past 6 months

• Housing affordability: Rent as percentage of family • Child behavioral problems income

• Adult mental health: Kessler-6 Psychological

• Adult children: Ever incarcerated as adult

Distress Scale

• Child educational outcomes: School absences in

• Adult children: Work for pay in past month

last month

While we plan to focus on the SUB versus all other conditions comparison, we will also estimate impacts for the SUB versus UC, CBRR versus UC, and PBTH versus UC pairwise comparisons.

We will compare SUB versus UC estimated impacts at 12 years to the impacts for this comparison at earlier followup time points. Although we do not expect to find impacts in the CBRR versus UC and PBTH versus UC comparisons, we will examine these comparisons in order to bolster the case for pooling CBRR and PBTH with UC in the SUB versus all other conditions comparison.

7.2 Outcomes and Samples

Another key decision for the analysis (and, of course, for survey instrument design) is which outcomes to examine. In general, we plan to examine the same or similar outcomes as were examined in previous study reports. One difference from previous reports is that we plan to expand the Child Well-Being Domain to include outcomes appropriate for children who have aged into adulthood. We also plan to add some outcomes related to experiences during the COVID-19 pandemic.

The 20- and 37-month study reports did not include impact estimates on outcomes collected during the previous tracking surveys (conducted at 6, 12, and 27 months after study enrollment).

For the 12-Year Study report, we propose to examine a few outcomes related to homelessness, doubling up, and working for pay that were collected on the 78-month tracking survey. The 78-month data have not yet been used for impact analysis. Understanding what these data show

Family Options 12-Year Study RD/DCAP

May 4, 2022 ▌40

C H A P T E R 7 : A N A L Y S I S

about continued impacts of the long-term rent subsidy will be helpful context for interpreting results based on the 12-year data. For completeness, we also propose to examine these same few outcomes from the 11-year tracking survey, which will be fielded approximately 6 months before the 12-year adult survey.

We plan to analyze outcomes for the following eight study samples:

Four family head samples:

• 78-month tracking survey respondent sample.

• 11-year tracking survey respondent sample.

• 12-year adult survey respondent sample.

• Full assigned sample (for PIC/TRACS and NDNH).

Four child samples:

• 12-year parent on child sample for minor children.

• 12-year parent on child sample for adult children.

• 12-year minor child survey respondent sample.

• 12-year adult child survey respondent sample.

Exhibit 7-3 shows the outcomes for adult heads of household and minor children that we plan to analyze. Exhibit 7-4 shows the proposed outcomes for adult children. Even with only one or two main impact comparisons, this is still a large set of outcomes and so will entail a large number of statistical tests. To address the multiple comparisons problem (the substantial likelihood of finding false positives when conducting a large number of tests), we plan to take a similar approach as in previous reports.

First, we will declare a single confirmatory statistical test. We will consider all other statistical tests as exploratory. In exhibit 7-3, the outcome examined in the confirmatory statistical test for the SUB versus all other conditions comparison — at least one night homeless or doubled up in the last 5 years—is indicated by bolding and a superscripted “a.”

Second, we will prespecify a small number of outcomes in each domain to be presented in the executive summary of the report. In exhibits 7-3 and 7-4, we indicate these outcomes in italics with a superscripted “b.”

Exhibit 7-3. Outcomes for Planned Analyses for Family Heads and Minor Children Domain

■ General Outcome

Sample/Data Source

― Specific Outcome

Housing Stability

 Experience of homelessness prior to 78-month tracking survey

78-mo tracking survey

− At least one night homeless or doubled up during past 6 months

respondents

− At least one night homeless during past 6 months

− At least one night doubled up during past 6 months

 Experience of homelessness prior to 11-year tracking survey

11-yr tracking survey

− At least one night homeless or doubled up during past 6 months

respondents

− At least one night homeless during past 6 months

Family Options 12-Year Study RD/DCAP

May 4, 2022 ▌41

C H A P T E R 7 : A N A L Y S I S

Domain

■ General Outcome

Sample/Data Source

― Specific Outcome

− At least one night doubled up during past 6 months

 Experience of homelessness during followup period

12-yr adult survey

− At least one night homeless or doubled up during past 5 yearsa

respondents

− At least one night homeless during past 5 years

− At least one night doubled up during past 5 years

− At least one night homeless or doubled up during past 6 monthsb

− At least one night homeless during past 6 months

− At least one night doubled up during past 6 months

− Number of days homeless or doubled up during past 6 months

− Number of days homeless during past 6 months

− Number of days homeless during past 6 months

 Housing independence

12-yr adult survey

− Living in own house or apartment at time of survey

respondents

− Living in own house or apartment at time of survey with no housing assistance

− Living in own house or apartment at time of survey with housing assistance

 Number of moves

12-yr adult survey

− Number of places lived/stayed during past 12 months b

respondents

 Evictions

12-yr adult survey

− Experience of eviction in past 5 years

respondents

 Housing affordability

12-yr adult survey

− Rent as percentage of family income

respondents

 Housing sufficiency and quality

12-yr adult survey

− Persons per room

respondents

− Crowded housing

− Housing quality is poor or fair

Family Preservation

 Child separations

12-yr adult survey

− Any separation from any child since study enrollment b

respondents

 Placements in foster care

12-yr adult survey

− Any child in a foster care arrangement since study enrollment b

respondents

Self-Sufficiency

 Employment status

78-mo tracking survey

− Work for pay in week before survey

respondents

 Employment status

11-yr tracking survey

− Work for pay in week before survey

respondents

 Employment status

12-yr adult survey

− Work for pay in week before survey b

respondents

− Work for pay in the last month

− Any work for pay in past 1 year

− Any work for pay in past 5 years

− Hours of work per week at main job

 Employment status

All family heads (NDNH

− Any employment by year, for available calendar years

wage records)

 Income sources/amounts

12-yr adult survey

− Total family income b

respondents

− Anyone in family received TANF in last month

− Anyone in family received SSDI in last month

− Anyone in family received SSI in last month

− Anyone in family received SNAP/Food Stamps in last month

Family Options 12-Year Study RD/DCAP

May 4, 2022 ▌42

C H A P T E R 7 : A N A L Y S I S

Domain

■ General Outcome

Sample/Data Source

― Specific Outcome

− Anyone in family received WIC in last month

− Received tax refund check

 Earnings

All family heads

− Earnings by year, for available calendar years

(NDNH wage records)

 Earnings

12-yr adult survey

− Annualized current earnings

respondents

− Anyone in family had earnings in last month

 Education and training

12-yr adult survey

− Participated in 2+ weeks of any school or training since study enrol ment

respondents

− Number of weeks in school/training programs since study enrollment

− Participated in 2+ weeks of school since study enrollment

− Participated in 2+ weeks of basic education since study enrollment

− Participated in 2+ weeks of vocational education since study enrol ment

 Food security

12-yr adult survey

− Household is food insecure in past 30 days b

respondents

− Food insecurity scale (past 30 days)

 Economic stressors

12-yr adult survey

− Economic stress scale

respondents

 Receipt of housing assistance

All family heads

− Any receipt of housing assistance by year, for calendar years since 2015

(PIC/TRACS records)

Adult Well-Being

 Adult physical health

12-yr adult survey

− Health in past 30 days was poor or fair b

respondents

 Adult mental health

12-yr adult survey

− Kessler-6 Psychological Distress Scale b

respondents

 Adult substance use

12-yr adult survey

− Alcohol dependence or drug abuse b

respondents

− Alcohol dependence

− Drug abuse

 Experience of domestic violence

12-yr adult survey

− Experienced intimate partner violence in past year b

respondents

Child Wel -Being

 Child education

Minor focal child in Parent

− School enrollment

on Child Module

− School absences in last month b

− Absent more than 15 days of school in past year

− Number of schools attended since study enrollment b

− Grade completion (not held back)

− Positive school experiences

− Positive school attitudes

− School grades

− School conduct problems in past 12 months

 Child physical health

Minor focal child in Parent

− Poor or fair health b

on Child Module

− Well-child checkup in past year

− Child has regular source of health care

− Sleep problems

− Low birth weight (children born after study enrollment only)

Family Options 12-Year Study RD/DCAP

May 4, 2022 ▌43

C H A P T E R 7 : A N A L Y S I S

Domain

■ General Outcome

Sample/Data Source

― Specific Outcome

 Child Behavioral Strengths and Difficulties

Minor focal child in Parent

− Behavior problems b

on Child Module

− Prosocial behavior

− Arrests or police involvement in the past 6 months

 Self-administered Strengths and Difficulties Questionnaire

Minor focal child survey

− Behavior problems

respondents

− Prosocial behavior

 Substance use

Minor focal child survey

− Any tobacco use last 30 days

respondents

− Any other substance use last 30 days

− Any other substance use at school or work

 School outcomes

Minor focal child survey

− School effort in past month

respondents

− Grades

− Absences

− Difficulties at school

 Parenting (child report)

Minor focal child survey

− Parental monitoring

respondents

− Parental involvement

 New outcomes

Minor focal child survey

− Expectations for future education

respondents

− Parental support

− Other adult support

− Involvement in school- and community-based activities

− Life satisfaction

COVID-19 Pandemic Experiences

 Health during pandemic

12-yr adult survey

− Anyone in household tested positive for COVID-19

respondents

− Anyone in household hospitalized due to COVID-19

− Anyone in household die due to COVID-19

− Received at least one dose of COVID-19 vaccine

 Employment effects of pandemic

12-yr adult survey

− Anyone in household permanently lose job due to COVID-19

respondents

− Anyone in household temporarily lose job due to COVID-19

− Use paid or unpaid leave to care for children

− Caring for children while working

− Cut work hours to care for children

− Leave job to care for children

− Lost job due to providing care for children

 Child education during pandemic

12-yr adult survey

− Child unable to attend school or daycare because of pandemic

respondents

− Children have trouble with remote education because of internet access

 Housing effects of pandemic

12-yr adult survey

− Fall behind on rent or mortgage payments during pandemic

respondents

− Currently behind on rent or mortgage payments

− Eviction very likely or somewhat likely in next 2 months

Key: SSDI=Social Security Disability Insurance. SSI=Supplemental Security Income. SNAP=Supplemental Nutrition Assistance Program.

TANF=Temporary Assistance for Needy Families. WIC=Special Supplemental Nutrition Program for Women, Infants, and Children.

a Prespecified confirmatory outcome.

b Prespecified exploratory outcome for executive summary.

Family Options 12-Year Study RD/DCAP

May 4, 2022 ▌44

C H A P T E R 7 : A N A L Y S I S

Exhibit 7-4. Outcomes for Planned Analyses with Adult Children

Domain

■ General Outcome

Sample/Data Source

― Specific Outcome

 Family formation

Adult focal child in Parent

− Married

on Child Module

− Has a child

 Employment

Adult focal child in Parent

− Work for pay in last month

on Child Module

− Work full-time

− Currently in school or working b

 Education

Adult focal child in Parent

− Highest grade completed

on Child Module

− Has high school diploma

− Has GED

− Has high school diploma or GED b

− Has college degree

− Participated in job training

 Housing stability

Adult focal child in Parent

− Experienced homelessness on their own

on Child Module

− Experienced doubling up on their own

 Police involvement

Adult focal child in Parent

− Been arrested since study enrollment

on Child Module

 Family formation

Adult focal child survey

− Married or long-term relationship

respondents

− Has a child

− Had child as a teenager

 Education

Adult focal child survey

− Has high school diploma

respondents

− Has GED

− Has college degree

− Currently in school

− Participated in job training

− Expect to complete more education

 Employment

Adult focal child survey

− Work for pay in last month

respondents

− Work full-time

 COVID-19 pandemic experiences

Adult focal child survey

− Work hours cut

respondents

− Temporarily lose job

− Permanently lose job

− Reduce hours or leave job due to child care responsibilities

− Prevent from entering labor market

 Economic stress

Adult focal child survey

− Economic stress

respondents

− Worried may not have stable housing in next 2 months

 Housing stability

Adult focal child survey

− Experienced homelessness on their own before age 18

respondents

− Experienced homelessness on their own after age 18

− Experienced doubling up on their own before age 18

− Experienced doubling up on their own after age 18

− Any homelessness or doubling up on their own b

− Living in own place

Family Options 12-Year Study RD/DCAP

May 4, 2022 ▌45

C H A P T E R 7 : A N A L Y S I S

Domain

■ General Outcome

Sample/Data Source

― Specific Outcome

 Substance use

Adult focal child survey

− Any tobacco use last 30 days

respondents

− Any other substance use last 30 days

− Any other substance use at school or work.

 Other outcomes

Adult focal child survey

− Ever arrested

respondents

− Ever incarcerated as an adult

− Ever experienced intimate partner violence

− Kessler-6 Psychological Distress Scale

− Have health insurance

− Health in past 30 days was poor or fair

− Household is food insecure in past 30 days

− Food insecurity scale (past 30 days)

− Involvement in school- and community-based activities

b Prespecified outcome for executive summary.

7.3 Descriptive and Impact Analyses

We will examine means of survey outcomes as in previous study reports. We will also examine use of long-term rent subsidies and employment and earnings outcomes over time based on administrative data. We will conduct a special analysis of those families who relinquish long-term rent subsidies, to characterize their post-assistance situations. This analysis will include families from all assignment groups.

For all assignment groups combined, we will examine the responses to newly added survey questions about experiences during the COVID-19 pandemic. These responses should provide valuable information about how the pandemic affected this sample of low-income families.

For the impact analysis, we plan to use the same impact estimation model as used in previous Family Options Study reports (see Gubits et al. 2013, Section 3.3; Gubits et al., 2016, Appendix C for a detailed description). In sum, impact estimation will be conducted on an intent-to-treat basis, with the focus being understanding the average impact of being assigned to SUB versus being assigned to one of the other three intervention conditions. 15 This reflects the average effect over all families offered the intervention, whether or not they actually took up the intervention that they had priority access to. Estimation will be conducted using a weighted least squares regression with heteroskedasticity-consistent standard errors (i.e., robust standard errors).

For adult outcomes, the regression will include (1) an indicator variable that equals 1 if a family was assigned to the SUB intervention and a 0 if assigned to one of the other three interventions, (2) a vector of background characteristics of the family from the baseline survey (for a full list see Gubits et al. [2016, Appendix C.2]), (3) an indicator for the site-by-random assignment regime for the family, (4) a constant term (intercept). We plan to use the same set of covariates 15 Specifically, the comparison pool is families who had SUB and at least one other intervention (CBRR, PBTH, or UC) available at the time of randomization and were assessed as being eligible at randomization for SUB. (See Gubits et al. [2013, Section 3.3] for additional details on implementation of random assignment.)

Family Options 12-Year Study RD/DCAP

May 4, 2022 ▌46

C H A P T E R 7 : A N A L Y S I S

and take the same approach to the construction of analysis weights. 16 The site-by-random assignment regime accounts for both differing probabilities of random assignment in some sites and one-time changes in assignment probabilities in two study sites. The robust standard errors address both (1) potential for some families to have higher variability in outcomes than other families on characteristics not included in the model and (2) use of linear probability model for binary outcomes (for ease of interpretation from least squares estimates). For minor child and adult child outcomes, the estimation model used is the same, with the exception of robust standard errors being clustered within family to allow for correlation between impacts on children in the same family.

To address non-response in the adult survey, we will prepare a set of non-response weights based on family characteristics measured in the baseline survey that attempt to adjust for non-response (Little, 1986). 17 These weights will be used in estimating impacts for all family and adult outcomes. For child and adult child survey outcomes, we will develop separate analysis weights that incorporate the adult survey non-response weight, the inverse probability of being selected as a focal child, and a child non-response weight. We note that new focal child selection weights will need to be created due to new focal children being selected in the present study. (See the sample descriptions provided for each instrument in Chapter 6.) The probability of being selected will be a joint function of the probability of focal child selection in the prior surveys and the probability of selection in the current followup survey. The aim of this analysis is to equally represent all children in all study families for child outcomes and all adult children in study families for adult child outcomes. Therefore, focal children in families with more children receive more weight in the analyses than do focal children from families with fewer children.

7.4 Moderation of Impacts (Subgroups)

Another important decision for the analysis is whether and how to conduct analyses of impact moderation (i.e., how impact magnitudes may vary for families with varied characteristics). In the 20- and 37-month analyses, we examined whether impacts were moderated by baseline psychosocial challenges and baseline housing barriers. We did not find clear evidence of moderation of effects in prior impact analyses for either moderator (Gubits et al., 2015, 2016).

For the present study, we do not plan to examine moderation of effects by these indicators.

Other participant characteristics, however, could be considered for moderation analyses. An important consideration is that analyses of distinct subgroups would have lower statistical power 16 Previously selected focal children will continue to be analyzed using their previously constructed sampling weights. For newly selected focal children, we will construct sampling weights that are the inverse of the probability of selection. We will construct non-response weights for the analysis using the same method used at the 20-month and 37-month analyses. These weights will address potential bias from unit non-response to the adult survey, the child survey, and the adult child survey. The final analysis weight for parent-reported child outcomes will be the product of the family non-response weight and the child sampling weight. The final analysis weight for child survey and adult child survey outcomes will be the product of the family non-response weight, the child sampling weight, and the child survey/adult child survey non-response weight.

17 A small amount of missing data on baseline covariates was addressed using a single stochastic imputation (see Gubits et al., 2013, Section 3.3).

Family Options 12-Year Study RD/DCAP

May 4, 2022 ▌47

C H A P T E R 7 : A N A L Y S I S

to detect effects than would the analyses of the continuous moderators previously tested. We propose conducting two subgroup analyses of substantive interest.

Differences in intervention impacts by race on housing outcomes were explored in additional research conducted with Family Options Study data. Solari et al. (2021) compared White and Black families offered SUB versus those offered usual care, finding similar voucher lease-up and exit rates in the SUB group and similar favorable impacts for returns to homelessness, doubling up, and residential moves as of the 37-month followup survey. Subgroup differences by Hispanic ethnicity have not been explored. It is possible that cumulative effects could differ from shorter-term effects, however, so we propose to test for subgroup differences by race and ethnicity. At baseline, 41 percent of the family heads were African American, non-Hispanic; 21 percent White, non-Hispanic; 20 percent Hispanic, all races; 11 percent Mixed race or other race, non-Hispanic; 7 percent Asian/Pacific Islander, non-Hispanic. We plan to focus on three groups: Black, non-Hispanic; White, non-Hispanic; and Hispanic and would subset the sample. 18 The moderation analysis would drop the existing race/ethnicity baseline covariates and add four terms to the standard impact estimation model: two dummy variables for Black, non-Hispanic and for Hispanic, respectively (with White, non-Hispanic as the reference category) and two dummy variables for the interaction of SUB with Black, non-Hispanic and the interaction of SUB with Hispanic. We plan to conduct this subgroup analysis for the set of pre-specified executive summary outcomes.

We also plan to test whether child age moderates impacts on child outcomes. As discussed in Section 2, we expect to see variability by age from prior research and prior study findings.

Among minor children, we propose two age groups based on age at baseline: age 2 or younger (who will generally be between ages 10 and 13 at the 12-year followup) and age 3 or older (who will generally be between ages 14 and 17 at followup). Among adult children, we propose dividing children age 10 or younger at baseline (who would be expected to be age 22 or younger at followup) from those age 11 or older at baseline (who would be between age 23 to 30 at followup).

We propose creating a summary variable for indicating whether a child is faring well or not in key outcomes for their age. For minor children, this variable would include (1) no behavior problems (as indicated by a parent- or child-reported total SDQ score below the screening cutoff for behavioral problems), (2) child on-track in educational progress for age, (3) report of at least one positive relationship with an adult. For adult children, this summary variable would include: (1) has a GED or high school diploma, (2) is currently enrolled in postsecondary education or is currently employed, (3) did not have children as a teenager, (4) is not incarcerated as an adult, and (5) has not experienced homelessness on their own as an adult. We would focus on this summary variable in our analyses but would also explore the pattern of results for tests on each individual outcome.

18 The Asian/Pacific Islander, non-Hispanic group is largely from the Honolulu site, which would complicate interpretation of findings—whether differences are reflective of factors particular to Honolulu relative to factors attributable to differences in race (i.e., site and race are confounded for this group). We expect that the sample size for the Mixed or Other race category would be too small to meaningfully interpret estimates.

Family Options 12-Year Study RD/DCAP

May 4, 2022 ▌48

C H A P T E R 7 : A N A L Y S I S

7.5 Sample Sizes and Statistical Power

For the confirmatory outcome of at least one night homeless or doubled up in the past 5 years, we estimated minimum detectable effects (MDEs) under two sets of assumptions about survey response rates. MDEs are the smallest intervention effects that researchers can be confident of detecting as statistically significant for analysis of a given sample size.

Exhibit 7-5 presents MDEs for the confirmatory outcome for a comparison of SUB versus the pooled comparison group for two sample sizes—one assuming a 50 percent response rate and one assuming a 75 percent response rate. 19 MDEs are presented as percentage point differences between SUB and the pooled control group. Consistent with prior study analyses, we propose using .10 as the level of statistical significance for hypothesis testing.

Exhibit 7-5. Minimum Detectable Effects for Confirmatory Homelessness Outcome for SUB vs.

Al Other Conditions, By Response Rate

Expected # of Completed

Followup Survey Interviews

MDE If Mean Outcome for Al Other Conditions Group Is:

Response Rate

SUB

All Other

Conditions

10% (or 90%)

30% (or 70%)

50%

75% Response Rate

440

855

4.3 pp

6.6 pp

7.2 pp

50% Response Rate

294

570

5.3 pp

8.0 pp

8.8 pp

Key: MDE=minimum detectible effect.

Notes: MDEs are based on calculations assuming two-sided tests are used at the 10 percent significance level, the desired power is 80

percent, and the regression R2 is 0.04. No finite population correction was used. Sample size calculations based on total number of non-deceased family heads as of 78-month fol owup survey who were eligible for inclusion in the comparison.

To provide a frame of reference, the full sample mean for the outcome at least 1 night homeless or doubled up during the past 6 months in the SUB vs. UC impact estimate was 34.0 percent (in exhibit 7-5, closest to the 0.3 column), with a mean difference of −18.2 percentage points.

Impacts were not analyzed from the 78-month tracking data, but the full-sample mean for this outcome was 19.5 percent. As discussed previously, the contrast in receipt of SUB relative to the other intervention conditions has declined since the 37-month survey, so it is possible that the mean difference may be smaller. The confirmatory outcome in the present study has a longer time horizon, which would tend to increase the proportion of both groups reporting any experience of homelessness. However, further time elapsing since an initial shelter stay may also reduce families’ risk of experiencing homelessness.

7.6 Cost and Non-Experimental Analysis

This unfunded subtask includes both a 12-year update of the cost analysis and non-experimental data analyses of the importance of long-term housing subsidy receipt. HUD would need to decide to execute this option by early 2023 to allow for sufficient time for analyses to be completed and results integrated into the final report. In October 2024, we will draft a summary 19 Though these cutoffs were selected as broad potential benchmarks, we note that the 78-month tracking survey had completed interviews with roughly half of the sample of family heads (49 percent) and determined that 75

percent of the family head sample was still viable at that time.

Family Options 12-Year Study RD/DCAP

May 4, 2022 ▌49

C H A P T E R 7 : A N A L Y S I S

memo that documents the data we collected, the time period covered, the data covered, and our findings. We will also incorporate the analyses into the study’s final report.

Cost Analyses

Should HUD choose to exercise this task, we will update the study’s original cost analysis that focused on the cost of all program use during the 37-month followup period. We will estimate the cost of program use for the post-37-month program use data that are available:

• Monthly data, derived from IMS/PIC and TRACS records, on the use of long-term rent subsidies;

• 78-month tracking survey data on the use of emergency shelter, permanent supportive housing, transitional housing, and rapid rehousing in the past six months;

• 11-year tracking survey data on the use of emergency shelter, permanent supportive housing, transitional housing, and rapid rehousing in the past six months; and

• 12-year followup survey data on the use of emergency shelter, permanent supportive housing, transitional housing, and rapid rehousing in the past six months.

We do not plan to estimate program costs for unobserved use of emergency shelter, permanent supportive housing, transitional housing, and rapid re-ousing in the years between survey waves.

Therefore, we will not calculate the cumulative cost of all program use over 12 years. Instead, we will present the cumulative cost of long-term rent subsidies since the 37-month followup and program costs for the time periods proximate to the 78-month tracking survey, the 11-year tracking survey, and the 12-year followup survey.

The costs of long-term rent subsidies can be calculated for each family directly from HUD

administrative data. The calculation would include housing assistance payments in the HCV and similar programs or imputed value of housing assistance provided by public housing (using Small Area Fair Market Rents to approximate economic cost of units with no market-based rental rate). Because program use and monthly costs to HUD are directly observed, this element of costs—the total housing assistance per family provided by HUD over the 12-year followup period—can be estimated precisely. We can report impact differences for this cost element as a separate outcome for the SUB versus all other conditions and the SUB versus UC comparisons.

For emergency shelter, transitional housing, and rapid rehousing, we will combine the available program use data with estimates of the monthly costs of these program types to calculate dollar costs of program use for the periods we observe program use. We will base the estimates of monthly costs on the monthly costs used in the 37-month analysis, updating for inflation and for any known changes in program delivery (for details on methodology, see Gubits et al. 2015,

[Appendix G]).

We will also include the cost of permanent supportive housing in the cost estimates for the time periods proximate to the survey waves. Since use of permanent supportive housing sometimes may be observed in administrative records as well as in survey data, we will reconcile the data sources so as not to inadvertently double count program use. In the 20-month and 37-month analyses, we performed this type of reconciliation (or cleaning) for all program types. In this 12-year analysis, the absence of Homeless Management Information System (HMIS) records simplifies the reconciliation process. We will look to administrative records for long-term rent

Family Options 12-Year Study RD/DCAP

May 4, 2022 ▌50

C H A P T E R 7 : A N A L Y S I S

subsidy use, survey data for emergency shelter, transitional housing, and rapid rehousing use, and reconcile administrative and survey data for permanent supportive housing.

Non-Experimental Analyses

Although the main impact analysis of the 12-Year Study will provide information about the effects of priority access to long-term rent subsidies, they will not directly address whether continuing use of subsidies has a cumulative effect on family outcomes. To what extent does longer use of subsidies confer greater benefits on families? And, for families who are not using a long-term subsidy at the 12-year point, to what extent does past use of a subsidy have enduring benefits? We plan to conduct a non-experimental analysis to address these questions. The analysis will explore the extent to which the length of use of long-term rent subsidies is correlated with study outcomes among all study families. Controlling for “never used a long-term subsidy” and “current use of a long-term subsidy,” this analysis will help us to better understand how current versus cumulative use associates with intervention impacts.

We will first use IMS/PIC and TRACS records to compute the total number of years of long-term subsidy use since baseline by the family head as of the date that they completed the 12-year followup survey. Then we will include this measure as an explanatory variable in a series of regressions on key study outcomes. We will use a weighted least-squares regression model with heteroskedasticity-consistent standard errors (i.e., robust standard errors). We expect that the regression model will include (1) an intercept term, (2) the same set of baseline covariates used in the impact models, (3) a continuous measure of long-term rent subsidy use, ranging from 0 to 12 years since baseline, (4) a binary variable for “ever used a long-term subsidy since baseline”

as of the time of the 12-year followup survey (i.e., more than 0 years of use), and (5) a binary variable for “current long-term subsidy use,” which indicates the family was using a long-term subsidy at the time of the 12-year followup survey. The coefficient for the years of program use would indicate the expected change in the outcome for each additional year of long-term subsidy use, controlling for whether the family was currently using a subsidy and whether they had ever used a subsidy since baseline. The final analytic methods will be documented and reported in a technical appendix to the report in which these findings fall.

7.7 Approach for Developing a Unified Dataset

The Family Options Study Dataset encompasses three types of data files:

1. Public use files with a limited set of variables, no direct identifying variables, and masked indirect identifying variables.

2. Restricted access files with all analysis variables, suitable for reproducing published results and conducting further research, but no direct identifying variables.

3. Files of personally identifiable information (PII) for families who have consented to release their PII to HUD in order for other research and administrative data linking to be conducted.

These data files and their accompanying documentation files are described in the Family Options Study Public Use Files (PUF) and Restricted Access Files User’s Guide (submitted to HUD in March 2017) and in an addendum to that guide submitted in September 2019. The 12-Year Study will build on the existing dataset structure for updating the public use files, restricted access files, and the PII files.

Family Options 12-Year Study RD/DCAP

May 4, 2022 ▌51

C H A P T E R 7 : A N A L Y S I S

Exhibits 7-7 (public use files), 7-8 (restricted access files), and 7-9 (PII files) list the data files to be included in the unified dataset. These exhibits note the new types of information that we will integrate into existing datasets and list the new datasets that we plan to prepare.

Some of the data file preparation is contingent on HUD choosing to fund data analysis of survey data collected in the 12-Year Study. The exhibits note which planned file contents are contingent upon this funding.

Below we highlight key additions and updates to the unified dataset that we plan to prepare for the 12-Year Study:

1. Adult survey files. We will prepare public use and restricted access datasets for survey information related to the family head and the household. These datasets will have similar formats to those prepared for the 20- and 37-month adult survey data.

2. Focal children files. We will prepare separate data files for minor focal child and adult focal child data collected in the 12-Year Study. We will also update the focal child information file to include newly selected focal children.

3. Family composition file. We will update the family composition file from the 11-year tracking survey and the 12-year followup survey. If individual-level information is available from an administrative data match on mortality, then we will also update the deceased flag (and, if available, add a “date of death” variable).

4. Consent to release PII file. Tracking and followup survey efforts may expand the set of individuals who have provided consent to release PII to HUD. This file will reflect consent status, including withdrawals of consent, should they occur.

5. HUD administrative data—program use events file. We will update the file generated in 2019

with HUD IMS/PIC and TRACS program use events through March 2025. (Should HUD

choose not to fund data analysis of 12-Year Study survey data, then we will update this file using IMS/PIC and TRACS records through March 2023.)

Exhibit 7-7. List of Planned Public Use Data Files and Updates to Files

Data File

Existing

Updates

1. Baseline Covariates and Random Assignment Records

Yes

No change

2. 20-Month Study Outcomes and 20-Month Non-Response

Yes

No change

Weights

3. Focal Child Data—Baseline Covariates, 20-Month Followup

Yes

No change

Adult Survey, 20-Month Child Assessments, 20-Month Study

Outcomes, and 20-Month Analytic Weights

4. 37-Month Study Outcomes and 37-Month Non-Response

Yes

No change

Weights

5. Focal Child Data—Baseline Covariates, 37-Month Followup

Yes

No change

Adult Survey, 37-Month Child Assessments, 37-Month Study

Outcomes, and 37-Month Analytic Weights

6. Program Use and Living Situation

Yes

No change

7. Program Use (87 families only, for replication of 3-Year

Yes

No change

Results, now outdated)

8. Housing Status

Yes

Update to add housing status from 12-

year followup survey data

Family Options 12-Year Study RD/DCAP

May 4, 2022 ▌52

C H A P T E R 7 : A N A L Y S I S

Data File

Existing

Updates

9. 12-Year Study Outcomes* and 12-Year Non-Response

No

New file to record analysis variables

Weights*

from the 12-year adult survey

10. Minor Focal Child Data—Baseline Covariates,* 12-Year

No

New file to record select items and

Followup Adult Survey, 12-Year Child Survey, 12-Year Study

minor focal child outcomes from 12-

Outcomes*, and 12-Year Analytic Weights*

year minor child data collection

11. Adult Focal Child Data—Baseline Covariates,* 12-Year

No

New file to record select items and

Followup Adult Survey, 12-Year Adult Child Survey, 12-Year

adult focal child outcomes from 12-year

Study Outcomes,* and 12-Year Analytic Weights*

adult child data collection

Note: * = contingent on HUD funding of survey data analysis.

Exhibit 7-8. List of Planned Restricted Access Data Files and Updates to Files Data File

Existing

Updates

1. Baseline Survey, Baseline Covariates, Random Assignment

Yes

No change

Records, and Enrollment Verification

2. 20-Month Followup Adult Survey, 20-Month Study Outcomes,

Yes

No change

and 20-Month Non-Response Weights

3. Focal Child Data—Baseline Covariates, 20-Month Followup

Yes

No change

Adult Survey, 20-Month Followup Child Survey, 20-Month

Child Assessments, 20-Month Study Outcomes, and 20-Month

Analytic Weights

4. 37-Month Followup Adult Survey, 37-Month Followup Study

Yes

No change

Outcomes, and 37-Month Non-Response Weights

5. Focal Child Data—Baseline Covariates, 37-Month Followup

Yes

No change

Adult Survey, 37-Month Followup Child Survey, 37-Month

Child Assessments, 37-Month Study Outcomes, and 37-Month

Analytic Weights

6. Program Use and Living Situation

Yes

No change

7. Program Use at 20 Months (for replication of Short-Term

Yes

No change

Impacts Report results, now outdated)

8. Program Use (87 families only, for replication of 3-Year

Yes

No change

Impacts Report results, now outdated)

9. Family Composition at Each Survey Wave

Yes

Update the file to add information from

new tracking survey, 12-year followup

adult survey, and, if applicable,

administrative mortality data

10. Housing Status

Yes

Update to add housing status from 12-

year followup survey data

11. Census Tracts, Census Block Groups, and ZIP Codes

Yes

No change

12. Focal Child Information

Yes

Update to add newly selected focal

children

13. Program Use and Living Situation Events

Yes

No change

14. Dates of Adult Surveys

Yes

Update to add information from 12-year

followup survey data

15. Weights for Families who Consented to Have HUD Receive

Yes

No change

PII

16. 78-Month Tracking Survey

Yes

No change

17. Program Use Events From HUD PIC and TRACS Records

Yes

Update to include program use events

Through March 2019

through March 2023/March 2025*

Family Options 12-Year Study RD/DCAP

May 4, 2022 ▌53

C H A P T E R 7 : A N A L Y S I S

Data File

Existing

Updates

18. Flag Variables for Consent to Release PI

Yes

Update to reflect additional consents

given (or any consents withdrawn)

19. 12-Year Followup Adult Survey, 12-Year Study Outcomes,*

No

New file to record 12-year adult survey

and 12-Year Non-Response Weights*

data and outcomes

20. Minor Focal Child Data— Baseline Covariates,* 12-Year

No

New file to record 12-year minor focal

Followup Adult Survey, 12-Year Child Survey, 12-Year Study

child data from each instrument and

Outcomes*, and 12-Year Analytic Weights*

minor focal child outcomes

21. Adult Focal Child Data— Baseline Covariates,* 12-Year

No

New file to record 12-year adult focal

Followup Adult Survey, 12-Year Adult Child Survey, 12-Year

child data from each instrument and

Study Outcomes,* and 12-Year Analytic Weights*

adult focal child outcomes

Note: * = contingent on HUD funding of survey data analysis.

Exhibit 7-9. List of Planned PII Data Files and Updates to Files

Data File

Existing

Updates

1. PII File for Family Heads

Yes

Updated to reflect changes in consent-

to-release-PII status

2. PII File for Children

Yes

Updated to reflect changes in consent-

to-release-PII status

3. PII File for Adult Children

No

New file to record PII from adult

children who have consented to

release their PII to HUD

Consistent with prior deliverables, we will create sample programs for reading in the data, new codebooks for each new dataset, and updated codebooks for any updated datasets. We will also create a new addendum to the User’s Guide that includes the list of revised and new files, how to use the additions, an update on PII data files, and the surveys for the primary data collection.

Family Options 12-Year Study RD/DCAP

May 4, 2022 ▌54

References

Andersson, F., Haltiwanger, J. C., Kutzbach, M. J., Palloni, G. E., Pollakowski, H. O., and Weinberg, D. H. (2016). Childhood housing and adult earnings: A between-siblings analysis of housing vouchers and public housing (No. w22721). National Bureau of Economic Research.

Baptista, I., Benjaminsen, L., Busch-Geertsema, V., and Pleace, N. (2017). Family homelessness in Europe. EOH Comparative Studies on Homelessness, Vol 7. European Observatory on Homelessness. https://www.feantsaresearch.org/download/feantsa-

studies_07_web3386127540064828685.pdf

Brown, S. (2021). How do housing interventions for families experiencing homelessness affect children’s functioning? [Unpublished doctoral dissertation]. Vanderbilt University.

Bush, H., and Shinn, M. (2017). Families’ experience of doubling up after homelessness.

Cityscape, 19(3), 331–356. NIHMSID 927086.

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5760191/

Chetty, R., and Hendren, N. (2018). The impacts of neighborhoods on intergenerational mobility I: Childhood exposure effects. The Quarterly Journal of Economics, 133(3), 1107–1162.

Chetty, R., Hendren, N., and Katz, L. (2016). The effects of exposure to better neighborhoods on children: New evidence from the Moving to Opportunity project. American Economic Review 106 (4). https://scholar.harvard.edu/hendren/publications/effects-exposure-better-

neighborhoods-children-new-evidence-moving-opportunity

Coley, R. L., Leventhal, T., Lynch, A. D., and Kull, M. (2013). Relations between housing characteristics and the well-being of low-income children and adolescents. Developmental Psychology, 49(9), 1775–1789. https://doi.org/10.1037/a0031033

Collins, C. C., Rong, B., Fischer, R., Crampton, D., Lalich, N., Liu, C., and Chan, T. (2020).

Housing instability and child welfare: Examining the delivery of innovative services in the context of a randomized controlled trial. Children and Youth Services Review, 108.

https://doi.org/10.1016/j.childyouth.2019.104578

Cowal, K., Shinn, M., Weitzman, B. C., Stojanovic, D., and Labay, L. (2002). Mother-child separations among homeless and housed families receiving public assistance in New York City. American Journal of Community Psychology, 30, 711-730.

Fisher, Benjamin W., Lindsay S. Mayberry, Marybeth Shinn, and Jill Khadduri. (2014).

“Leaving Homelessness Behind: Housing Decisions Among Families Exiting Shelter,”

Housing Policy Debate, 24(2), 364-386.

Fowler, P. J., and Chavira, D. (2014). Family Unification Program: Housing services for homeless child welfare–involved families. Housing Policy Debate, 24(4), 802‒814.

https://doi.org/10.1080/10511482.2014.881902

Abt Associates

Family Options 12-Year Study RD/DCAP

May 4, 2022 ▌55

R E F E R E N C E S

Fowler, P. J., Henry, D. B., and Marcal, K. E. (2015). Family and housing instability: Longitudinal impact on adolescent emotional and behavioral well-being. Social Science Research, 53, 364–374. http://dx.doi.org/10.1016/j.ssresearch.2015.06.012

Fowler, P. J., Henry, D. B., Schoeny, M., Taylor, J., and Chavira, D. (2014). Developmental timing of housing mobility: longitudinal effects on externalizing behaviors among at-risk youth. Journal of the American Academy of Child and Adolescent Psychiatry, 53(2), 199–

208. https://doi.org/10.1016/j.jaac.2013.12.003

Gubits, Daniel, Brooke Spellman, Lauren Dunton, Scott Brown, and Michelle Wood. (2013).

Interim Report: Family Options Study. Report prepared by Abt Associates. Washington, DC: Government Printing Office, U.S. Department of Housing and Urban Development.

https://www.huduser.gov/publications/pdf/HUD_503_Family_Options_Study_Interim_Repo

rt_v2.pdf

Gubits, Daniel, Marybeth Shinn, Stephen Bell, Michelle Wood, Samuel Dastrup, Claudia D.

Solari, Scott R. Brown, Steven Brown, Lauren Dunton, Winston Lin, Debi McInnis, Jason Rodriguez, Galen Savidge, and Brooke E. Spellman. (2015). Family Options Study: Short-Term Impacts of Housing and Services Interventions for Homeless Families. Report prepared by Abt Associates. Washington, DC: Government Printing Office, U.S. Department of Housing and Urban Development.

http://www.huduser.gov/portal/portal/sites/default/files/pdf/FamilyOptionsStudy_final.pdf

Gubits, Daniel, Marybeth Shinn, Michelle Wood, Stephen Bell, Samuel Dastrup, Claudia D.

Solari, Debi McInnis, Tom McCall, and Utsav Kattel. (2016). Family Options Study: Three-Year Impacts of Housing and Services Interventions for Homeless Families. Report prepared by Abt Associates. Washington, DC: Government Printing Office, U.S. Department of Housing and Urban Development.

https://www.huduser.gov/portal/sites/default/files/pdf/Family-Options-Study-Full-Report.pdf

Gubits, D., Shinn, M., Wood, M., Brown, S., Dastrup, S. R., and Bell, S. H. (2018). What interventions work best for families who experience homelessness? Impact estimates from the Family Options Study. Journal of Policy Analysis and Management 37, 835-866.

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6168747/

Grant, R., Shapiro, A., Joseph, S., Goldsmith, S., Rigual-Lynch, L., and Redlener, I. (2007). The health of homeless children revisited. Advances in Pediatrics, 54(1), 173–187.

https://doi.org/10.1016/j.yapd.2007.03.010

Harvey, H. (2020). Cumulative effects of doubling up in childhood on young adult outcomes. Demography, 1–28. http://dx.doi.org/10.1007/s13524-020-00860-0

Haveman, R., Wolfe, B., and Spaulding, J. (1991). Childhood events and circumstances influencing high school completion. Demography, 28(1), 133–157.

Huston, A. C. (2017). U.S. commentary: Effects of housing subsidies on the well-being of children and their families in the Family Options Study. Cityscape, 19(3), 265-270.

https://www.huduser.gov/portal/periodicals/cityscpe/vol19num3/article15.html

Family Options 12-Year Study RD/DCAP

May 4, 2022 ▌56

R E F E R E N C E S

Jacob, B. A., and Ludwig, J. (2012). The effects of housing assistance on labor supply: Evidence from a voucher lottery. American Economic Review, 102, 272-304.

Johnson, G., and Watson, J. (2017). International commentary: The implications of the Family Options Study for family homelessness in Australia. Cityscape, 19(3), 211–217.

Levitt, A. J., Mitchell, K., Pareti, L., DeGenova, J., Heller, A., Hannigan, A., and Gholston, J.

(2013). Randomized trial of intensive housing placement and community transition services for episodic and recidivist homeless families. American Journal of Public Health, 103(S2) , 348–354. https://doi.org/10.2105/AJPH.2013.301521

Little, R. J. A. (1986). Survey nonresponse adjustments for estimates of means, International Statistical Review, 54 (2), 139-157.

Mahathey, A., Brown, S., de Sousa, T., Loya, K., Walker, J.T., and Santucci, A. (2021).

Findings from the Transitional Living Program Youth Outcomes Study. OPRE Report #

2021-191, Washington, DC: Office of Planning, Research, and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services.

Mayberry, L. S. (2016). The hidden work of exiting homelessness: Challenges of housing service use and strategies of service recipients. Journal of Community Psychology, 44, 293–310.

https://doi.org/10.1002/jcop.21765

Mayberry, L. S., Shinn, M., Benton, J. G., and Wise, J. (2014). Families experiencing housing instability: The effects of housing programs on family routines and rituals. American Journal of Orthopsychiatry, 84(1), 95–109. https://doi.org/10.1037/h0098946

Mills, G., Gubits, D., Orr, L., Long, D., Feins, J., Kaul, B., … and Joes, A. (2006). Effects of housing vouchers on welfare families: Final Report. Washington DC: Office of Policy Development and Research, U.S. Department of Housing and Urban Development.

Morton, M. H., Dworsky, A., and Samuels, G. M. (2017). Missed opportunities: Youth homelessness in America. National estimates. Chapin Hall at the University of Chicago.

https://voicesofyouthcount.org/wp-content/uploads/2017/11/VoYC-National-Estimates-

Brief-Chapin-Hall-2017.pdf

O’Sullivan, E. (2017). International commentary: Family Options Study observations from the periphery of Europe. Cityscape, 19(3), 203–209.

Park, J. M., Metraux, S., Brodbar, G., and Culhane, D. (2004). Child welfare involvement among children in homeless families. Child Welfare, 83(5), 423-436.

Pergamit, M., Cunningham, M., Hanson, D., and Stanczyk, A. (2019). Does supportive housing keep families together? The Urban Institute.

https://www.urban.org/research/publication/does-supportive-housing-keep-families-together

Rafferty, Y., Shinn, M., and Weitzman, B. C. (2004). Academic achievement among formerly homeless adolescents and their continuously housed peers. Journal of School Psychology, 42, 179-199.

Family Options 12-Year Study RD/DCAP

May 4, 2022 ▌57

R E F E R E N C E S

Ripka, S. (2018). Randomized control trial of Housing First for families in Brno [Paper presentation]. FEANTSA Research Conference, Budapest, Hungary.

https://www.feantsaresearch.org/download/101_midday_ws_10_ripka693946094032534673

4.pdf

Rodriguez, J. M., and Shinn, M. (2016). Intersections of family homelessness, CPS involvement, and race in Alameda County, California. Child Abuse & Neglect, 57, 41–52.

https://doi.org/10.1016/j.chiabu.2016.06.004

Rog, D. J., Henderson, K. A., Lunn, L. L., Greer, A. L., and Ellis, M. L. (2017). The interplay between housing stability and child separation: Implications for practice and policy.

American Journal of Community Psychology, 60, 114-124.

https://doi.org/10.1002/ajcp.12148

Rog, D. J., Henderson, K. A., Wagner, C. A., and Abbruzzi, E. L. (2021). Housing matters, services might: Findings from the high needs families program evaluation. Annals of the American Academy of Political and Social Science, 693(1), 209‒229.

https://doi.org/10.1177/0002716221997037

Rollins, C., Glass, N., Perrin, N., Billhardt, K. A., Clough, A., Barnes, J., Hanson, G., and Bloom, T. L. (2012). Housing instability is as strong a predictor of poor health outcomes as level of danger in an abusive relationship: Findings from the SHARE study . Journal of Interpersonal Violence, 27(4), 623–643. https://doi.org/10.1177/0886260511423241

Samuels, J., Fowler, P. J., Ault-Brutus, A., Tang, D. I., and Marcal, K. (2015). Time-limited case management for homeless mothers with mental health problems: Effects on maternal mental health. Journal of the Society for Social Work and Research, 6(4), 515–539.

https://doi.org/10.1086/684122

Shinn, M., Samuels, J., Fischer, S. N., Thompkins, A., and Fowler, P. J. (2015). Longitudinal impact of Family Critical Time Intervention on children in high-risk families experiencing homelessness: A randomized trial. American Journal of Community Psychology, 56, 205–

216 . https://doi.org/10.1007/s10464-015-9742-y

Solari, C.D., Walton, D., and Khadduri, J. (2021). How well do housing vouchers work for Black families experiencing homelessness? Evidence from the Family Options Study. The Annals of the American Academy of Political and Social Science, 693 (1), 193-208.

https://doi.org/10.1177%2F0002716221996678

Suglia, S. F., Duarte, C. S., and Sandel, M. T. (2011). Housing quality, housing instability, and maternal mental health. Journal of Urban Health 88(6), 1105–1116.

https://doi.org/10.1007/s11524-011-9587-0

Swanson, C. B., and Schneider, B. (1999). Students on the move: Residential and educational mobility in America’s schools. Sociology of Education, 72(1), 54–67.

Family Options 12-Year Study RD/DCAP

May 4, 2022 ▌58

A P P E N D I X B . A D M I N I S T R A T I V E D A T A E L E M E N T S

Appendix A. Family Options Study as a Platform for

Research

In addition to the project deliverables authored for HUD, the Family Options Study has provided a platform for extensive research on family homelessness and interventions to end it by team members and other scholars. HUD has archived the study’s data at the U.S. Census Bureau to allow for continued analyses.

Below we list some of the publications and research briefs, beginning with two compendia—a set of research briefs commissioned by the U.S. Department of Health and Human Services (HHS), Administration for Children and Families (ACF) and a special issue of Cityscape coedited by Anne Fletcher and Michelle Wood. We then list the additional referenced journal articles of which we are aware, two additional HUD reports that draw on Family Options Study data, and one dissertation.

ACF Homeless Families Research Briefs

The Administration for Children and Families and the Office of Planning, Research, and Evaluation (OPRE) contracted with Abt Associates to produce ten research briefs using data from the Family Options Study. The analyses focused not on experimental impacts but on issues related to the well-being and economic self-sufficiency of families and children experiencing homelessness.

2016

Burt, M. R., Khadduri, J., and Gubits, D. (2016). Are homeless families connected to the social

safety net? Homeless Families Research Brief, OPRE Report No. 2016-33.

• This brief shows that families experiencing homelessness had participation rates in social safety net programs that are greater than or equal to those of other deeply poor families.

Walker, J. T., Brown, S. R., and Shinn, M. (2016). Adolescent well-being after experiencing

family homelessness. Homeless Families Research Brief, OPRE Report No. 2016-42.

• This brief shows that 20 months after staying in an emergency shelter with their families, adolescents exhibited more behavior problems, less positive behavior, and more chronic school absence than their peers nationally at all income levels.

2017

Brown, S. R., Shinn, M., and Khadduri, J. (2017). Well-being of young children after

experiencing homelessness. Homeless Families Research Brief, OPRE Report No 2017-06.

• This brief shows that 20 months after a stay in an emergency shelter, young children were disadvantaged in many, but not all, areas of development compared to same-age peers nationally.

Walton, D., Dunton, L., and Groves, L. (2017). Child and partner transitions among families

Abt Associates

Family Options 12-Year Study RD/DCAP

May 4, 2022 ▌59

A P P E N D I X A . F A M I L Y O P T I O N S S T U D Y A S A P L A T F O R M F O R R E S E A R C H

experiencing homelessness. Homeless Families Research Brief, OPRE Report No. 2017-26.

• This brief shows that families experiencing homelessness have high levels of separation among family members, and a bidirectional relationship between continued housing instability and family separations.

Khadduri, J., Burt, M. R., and Walton, D. (2017). Patterns of benefit receipt among families who

experience homelessness. Homeless Families Research Brief, OPRE Report No. 2017-42.

• This brief explores family characteristics associated with receipt of benefit and finds a bidirectional relationship between ongoing housing instability and benefit receipt.

Khadduri, J., Walton, D., López, M., and Burt, M. R. (2017). Hispanic families experiencing

homelessness. Homeless Families Research Brief, OPRE Report No. 2017-78.

• This brief shows that 20 months after a shelter stay, Hispanic families fared better than White or African American families on some measures of housing instability and well-being, with regional differences among Hispanic families mirroring regional differences among other groups.

2018

Walton, D., Wood, M., and Dunton, L. (2018). Child separation among families experiencing

homelessness. Homeless Families Research Brief, OPRE Report No. 2018-39.

• This study showed that nearly 40 percent of families experiencing homelessness were separated at some point from one or more children, often for long periods, with housing instability associated with separations.

Walton, D., Dastrup, S., and Khadduri, J. (2018). Employment of families experiencing

homelessness. Homeless Families Research Brief, OPRE Report No. 2018-56.

• This brief shows that employment rates of families experiencing homelessness were lower than for other deeply poor families in the same communities. Rates rose over time (to 38 percent after three years), but employment remained unstable.

Shinn, M., Gubits, D., and Dunton, L. (2018). Behavioral health improvements over time among

adults in families experiencing homelessness. Homeless Families Research Brief, OPRE

Report No. 2018-61.

• This brief showed substantial improvements in mothers’ and fathers’ behavioral health over time, especially among families who attained housing stability.

Glendening, Z., and Shinn, M. (2018). Predicting repeated and persistent family homelessness:

Do Families’ characteristics and experiences matter? Homeless Families Research Brief, OPRE

Report No.2018-104.

Family Options 12-Year Study RD/DCAP

May 4, 2022 ▌60

A P P E N D I X A . F A M I L Y O P T I O N S S T U D Y A S A P L A T F O R M F O R R E S E A R C H

• This brief showed that family characteristics have little relationship to persistent homelessness either before or after families were recruited to the Family Options Study.

Special Issue of Cityscape, 19(3) (2017).

The issue of Cityscape included an introduction from the editors, six substantive papers based on analyses of data from the Family Options study, and commentaries from U.S. and international scholars.

Fletcher, A., and Wood, M. (2017). Next steps for the Family Options Study. Cityscape, 19(3), 191-202. https://www.huduser.gov/portal/periodicals/cityscpe/vol19num3/guest2.html

• This introduction provides a short overview of findings from the Family Options Study and an overview of the remaining papers.

Wood, M., and Fletcher, A. (2017). Lessons for conducting experimental evaluations in complex field studies: Family Options Study. Cityscape, 19(3), 271-292.

https://www.huduser.gov/portal/periodicals/cityscpe/vol19num3/article16.html

• This article reviews four key lessons learned in the process of executing the study: the need for flexibility when testing existing models of assistance, the complexity of identifying study sites, the importance of ethical considerations, and methods for conducting a rigorous experiment in the context of program restrictions on the families they would serve.

Shinn, M., Brown, S. R., Spellman, B. E., Wood, M. L., Gubits, D., and Khadduri, J. (2017).

Mismatch between homeless families and the homelessness service system. Cityscape, 19(3), 293-307. NIHMSID 927088.

https://www.huduser.gov/portal/periodicals/cityscpe/vol19num3/article17.html

• This article examines the enrollment phase of the Family Options Study and shows how initial screening of families for program requirements prior to random assignment, subsequent screening by programs, and choices by families combine so that fewer than a third of families screened for transitional housing moved in, compared to about half of those screened for rapid rehousing and nearly three quarters of those screened for long-term rental subsidies.

Glendening, Z., and Shinn, M. (2017) Risk models for returns to housing instability among families experiencing homelessness. Cityscape, 19(3), 309-330. NIHMSID 927087.

https://www.huduser.gov/portal/periodicals/cityscpe/vol19num3/article18.html

• This article shows that although housing interventions have strong relationships with subsequent homelessness and housing stability, few family characteristics measured at the time of shelter entry, other than previous housing instability, have predictive utility.

Bush, H., and Shinn, M. (2017). Families’ experience of doubling up after

Family Options 12-Year Study RD/DCAP

May 4, 2022 ▌61

A P P E N D I X A . F A M I L Y O P T I O N S S T U D Y A S A P L A T F O R M F O R R E S E A R C H

homelessness. Cityscape, 19(3), 331-356. NIHMSID927086.

https://www.huduser.gov/portal/periodicals/cityscpe/vol19num3/article19.html

• This study uses qualitative data to show that many of the doubled-up situations families can access after an episode of homelessness are unstable and potentially harmful.

Gubits, D., McCall, T., and Wood, M. (2017). Family Options Study: Effects on family living situation. Cityscape, 19(3), 357-386.

https://www.huduser.gov/portal/periodicals/cityscpe/vol19num3/article20.html

• This article examines families’ living situations, month by month, during the first 32

months after random assignment, and the impacts of the interventions on living in the family’s own place, on leaving that place, and on doubling up with a relative or friend.

Solari, C. D., and Khadduri, J. (2017). Family Options Study: How homeless families use housing choice vouchers. Cityscape, 19(3), 387-412.

https://www.huduser.gov/portal/periodicals/cityscpe/vol19num3/article21.html

• This article shows that families given priority access to voucher subsidies leased up at very high rates, 82 percent, with self-reported substance abuse and felony convictions reducing, but not eliminating families’ ability to use vouchers. Among families without priority access to a voucher, those with a prior history of doubling up were more likely to later use vouchers.

The issue of Cityscape also included multiple comments from U.S.-based and international scholars. The U.S. scholars focused on family well-being (Curtis), food security (Waxman), interpersonal violence (Allen), child welfare (Fowler), and the well-being of children (Huston).

The international scholars provided comparative insights from Ireland (O’Sullivan), Australia (Johnson & Watson), and Canada (Nelson).

Curtis, M. A. (2017). U.S. commentary: The Family Options Study and family well-being

outcomes. Cityscape,19(3), 229-234.

Waxman, E. (2017). U.S. commentary: The Family Options Study and food

insecurity. Cityscape, 19(3), 235-244.

Allen, N. E. (2017). U.S. commentary: Insights from the Family Options Study regarding

housing and intimate partner violence. Cityscape, 19(3), 245-254.

Fowler, P. J. (2017). U.S. commentary: Implications from the Family Options Study for

homeless and child welfare services. Cityscape, 19(3), 255-264.

Huston, A. C. (2017). U.S. commentary: Effects of housing subsidies on the well-being of

children and their families in the Family Options Study. Cityscape, 19(3), 265-270.

O’Sullivan, E. (2017). International commentary: Family Options Study observations from the

periphery of Europe. Cityscape, 19(3), 203-210.

Family Options 12-Year Study RD/DCAP

May 4, 2022 ▌62

A P P E N D I X A . F A M I L Y O P T I O N S S T U D Y A S A P L A T F O R M F O R R E S E A R C H

Johnson, G., and Watson, J. (2017). International commentary: The implications of the Family

Options Study for family homelessness in Australia. C ityscape, 19(3), 211-218.

Nelson, G. (2017). International commentary: Eliminating family homelessness and the Family

Options Study. Cityscape, 19(3), 219-228.

Additional Referenced Journal Articles

The Family Options Study has provided a valuable platform for a variety of additional analyses, using the study data archived at the census, the qualitative data funded by the National Institute of Child Health and Human Development, and additional data matched to study data by other authors. Both members of the study team and researchers unaffiliated with the team have made numerous contributions. It is unlikely that we are aware of all of them.

Cutuli, J. J., and Herbers, J. E. (2018). Housing interventions and the chronic and acute risks of family homelessness: Experimental evidence for education. Child Development, 90(5), 1664-1683.

https://srcd.onlinelibrary.wiley.com/doi/full/10.1111/cdev.13041

• This study matched study families in one site with housed low-income students to examine educational outcomes. Children randomly assigned to UC perform as well or better than children assigned to housing interventions in this municipality.

Fisher, B. W., Mayberry, L. S., Shinn, M., and Khadduri, J. (2014). Leaving homelessness behind: Housing decisions among families exiting shelter. Housing Policy Debate, 24(2), 364-386.

http://www.ncbi.nlm.nih.gov/pmc/articles/PMC4170684/

• This study uses qualitative data to understand families’ views of the options to which they were assigned, their reasons for turning down priority offers of assistance, and unhappy compromises many made. Familiar neighborhoods near children’s schools, transportation, family and friends, and stability were important to families across conditions.

Glendening, Z., McCauley, E., Shinn, M., and Brown, S.R. (2018). Long-term housing subsidies and SSI/SSDI income: Creating health-promoting contexts for families experiencing housing instability with disabilities. Disability and Health Journal, 11, 214-220.

https://www.ncbi.nlm.nih.gov/pubmed/28851508

• This article showed that the third of study families that reported a family member with a disability at study entry experienced greater housing instability, food insecurity, and economic stress and less work and income than families without a disability, but among these families, receipt of SSI/SSDI income predicted fewer returns to emergency shelter, and more income despite less work. Offers of long-term housing subsidies increased SSI/SSDI receipt.

Family Options 12-Year Study RD/DCAP

May 4, 2022 ▌63

A P P E N D I X A . F A M I L Y O P T I O N S S T U D Y A S A P L A T F O R M F O R R E S E A R C H

Gubits, D., Shinn, M., Wood, M., Brown, S., Dastrup, S. R., and Bell, S. H. (2018). What interventions work best for families who experience homelessness? Impact estimates from the Family Options Study. Journal of Policy Analysis and Management 37, 835-866.

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6168747/

• This paper, which won the 2018 Raymond Vernon Memorial Award for best paper in the Journal of Policy Analysis and Management, summarized study findings about how the usual care group fared over time and the effects of interventions at 20 and 37 months. It is an article-length summary of findings from the short-term and long-term outcomes report.

Mayberry, L. S. (2016). The hidden work of exiting homelessness: Challenges of housing service use and strategies of service recipients. Journal of Community Psychology, 44, 293-310.

https://onlinelibrary.wiley.com/doi/full/10.1002/jcop.21765

• This study used qualitative interviews to explore challenges and “Catch-22s” parents experienced in using services, strategies they employed to navigate services, and characteristics of positive and negative service experiences.

Mayberry, L. S., Shinn, M., Benton, J. G., and Wise, J. (2014). Families experiencing housing instability: The effects of housing programs on family routines and rituals. American Journal of Orthopsychiatry, 84(1), 95-109.

http://www.ncbi.nlm.nih.gov/pmc/articles/PMC4089513/

• This study used qualitative interviews to understand the challenges that living situations created for maintenance of family processes that can protect parents, children, and families from the detrimental effects of stressors such as homelessness. Challenges were greatest in shelters, transitional housing, and doubled-up situations.

McInnis, D. and Rodriguez, B. (2016). Tracking and interviewing Family Options Study participants. Cityscape, 18(2), 201-219.

https://www.huduser.gov/portal/periodicals/cityscpe/vol18num2/ch12.pdf

• This study describes methods of tracking and interviewing participants that yielded an 80% response rate at 20 months and a 78% response rate 3 years after study enrollment.

Rodriguez, J. M., and Shinn, M. (2016). Intersections of family homelessness, CPS involvement, and race in Alameda County, California. Child Abuse and Neglect, 57, 41-52.

https://www.ncbi.nlm.nih.gov/pubmed/27318034

• Matched data from a single site showed that over half of study families were reported to child protective services at some point. Reports increased in the months leading up to shelter entry and unsubstantiated reports spiked immediately afterward. Black families were disproportionately likely to be reported, but race was not related to substantiated reports or removals.

Shinn, M., Brown, S. R., and Gubits, D. (2017). Can housing and service interventions reduce

Family Options 12-Year Study RD/DCAP

May 4, 2022 ▌64

A P P E N D I X A . F A M I L Y O P T I O N S S T U D Y A S A P L A T F O R M F O R R E S E A R C H

family separations for families who experience homelessness? American Journal of Community Psychology, 60, 79-90.

https://www.ncbi.nlm.nih.gov/pubmed/28012168

• This paper examines the extent of separations of children from parents and of partners from each other and shows that as of the 20-month followup, long-term rental subsidies almost halved rates of child separation and more than halved rates of foster care placements. Homelessness and substance abuse additionally predicted child separations.

Shinn, M., Brown, S. R., Wood, M., and Gubits, D. (2016) Housing and service interventions for families experiencing homelessness in the United States: An experimental

evaluation. European Journal of Homelessness, 10(1), 13-30.

https://www.feantsaresearch.org/download/10-1_article_18262665053505208916.pdf

• This article invited by the European Journal of Homelessness summarized study findings from the 20-month followup.

Shinn, M., Gibbons-Benton, J., and Brown, S. R. (2015). Poverty, homelessness, and family break-up. Child Welfare, 94 (1), 105-122. NIHMSID 927084.

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5760188/

• This study examines the extent and correlates of family separations in families experiencing homelessness and documents the reasons for separations from parents’

perspectives, including economic hardship, shelter conditions, and family circumstances.

Solari, C. D., Walton, D., and Khadduri, J. (2021). How well do housing vouchers work for Black families experiencing homelessness? Evidence from the Family Options Study. The Annals of the American Academy of Political and Social Science, 693 (1), 193-208.

https://journals.sagepub.com/doi/full/10.1177/0002716221996678

• This paper shows that the Housing Choice Voucher program worked similarly for Black and White families. Lease-up rates were uniformly high across groups, and rates of exit did not differ by race.

Additional HUD reports

Two additional reports are archived on the HUD website. One by the study team is an additional analysis of rapid rehousing programs in the context of the broader literature. The other uses qualitative data to understand facilitators and barriers to preschool enrollment for study families.

Stillman, L., Hurd, K., Kieffer, C., Taylor, J., and Gibson, B. (2016). A qualitative assessment of

parental preschool choices and challenges among families experiencing homelessness.

Cloudburst Group.

• This study involved interviews and focus groups with 28 participants in two Family Options sites. Parents encountered multiple barriers in enrolling their children in

Family Options 12-Year Study RD/DCAP

May 4, 2022 ▌65

A P P E N D I X A . F A M I L Y O P T I O N S S T U D Y A S A P L A T F O R M F O R R E S E A R C H

preschool; housing stability, access to social-support networks, and support from early childhood and social service systems facilitated enrollment.

Walton, D., Wood, M., Rodriguez, J., Khadduri, J., Gubits, D., Dunton, L., and Shinn, M.

(2018). Understanding rapid re-housing: Supplemental analysis of data from the Family

Options Study. Washington, DC: U.S. Department of Housing and Urban Development.

• This report synthesized available literature on rapid rehousing programs and presented new analysis about short-term rent subsidies and associated services provided by rapid rehousing programs in the Family Options Study.

Dissertation

The most comprehensive analysis of child outcomes across the 37 months of the study is a dissertation.

Brown, S. R. (2021). How do housing interventions for families experiencing homelessness affect children’s functioning? Dissertation, Vanderbilt University. ProQuest Dissertations Publishing, 2021. 28842765.

• This dissertation is organized as three empirical articles. The first uses qualitative data to explore parents’ perceptions of how their housing environments affect children’s functioning. The second explores heterogeneity in functioning of 8–17-year-old children, showing that 60 percent of children functioned well across educational, behavioral, and health outcomes, but 40 percent had more serious challenges. Access to long-term rental subsidies was associated with membership in the higher functioning class, with effects mediated by increased housing stability and quality, reduced economic stress and family stress, and improved family routines. The third paper conducts a parallel analysis for 3–7-year-old children who also fell into higher and lower functioning groups, with some of the same mediators as for older children. Long-term rental subsidies were important predictors of higher functioning for 3–4-year-olds, but interventions did not matter for 5–

7-year-olds.

Family Options 12-Year Study RD/DCAP

May 4, 2022 ▌66

A P P E N D I X B . A D M I N I S T R A T I V E D A T A E L E M E N T S

Appendix B. Administrative Data Elements

This section describes the data elements retrieved from administrative data sources. Exhibits B-1

and B-2 describe data elements collected from HUD IMS-PIC and HUD TRACS administrative data, respectively. Exhibit B-3 describes administrative data collected from NDNH.

Appendix Exhibit B-1. IMS-PIC Data Elements

Variable Name

Description

Format

SSN_HEAD

Social Security number of head of household

Character

PARTICIPANT_CODE

Public Housing Authority code number

Character

PROGRAM_TYPE

Program type

Character

TYPE_OF_ACTION

Type of action

Character

HOMELESS_IND

Homeless at entry

Character

TOT_A_INCOME

Total household annual income

Numeric

ADJUSTED_A_INCOME

Adjusted annual income

Numeric

TTP

Total tenant payment

Numeric

EFFECTIVE_DATE

Effective date

Character

ADMISSION_DATE

Admission date

Character

SSN_MBR

Social Security number of household member

Character

MBR_NUMBER

Household member number

Character

MBR_LAST_NAME

Household member last name

Character

MBR_FIRST_NAME

Household member first name

Character

MBR_M_INITIAL

Household member middle initial

Character

MBR_SEX

Household member sex

Character

RELATIONSHIP

Household member relationship to HoH

Character

MBR_DOB

Household member DOB

Character

index

Concatenated First Name, Last Name, and DOB

Character

updt

Last updated date

Numeric

Abt Associates

Family Options 12-Year Study RD/DCAP

May 4, 2022 ▌67

A P P E N D I X B . A D M I N I S T R A T I V E D A T A E L E M E N T S

abt_id

Abt-provided ID number for study participant heads of

Numeric

household and their spouse/partner (Note: not a family-

level ID)

sub

Abt-provided variable for whether family was randomly

Numeric

assigned to SUB intervention condition

matched_on_SSN

Household member was matched on SSN

Character

dups

Frequency count (=1 al rows)

Numeric

matched_on_NameDOB

Household member was matched on Name + DOB

Character

member_count

Total number of household members

Numeric

dependent_count

Total number of dependents in household

Numeric

PAYMENT_STANDARD

Payment standard for family

Numeric

UTIL_ALLOW

Utility al owance

Numeric

RENT_TO_OWNER

Rent to owner

Numeric

GROSS_RENT

Gross rent of unit (Rent + Utility Al owance)

Numeric

HAP_TO_OWNER

HAP to owner (HCV: Tenant Based-Vouchers)

Numeric

FAMILY_RENT_TO_OWNER

Family rent to owner (HCV: Tenant-Based Vouchers)

Numeric

NORMAL_TOT_HAP

Total HAP

Numeric

CEILING_RENT

Income-based ceiling rent, if any (amount)

Numeric

LOWER_RENT

Lower of TTP or income-based ceiling rent

Numeric

MAX_RENT

Public housing maximum rent

Numeric

MAX_SUBSIDY

Public housing maximum subsidy

Numeric

FLAT_RENT_AMT

Unit’s flat rent amount

Numeric

TYPE_OF_RENT_CODE

Type of rent selected (income-based or flat)

Numeric

FLAT_INC_BASED_SUBSIDY Flat income-based subsidy

Numeric

CEILING_RENT_IND

Ceiling rent applied? (Y/N)

Character

prog

HUD program type

Character

program

Regular or MTW program

Character

UNIT_STREET_NAME

Unit street name

Character

UNIT_STREET_SUFFIX

Unit street suffix

Character

Family Options 12-Year Study RD/DCAP

May 4, 2022 ▌68

A P P E N D I X B . A D M I N I S T R A T I V E D A T A E L E M E N T S

UNIT_DIRECTIONAL_SIGN

Unit street directional sign

Character

UNIT_APT_NUM

Unit apartment number

Character

STATE_CODE

Unit state code

Character

ZIP_CODE

Unit ZIP Code (+4)

Character

STD_ADDR

Standardized street address

Character

STD_CITY

Standardized city

Character

STD_ST

Standardized state

Character

STD_ZIP5

Standardized ZIP

Character

STD_ZIP9

Standardized ZIP + 4 Code

Character

city

Unit city

Character

DEVELOPMENT_CODE

Development code

Character

ADDRESS_LINE2_TEXT

Address line 2

Character

dev_buil_nu_entrance_id

PHA Code Plus Unit and Building Entrance Number

Character

Appendix Exhibit B-2. TRACS Data Elements

Variable Name

Description

Format

ssn_mbr

Social Security number of household member

Character

mbr_number

Household member number

Character

mbr_first_name

Household member first name

Character

mbr_m_initial

Household member middle initial

Character

mbr_last_name

Household member last name

Character

relationship

Household member relationship to HoH

Character

mbr_sex

Household member sex

Character

ssn_head

Social Security number of head of household

Character

index

Concatenated First Name, Last Name, and DOB

Character

mbr_dob

Household member DOB

Character

CERTIFICATION_TYPE

Certification type

Character

Family Options 12-Year Study RD/DCAP

May 4, 2022 ▌69

A P P E N D I X B . A D M I N I S T R A T I V E D A T A E L E M E N T S

MOVE_IN_DATE

Move in date

Character

type_of_action

Transaction type

Character

program_type

Program type (subsidy type)

Character

TOTAL_TENANT_PMT

Total tenant payment

Numeric

TENANT_RENT_AMT

Tenant rent amount

Numeric

adjusted_a_income

Adjusted income amount

Numeric

ASSTANCE_PMT_AMT

Assistance payment amount

Numeric

tot_a_income

Annual income amount

Numeric

abt_id

Abt-provided ID number for study participant heads of

Numeric

household and their spouse/partner (Note: Not a family-

level ID)

sub

Abt-provided variable for whether family was randomly

Numeric

assigned to SUB intervention condition

matched_on_SSN

Household member was matched on SSN

Character

program

Regular or MTW program

Character

dups

Frequency count (=1 al rows)

Numeric

matched_on_NameDOB

Household member was matched on Name + DOB

Character

rent_to_owner

Contract rent amount

Numeric

util_allow

Utility al owance amount

Numeric

GROSS_RENT

Gross rent

Numeric

WELFARE_RENT

Welfare rent

Numeric

MARKET_RENT

Market rent

Numeric

member_count

Total number of household members

Numeric

dependent_count

Total number of dependents in household

Numeric

STATE_CODE

Unit state code

Character

STR_ADDR

Standardized street address

Character

STD_CITY

Standardized city

Character

STD_ST

Standardized state

Character

STD_ZIP5

Standardized ZIP

Character

Family Options 12-Year Study RD/DCAP

May 4, 2022 ▌70

A P P E N D I X B . A D M I N I S T R A T I V E D A T A E L E M E N T S

STD_ZIP9

Standardized ZIP + 4 Code

Character

unit_street_name

First address line

Character

address_line2_text

Second address line

Character

city

Unit city

Character

unit_apt_num

Unit number

Character

address_line3_text

Second address line

Character

zip_code

Unit ZIP Code

Character

Appendix B-3. NDNH Data Elements

Variable Name

Description

Format

processed_date

Quarterly wage processed date

Numeric

pseudo_fein

Pseudo federal employer identification number (FEIN)

Character

QW_employer_state

Employer state

Character

QW_employee_wage_amt

Employee wage amount

Character

QW_reporting_quarter

Reporting period—quarter

Character

QW_reporting_year

Reporting period—year

Character

QW_submit ed_State_code

Submit ed state code

Character

QW_transmit er_agency_code

Transmit er agency code

Character

QW_transmit er_state_code

Transmit er state code

Character

i_randset_group

Random assignment group and groups for which the

Numeric

family appeared eligible at random assignment

i_ra_quarter

Calendar quarter of random assignment

Numeric

i_site_ratio

Site random assignment regime

Numeric

fam_ineligible

Eligibility indicator (to distinguish families dropped from

Numeric

final analysis sample)

hoh

Head of household indicator (to distinguish adult heads

Numeric

of household from spouses)

Family Options 12-Year Study RD/DCAP

May 4, 2022 ▌71

U.S. Department of Housing and Urban Development

Office of Policy Development and Research

Washington, DC 20410-6000

October 2022

Document Outline

Research Design, Data Collection, and Analysis Plan The Family Options 12-Year Study

CONTENTS

List of Exhibits

1. Introduction

1.1 Background on the Family Options Study

1.2 Organization of Document

2. Previous Findings from the Family Options Study

2.1 Key Findings from the 20- and 37-Month Analyses

Program Use After a Stay in Shelter

Outcomes for Families Assigned to Usual Care

Insights from 80 In-Depth Interviews with Families

Findings from the Impact Analysis Comparing the Randomized Groups

Costs of the Interventions

2.2 Creating a Research Platform for Learning About Family Homelessness

2.3 Rationale for 12-Year Followup on Family Options Study

3. Research Questions for the 12-Year Study

3.1 Literature Review and Rationale

Research Question 1: Outcomes Over 12 Years for Families Not Randomized to SUB

Research Questions 2a/c: Impact of SUB on Housing Stability and Family Well-Being

Research Question 2b: Impact of SUB on Children as They Age

Research Question 3: COVID-19, Housing Stability, and Health

3.2 Data Sources

4. Administrative Data Sources

4.1 Currently Funded Administrative Data Sources

IMS/PIC and TRACS

NDNH

4.2 Optional Administrative Data Sources

IMS/PIC and TRACS Data on Adult Children

Additional NDNH Data Collection

National Student Clearinghouse Postsecondary Education Data

Research Data Assistance Center for Medicaid Claims

National Center for Health Statistics for Mortality Data

5. Primary Data Collection

5.1 Adult Head of Household Survey

Survey Topics

Parent on Child Modules

Sample

Informed Consent

5.2 Child Survey (Children Ages 10-17)

Survey Topics

Sample

Informed Consent

5.3 Adult Child Survey (Ages 18-30)

Survey Topics

Sample

Informed Consent

6. Survey Administration

6.1 Participant Tracking and Re-Engagement

6.2 Timing of Survey Activities

Administering the Adult Head of Household Survey

Administering the Child and Adult Child Surveys

Incentives

6.3 Monitoring Survey Data Collection

6.4 Data Collection Metrics

7. Analysis

7.1 Impact Comparisons

7.2 Outcomes and Samples

7.3 Descriptive and Impact Analyses

7.4 Moderation of Impacts (Subgroups)

7.5 Sample Sizes and Statistical Power

7.6 Cost and Non-Experimental Analysis

Cost Analyses

Non-Experimental Analyses

7.7 Approach for Developing a Unified Dataset

References

Appendix A. Family Options Study as a Platform for Research

ACF Homeless Families Research Briefs

2016

2017

2018

Special Issue of Cityscape, 19(3) (2017).

Additional Referenced Journal Articles

Additional HUD reports

Dissertation

Appendix B. Administrative Data Elements