name: title class: center, middle, inverse, my-one-page-font <style type="text/css"> .remark-slide-content { font-size: 25px; padding: 1em 4em 1em 4em; } .my-one-page-font { font-size: 20px; } </style> # Understanding the relationships between risk factors, intersectional identities and criminal career trajectories: A multilevel approach Ben Matthews¹, Susan McVie¹, Carleen Thompson² and Anna Stewart² Q-Step Exeter, 2020-11-11 ¹University of Edinburgh ²Griffith University .pull-left[ <!-- --> ] .pull-right[ <img src="https://esrc.ukri.org/esrc/includes/themes/MasterSite/images/logo-1.png" width="60%" /> ] --- # Overview 1. __Research background__: Exploring inequalities in the cumulative impact of childhood system contact on adult criminal conviction trajectories by sex and Indigenous status 2. __Methods__: multilevel approaches to quantitative intersectionality 3. __Results __ --- class: center, middle, inverse # Research Background --- # Research background 1. How do __childhood experiences__ affect __adult convictions__ histories? 2. Adult convictions patterns are __heterogeneous__, and so we often summaries them as a set of __discrete trajectories__ with different shapes 3. Key insight one: childhood experiences affect adult convictions __cumulatively__. Our focus is on 'Crossover children' [(Baidawi, 2020:280)](https://www.tandfonline.com/doi/full/10.1080/0312407X.2019.1686765) who are involved with both youth justice __and__ child projection, and are expect to have an __extra__ disbenefit in criminal careers from being in contact with both systems 4. Key insight two: the ways in which childhood experiences shape adult outcomes may __vary by both sex and race/ethnicity__ [(Broidy et al. 2015:145)](https://link.springer.com/article/10.1007/s40865-015-0008-z) --- # Summarising criminal careers <img src="https://raw.githubusercontent.com/benmatthewsed/qstep-nov-2020-exeter/main/figures/nagin_land_quote.png" width="80%" style="display: block; margin: auto;" /> _Nagin and Land (1993:346) Age, Criminal Careers, and Population Heterogeneity: Specification and Estimation of a Nonparametric, Mixed Poisson Model, Criminology, Vol. 31, No. 3, pp. 327:362_ --- # Latent Class Growth Curves - Latent Class Growth Curve models (LCGM, [Nagin and Land 1993](https://onlinelibrary.wiley.com/doi/abs/10.1111/j.1745-9125.1993.tb01133.x) __make groups out of data__ based on a repeated measures dependent variable - The data are split up into distinct __groups__ which each have their __own trajectories__ (in our case, probability of conviction in a given year) - LCGMs are very popular (if divisive) in criminology [(Sampson and Laub 2005)](https://scholar.harvard.edu/files/sampson/files/2005_crim_laub.pdf) - Can give you a __categorical output__ (i.e. trajectory membership) for each person in the data that you can then analyse as outcome in a regression model - You also get the probability that they belong to each trajectory --- # Latent Class Growth Curves <img src="https://raw.githubusercontent.com/benmatthewsed/qstep-nov-2020-exeter/main/figures/dunger_figure.png" width="70%" style="display: block; margin: auto;" /> _D’Unger et al. (1998), How Many Latent Classes of Delinquent/Criminal Careers? Results from Mixed Poisson Regression Analyses, American Journal of Sociology, Vol. 103, No. 6, pp. 1593-1630_ --- # Intersectionality and criminal careers <img src="https://raw.githubusercontent.com/benmatthewsed/qstep-nov-2020-exeter/main/figures/broidy_quote.png" width="78%" style="display: block; margin: auto;" /> --- # Research questions - How do the __cumulative__ effects of youth justice and child welfare contact on adult convictions trajectories __vary by sex and race/ethnicity__? - Is there are a specific __additional association__ with adult convictions trajectory of having contact with youth justice __and__ child welfare? --- class: center, middle, inverse # Methods: Quantiative intersectionality --- # What is intersectionality? - The idea has a complex history [(McCall 2005)](https://www.jstor.org/stable/10.1086/426800?seq=1#metadata_info_tab_contents), but as a term was popularized by [Kimberlé Crenshaw (1989)](https://chicagounbound.uchicago.edu/cgi/viewcontent.cgi?article=1052&context=uclf) - The [Wikipedia](https://en.wikipedia.org/wiki/Intersectionality) definition is: "a theoretical framework for understanding how __aspects of a person's social and political identities... combine__ to create different modes of discrimination and privilege. Intersectionality identifies advantages and disadvantages that are felt by people due to a combination of factors." --- # What is intersectionality? - Analytically, "intersectionality" covers researchers in different theoretical traditions and with different research methods - [Lesley McCall (2005)](https://www.jstor.org/stable/10.1086/426800?seq=1#metadata_info_tab_contents) talks about __three broad kinds__ of intersectional "complexity": - Anticategorical - __deconstructs__ analytical categories - Intracategorical - focuses on particular social groups at __neglected points of intersection__ to reveal the complexity of lived experience - Intercategorical - provisionally adopt existing analytical categories to __document relationships of inequality__ among social groups and changing configurations of inequality - Each of these archetypal approaches has its own typical set of methods. __Quantitative__ studies of intersectionality are typically doing __intercategorical__ research --- # Quantiative intersectionality - (NB this is only a partial introduction - my route in was through epidemiology, though McCall herself is a sociologist and political scientist. Sorry if I miss anything important!) - So far as I can tell, quantitative intersectionality __two main methodological components__: 1. Fits (often quite complicated) models with __interactions__ between the effects of different independent variables (so, social characteristics) and the outcome 2. Wants to avoid essentializing social groups by accounting for __variation__ in outcomes rather than just average outcomes __for people with the same observable characteristics__ - Juan Merlo calls this combination ["Multilevel Analysis of Individual Heterogeneity and Discriminatory Accuracy" (MAIHDA)](https://www.sciencedirect.com/science/article/pii/S0277953617307566?via%3Dihub) --- class: center, middle, inverse, my-one-page-font # Part One: Intersectionality and interactions --- # Problems with interactions - In the MAIHDA approach we want to fit interactions between __all__ our independent variables - Why do we need a fancy multilevel model for this? - Lots of interactions means lots of significance tests, and so problems with __multiple testing__ - we'd [get some statistically significant results by chance](https://psycnet.apa.org/fulltext/2019-30571-005.html) - Also, adding too many interaction effects in a standard regression model creates problems because of the __'curse of dimensionality'__ - Every extra (categorical) variable in the interaction __splits the sample up further and further__, making estimates unreliable as interactions for many combinations of variables are based on increasingly small amounts of data --- # Curse of dimensionality: worked example ```r library(tidyverse) set.seed( nchar("curse of dimensionality") ^ 4 ) n <- 80000 ``` --- # Curse of dimensionality: worked example ```r df <- tibble( gender = sample(c("female", "male"), # we don't have data for nonbinary size = n, replace = TRUE, prob = c(0.49, 0.49)), Indigenous_status = sample(c("IS", "non-IS"), size = n, replace = TRUE, prob = c(0.04, 0.96)), child_welfare = sample(c("CW contact", "no CW contact"), size = n, replace = TRUE, prob = c(0.5, 0.80)), youth_justice = sample(c("YJ contact", "no YJ contact"), size = n, replace = TRUE, prob = c(0.10, 0.80)), convicted = sample(c("1", "0"), size = n, replace = TRUE, prob = c(0.2, 0.8)), ) ``` --- # Curse of dimensionality: worked example <img src="https://raw.githubusercontent.com/benmatthewsed/qstep-nov-2020-exeter/main/figures/base_plot.png" width="78%" style="display: block; margin: auto;" /> --- # Curse of dimensionality <img src="https://raw.githubusercontent.com/benmatthewsed/qstep-nov-2020-exeter/main/figures/yj_plot.png" width="78%" style="display: block; margin: auto;" /> --- # Curse of dimensionality <img src="https://raw.githubusercontent.com/benmatthewsed/qstep-nov-2020-exeter/main/figures/both_plot.png" width="78%" style="display: block; margin: auto;" /> --- # Interactions, bias and variance - So how to resolve this? - Conceptually, we want the model to find interactions __if there's enough data to support them__, but if there isn't enough data for a given combination of variables we want the model to be __sceptical__ about estimating a huge interaction effects and move its estimate towards zero - In the stats lingo, we're trading off [bias for variance](https://en.wikipedia.org/wiki/Bias%E2%80%93variance_tradeoff) - We want to make our model actually fit the data in our dataset __worse__ (i.e. add bias) by making it __less sensitive to extreme values__ of these interaction effects when there isn't a lot of data to support them --- # Scepticism through shrinkage - __Multilevel models__ are one way to let us be a bit __sceptical__ of extremely large effects for parameters when there isn't a lot of data - They do this by __'pooling' information__/'borrowing strength'/'shrinking' estimates across clusters - Often these clusters are schools or hospitals or courts, in this case our clusters are combinations of independent variables (i.e. our interactions) - This is because the multilevel model assumes that clusters are __drawn from a Normal distribution__, making it sceptical of extreme values - Importantly for us, the model figures out how much information should be pooled __based on the data itself__: as [Michael Clark says](https://m-clark.github.io/posts/2019-05-14-shrinkage-in-mixed-models): "Data nuances will determine the relative amount of ‘strength borrowed’, but in general, [multilevel] models provide a good way for the __data to speak for itself when it should, and reflect an ‘average’ when there is little information__." - See also the [Bell and colleagues paper (2019)](https://psycnet.apa.org/fulltext/2019-30571-005.html) on this point --- # What shrinkage looks like .left-column[ From [Michael Clark's excellent blog post](https://m-clark.github.io/posts/2019-05-14-shrinkage-in-mixed-models) on shrinkage: estimates from individual clusters get 'shrunk' towards the global mean ] .right-column[ <img src="https://m-clark.github.io/img/shrinkage/baseline.gif" width="80%" /> ] --- # What shrinkage looks like .left-column[ Here the data are __unbalanced__, so we have more people in some groups than others; estimates from the smaller groups get shrunk more towards the average effect. ] .right-column[ <img src="https://m-clark.github.io/img/shrinkage/unbalanced.gif" width="80%" /> ] --- # Other approaches - You don't have to use a multilevel model though! There are also __other methods__ of doing this regularization: - Penalized regression (see discussion between [Lizotte and colleagues (2020)](https://www.sciencedirect.com/science/article/pii/S0277953619304939) and [Evans et al. (2020)](https://www.sciencedirect.com/science/article/pii/S0277953619304927#bib14)) - ['Sceptical priors' in a Bayesian model](https://www.fharrell.com/post/journey/) - The important thing I think is the __conceptual__ framing: we think the social world is complex so we want to fit a complicated statistical model, but we also want a mechanism to stop overfitting to small strata - The multilevel model __balances__ these two concerns, [although it's not magic (Bell et al. 2019)](https://psycnet.apa.org/fulltext/2019-30571-005.html) --- class: center, middle, inverse # _Intermezzo_ --- # Fit these models with Bayes - Okay we're really getting technical here but I __think__ there's also a difference here in how you'd understand what a multilevel model _is_ between Frequentist and Bayesian senses - the TLDR is that a Frequentist approach treats the 'levels' in a multilevel model as part of the model's error term (and so ["integrates them out of the likelihood"](https://discourse.mc-stan.org/t/hierarchical-linear-models-bayes-vs-frequentist/12012/2)) - and the error term is by definition what you're __not__ modelling. - Whereas for Bayes these extra levels are (hyper-)parameters that you estimate directly - The upshot is that I think you only get __uncertainty estimates__ for all your interaction terms in the multilevel specification with __Bayes__, and so this is the recommended approach - I fit these models in `R` using the `{brms}` package. There's a very involved __worked example__ of the approach I took [here](https://gist.github.com/benmatthewsed/a8136944dc31fcc26d2f2e433edaab6d). Others who have adopted this approach have used MLWiN which you can call through Stata --- class: center, middle, inverse # Part Two: Variation in the outcome --- # Variation in outcomes - What about the describing variation in the outcome? - We want to __avoid the ["Tyranny of the averages"](https://www.sciencedirect.com/science/article/pii/S0277953617307566?via%3Dihub#sec5)__: assuming that everyone with the same characteristics has the same relationship between a 'risk factor' and the outcome we're interested in - For __non-linear models__ we also have the general problem that any parameter estimate is "extraordinarily hard to interpret" in isolation because the effects of every variable are _already_ conditional on the values of other variables [(McElreath, 2016)](https://github.com/rmcelreath/statrethinking_winter2019) --- # Solution: prediction for inference - Use Average Predictive Comparisons [(Gelman and Pardoe 2007)](http://www.stat.columbia.edu/~gelman/research/published/ape17.pdf) Specifically, in our case we used __Marginal Effects at Representative values__ (MERs) - Decide what combinations of independent variables we're interested in (i.e., pick our representative values), use the model to __estimate outcomes for people with that combination of independent variables__ [(see also King et al. 2000)](https://gking.harvard.edu/files/abs/making-abs.shtml) - These predictions __incorporate the effects of all our interactions__ - that is, the levels of our 'multilevel' model - as well as the main effects of these variables - For categorical models we get a __distribution of outcomes__ across the categories of our outcome - just what we want! - We can still look at parameter estimates for interaction terms to see if specific interactions are important... but see the warning on the previous slide! --- # Problem: interactions - We still have another, sutble, problem in understanding cumulative effects though! - Our MERs are __collapsible__: the amount your probability changes from a change in the presence of an independent variable depends on your probability in the absence of the independent variable - So the using predictive MERs incorporates both the interactions from our intersectional random effect, but __also__ [__mathematical__, not subject-matter-relevant, interactions (Harrell, 2020)](https://www.fharrell.com/post/marg/) as a consequence of converting parameter estimates from the __log-odds to the probability scale__ - And so on the probability scale we could/will find __cumulative__ effects of having multiple childhood system contacts because of these __mathematical__ interactions, even if there is no __additional__ effect of having both youth justice and child welfare contact --- # Problem: interactions ```r library(tidyverse) # say we have four independent variables which each # increase the log-odds of the outcome by 1, but # our two groups have very different baseline log-odds (i.e. intercepts) vals <- seq(-4, by = 1, length.out = 5) vals2 <- seq(-1, by = 1, length.out = 5) # turn the log-odds into probabilities df <- tibble( n_risk_factors = list(c("0", "1", "2", "3", "4")), logit_scale = list(vals, vals2), condition = list("low baseline", "high baseline") ) %>% unnest() %>% mutate(prob_scale = boot::inv.logit(logit_scale)) ``` --- # Problem: interactions <img src="https://raw.githubusercontent.com/benmatthewsed/qstep-nov-2020-exeter/main/figures/logit_plot.png" width="78%" style="display: block; margin: auto;" /> --- # (Possible) solutions: compare predictions across models - To account for this cumulative effect of transforming our results into probabilities, we can __compare__ the estimated probabilities from our intersectional model to a model which __has all the fixed effects but doesn't have our intersectional random effect__ - This comparison gives us an estimate __additional__ difference in the effect of our intersectional random effect compared to a simpler model without the random effects --- name: review class: center, middle, inverse # A quick review --- # In review - We want to see how the __cumulative effects__ of contact with child welfare and youth justice on adult convictions trajectories __vary by sex and Indigenous status__ - And to see if there are particular extra effects of having contact with both child welfare and youth justice - We want to fit a model with __lots of interactions__ and which helps us understand __variation in our outcome__ between people with the same observables - We can do this by fitting our interactions as a level in a __multilevel model__ and by interpreting the model through __Marginal Effects at Representative values__ to get a distribution of outcomes for people with a given set of characteristics --- name: our-study class: center, middle, inverse # Our study --- # Data - Comes from __Queensland Cross-sector Research Collaboration__ (QCRC) - __Linked data sources__ from different administrative departments in Queensland, Australia: - Youth justice - Adult justice - Child protection - Birth register - Marriages - Cohorts registered born in Queensland in 1983 and 1984, followed to age 29 --- class: my-one-page-font # Measures <table> <thead> <tr> <th style="text-align:left;"> Variable </th> <th style="text-align:left;"> Level </th> <th style="text-align:right;"> Count </th> <th style="text-align:right;"> Proportion </th> </tr> </thead> <tbody> <tr> <td style="text-align:left;"> Adult conviction </td> <td style="text-align:left;"> Convicted </td> <td style="text-align:right;"> 16802 </td> <td style="text-align:right;"> 20.20 </td> </tr> <tr> <td style="text-align:left;"> </td> <td style="text-align:left;"> Not convicted </td> <td style="text-align:right;"> 66569 </td> <td style="text-align:right;"> 79.80 </td> </tr> <tr> <td style="text-align:left;"> Sex </td> <td style="text-align:left;"> Female </td> <td style="text-align:right;"> 40416 </td> <td style="text-align:right;"> 0.48 </td> </tr> <tr> <td style="text-align:left;"> </td> <td style="text-align:left;"> Male </td> <td style="text-align:right;"> 42946 </td> <td style="text-align:right;"> 0.52 </td> </tr> <tr> <td style="text-align:left;"> Indigenous Status </td> <td style="text-align:left;"> Indigenous </td> <td style="text-align:right;"> 4821 </td> <td style="text-align:right;"> 0.06 </td> </tr> <tr> <td style="text-align:left;"> </td> <td style="text-align:left;"> Non-indigenous </td> <td style="text-align:right;"> 78541 </td> <td style="text-align:right;"> 0.94 </td> </tr> <tr> <td style="text-align:left;"> Child Welfare/Youth Justice </td> <td style="text-align:left;"> 0.0 </td> <td style="text-align:right;"> 71232 </td> <td style="text-align:right;"> 0.85 </td> </tr> <tr> <td style="text-align:left;"> </td> <td style="text-align:left;"> 1.0 </td> <td style="text-align:right;"> 2048 </td> <td style="text-align:right;"> 0.02 </td> </tr> <tr> <td style="text-align:left;"> </td> <td style="text-align:left;"> 0.1 </td> <td style="text-align:right;"> 8528 </td> <td style="text-align:right;"> 0.10 </td> </tr> <tr> <td style="text-align:left;"> </td> <td style="text-align:left;"> 1.1 </td> <td style="text-align:right;"> 1554 </td> <td style="text-align:right;"> 0.02 </td> </tr> <tr> <td style="text-align:left;"> Total </td> <td style="text-align:left;"> </td> <td style="text-align:right;"> 83371 </td> <td style="text-align:right;"> 1.00 </td> </tr> </tbody> </table> --- # Methods Three steps: 1. Make __convictions trajectory groups__ with Latent Class Growth Curves [(Nagin and Land 1993)](https://onlinelibrary.wiley.com/doi/abs/10.1111/j.1745-9125.1993.tb01133.x) 2. Fit intersectional [(Bell et al. 2019)](https://psycnet.apa.org/fulltext/2019-30571-005.html) multinomial logistic regression model to __estimate associations__ between youth justice and child welfare __system contact and adult convictions class__ 3. Interpret model results through __estimated probability of convictions class membership__, conditional different configurations of our independent variables --- name: results class: center, middle, inverse # Results --- # Estimated conviction trajectories <img src="https://raw.githubusercontent.com/benmatthewsed/esc-2020/master/docs/figures/y18_trajectories.png" width="78%" style="display: block; margin: auto;" /> --- # Regression model fit <table> <thead> <tr> <th style="text-align:left;"> model_names </th> <th style="text-align:left;"> model_formulas </th> <th style="text-align:left;"> elpd_diff </th> <th style="text-align:left;"> se_diff </th> </tr> </thead> <tbody> <tr> <td style="text-align:left;"> Intersectional </td> <td style="text-align:left;"> category ~ sex * IS + any_cw + any_yj + (1 | stratum) </td> <td style="text-align:left;"> 0.0 </td> <td style="text-align:left;"> 0.0 </td> </tr> <tr> <td style="text-align:left;"> Intersectional (extra interaction) </td> <td style="text-align:left;"> category ~ sex * IS + any_cw * any_yj + (1 | stratum) </td> <td style="text-align:left;"> -1.5 </td> <td style="text-align:left;"> 1.2 </td> </tr> <tr> <td style="text-align:left;"> Fixed effects (interaction) </td> <td style="text-align:left;"> category ~ sex * IS + any_cw * any_yj </td> <td style="text-align:left;"> -56.6 </td> <td style="text-align:left;"> 11.1 </td> </tr> <tr> <td style="text-align:left;"> Fixed effectes </td> <td style="text-align:left;"> category ~ sex * IS + any_cw + any_yj </td> <td style="text-align:left;"> -58.1 </td> <td style="text-align:left;"> 11.4 </td> </tr> <tr> <td style="text-align:left;"> Sex and IS only </td> <td style="text-align:left;"> category ~ sex * IS </td> <td style="text-align:left;"> -3211.2 </td> <td style="text-align:left;"> 88.9 </td> </tr> <tr> <td style="text-align:left;"> Intercept only </td> <td style="text-align:left;"> category ~ 1 </td> <td style="text-align:left;"> -7706.0 </td> <td style="text-align:left;"> 130.0 </td> </tr> </tbody> </table> --- # Parameter estimates: fixed effects <img src="https://raw.githubusercontent.com/benmatthewsed/qstep-nov-2020-exeter/main/figures/20201109-full_model_fixed_params.png" style="display: block; margin: auto;" /> --- # Interactions parameters <img src="https://raw.githubusercontent.com/benmatthewsed/qstep-nov-2020-exeter/main/docs/figures/20201110-interaction_plot.png" width="78%" style="display: block; margin: auto;" /> --- # Conviction trajectory probabilities <img src="https://raw.githubusercontent.com/benmatthewsed/qstep-nov-2020-exeter/main/docs/figures/20201109-step_preds_plot.png" width="78%" style="display: block; margin: auto;" /> --- # Difference in estimated trajectories <img src="https://raw.githubusercontent.com/benmatthewsed/qstep-nov-2020-exeter/main/docs/figures/20201110-difference_plot.png" width="78%" style="display: block; margin: auto;" /> --- name: discussion class: center, middle, inverse # Discussion --- # Discussion - Childhood experiences' relationships with the probability of adult convictions trajectories __do seem to differ by sex and Indigenous status__: - Non-Indigenous Status men and women have the largest decrease in probability of no/low class membership in presence of all childhood contacts - In contrast Indigenous Status men have the smallest decrease in probability of no/low trajectory membership in presence of all childhood contacts - … but the __cumulative effect__ of these childhood system contacts leads to __reasonably similar outcomes by sex`*`IS__, despite the very __different baselines__ by sex and IS --- # Discussion - There was an __additional cross-over effect__ for Non-Indigenous Status women in the Low/Declining trajectory, but not much for other trajectories or groups - Possible explanations: - Non-Indigenous Status women in contact with both systems have (on average) worse circumstances for things we don’t measure? - Particularly strong effect of labelling by both systems for Non-Indigenous Status women? - Mathematically, we see these bigger effects for Non-Indigenous Status women because their intercept in the model (when they have no contact with any system) is very, very low, in the part of the distribution where converting log-odds to probabilities has an outsized effect? - We see a __lot of variation__ in estimated trajectories for people with the same observable characteristics. This reflects both things we haven’t included in the model, but also people’s differential responses to these childhood experiences --- # Concluding thoughts - This particular way of quantitatively modelling 'intersectionality' with multilevel models seems pretty good! - I like that it's __exploratory__ - but you have to bear in mind that no 'significant' effect doesn't mean that no effect - it may just be that estimates were very aggressively shrunk towards zero ("Absence of evidence is not evidence of absence" and all that) - If we had a more informed theoretical background perhaps we would have used a [more bespoke model](https://m-clark.github.io/posts/2019-05-14-shrinkage-in-mixed-models) to look for interactions and do this shrinkage? - In the __multinomial__ case it's a bit of a pain though! --- # Concluding thoughts - What does IS measure here? Must __avoid deficit interpretation__ [(Walter and Andersen 2013)](https://www.taylorfrancis.com/books/e/9781315426570) - Same with sex/gender! - Even though I've talked about it less today I think emphasising the variation in outcomes for people with the same observables has had a more profound impact on how I think about quantitative modelling (see also [this](http://nautil.us/issue/92/frontiers/how-eugenics-shaped-statistics)) - If you take anything from this talk please go and read [McCall (2005)](https://www.jstor.org/stable/10.1086/426800?seq=1#metadata_info_tab_contents) --- name: thanks class: center, middle, inverse, my-one-page-font # Thank you! Thank you to Heidi Perrett at Griffith University for facilitating data access Ben Matthews¹, Susan McVie¹, Carleen Thompson² and Anna Stewart² Q-Step Exeter, 2020-11-11 ¹University of Edinburgh ²Griffith University .pull-left[ <!-- --> ] .pull-right[ <img src="https://esrc.ukri.org/esrc/includes/themes/MasterSite/images/logo-1.png" width="60%" /> ] `@benmatthewsed` | `@U_Inequalities`