Predictive Algorithms: Help or Hindrance?

By Khadijah Majid
CAIDE Intern, Faculty of Arts

From hiring algorithms to shopping experiences to generating exam results, predictive algorithms can increase efficiency and convenience in numerous aspects of our lives and are revolutionary in the way that they can pre-empt problems before they arise. However, this technology does not come without flaws, and there are numerous ethical concerns regarding its use, as such a powerful tool can have disastrous effects if not wielded carefully.

Image by Christina Morillo

One area in which the use of predictive algorithms can prove to be efficient yet simultaneously problematic is predictive policing. Whilst there are many potential benefits in using algorithms to predict crime, caution must be exercised in order to ensure the fair treatment of all individuals who are affected by such tools. Some claim that predictive policing merely “helps police departments rationally allocate scarce resources”. [1] However, others argue that using predictive algorithms has the potential to do more harm than good, claiming that algorithms can be racist. [2] This brings us to one of the key issues that arise when utilising predictive algorithms - the fact that these systems base their predictions on historically biased data. All data input into these algorithms is a product of historical human bias, which is nearly impossible to eliminate without sacrificing the accuracy of the predictions the algorithm will generate. Consequently, using previous policing patterns will inevitably generate a subjective set of results.

A significant example of algorithmic bias can be seen in the controversial use of the Correctional Offender Management Profiling for Alternative Sanctions, also known as COMPAS, a program used to determine the likelihood of reoffending for defendants in the US. The software uses a “set of scores derived from 137 questions that are either answered by defendants or pulled from criminal records”, and assigns them scores from one to ten, ten being the highest risk category.[3] Factors considered include age, sex, and criminal history; “notably, race is not used”. [4]

Northpointe, COMPAS’ creator, argues the predictions are fair “because scores mean essentially the same thing regardless of the defendant’s race”. [4] This is true to some extent if we consider the defendants who did reoffend, with sixty percent of white defendants and sixty-one percent of black defendants who scored a seven on the COMPAS scale reoffending. [4] However, among defendants who did not reoffend, “blacks were more than twice as likely as whites to be classified as medium or high risk”, meaning that those who did not commit a further offence were “subjected to harsher treatment” in court. [4] Despite the algorithm not explicitly using race as a determining factor, the traits that predict reoffending vary according to race; for example, “black defendants are more likely to have prior arrests, and since prior arrests predict reoffending, the algorithm flags more black defendants as high risk”. [4] Not only does this system disproportionally affect people of colour, but the predictions also do little to indicate whether an individual is actually a threat to society.

Image by RODNAE_Productions

Another issue arises when considering the use of algorithms in predictive policing; studies show that that predictive policing doesn’t reduce crime, but actually increases the number of arrests made. Following the discussion of how historical bias influences predictive algorithms, it is evident that even if algorithms function as they are supposed to, because the data they are supplied with is biased they will never generate a completely fair set of results. This is particularly apparent in predictive policing software like COMPAS or STMP (a predictive algorithm used in New South Wales). Lucia M. Sommerer states that there are two types of predictive policing: individual and geospatial policing. [5] Geospatial policing differs from individual policing in the way that an entire area rather than one individual is placed under surveillance based on historical crime rates. As a result, this ultimately leads to more arrests and convictions as more police are around to witness minor crimes that would usually go unpunished. Building on this theory, data scientist Cathy O’Neil claims that predictive policing doesn’t solve the crime problem, it just highlights the issue and in some cases exacerbates it. Most of these arrests occur in “impoverished black or Hispanic neighborhoods”, showing us that “geography is a highly effective proxy for race”. [6]

Clearly, the issue of historical and racial bias in predictive policing necessities an ethics of care approach. In his article on the ethics of predictive policing, Peter M. Asaro defines the differences between the Models of Threat and the Ethics of Care models. While the Models of Threat strategy treats individuals as “risks and threats” which must be “managed” and “eliminated”, the Ethics of Care model is “holistic”, and “does not expect more and better data to simply solve complex social and institutional problems”. [7] There are also various alternatives to predictive policing that also aim to reduce crime which should be considered when discussing predictive policing strategies. O’Neil mentions several alternative policing models such as “broken windows” policing, which focuses on cleaning up neighbourhoods to lessen the likelihood of serious crime, and “highly tolerant” policing, an initiative in New Jersey which proved to be extremely successful. [6] Other initiatives include installing schemes for youths or community groups to minimise the opportunities for youths to become involved with drugs or loiter on the streets. At the very least, focus should be placed on collecting “a current data set that is representative of the current cohort”; that is, gathering and adapting the data used by predictive algorithms based on current criminal activity rather than relying on historical patterns to make predictions as accurate as possible. [8]

References

[1] Mains, P., 2021, ‘Predictive Policing Can Reduce Racial Discrimination’ <https://canvas.lms.unimelb.edu.au/courses/129165/files/12502014?wrap=1>

[2] Benjamin, R., (2019), ‘Captivating Technology: Race, Carceral Technoscience, and Liberatory Imagination in Everyday Life’, Duke University Press <https://doi.org/10.2307/j.ctv11sn78h>

[3] Angwin, J, et al., 2016, ‘Machine Bias’, ProPublica, https://canvas.lms.unimelb.edu.au/courses/129165/files/12262792?wrap=1

[4] Corbett-Davies, Sam, et al., 2016, ‘A Computer Program Used for Bail and Sentencing Decisions Was Labelled Biased against Blacks. It’s Actually Not That Clear‘, The Washington Post <(https://canvas.lms.unimelb.edu.au/courses/129165/files/12262766?wrap=1)>

[5] Sommerer, L. M, 2017, ‘Geospatial Predictive Policing – Research Outlook & A Call For Legal Debate’, Neue Kriminalpolitik <http://www.jstor.org/stable/26315807>

[6] O'Neill, Cathy, 2017, Weapons of Maths Destruction: How Big Data Increases Inequality and Threatens Democracy', Penguin Press, <https://canvas.lms.unimelb.edu.au/courses/129165/files/12262768?wrap=1>

[7] Asaro, Peter M., 2019, ‘From Models of Threat to an Ethics of Care: AI Ethics in Predictive Policing’, IEEE Technology and <https://canvas.lms.unimelb.edu.au/courses/129165/files/12262844?wrap=1>

[8] Human Rights Commission Technical Paper, 2020, ‘Addressing Algorithmic Bias’ <https://canvas.lms.unimelb.edu.au/courses/129165/files/12262901?wrap=1>