The dark side of AI: algorithmic bias and global inequality

Algorithmic bias has huge potential for diversity, inclusion and marginalisation in the workplace and beyond.

We talk about data ‘colonisation’ with regards to potential harm suffered by workers in the Global South who clean up data and develop algorithms for the benefit of those using algorithms in the Global North. Data is central to AI technologies, so the access and development of this data can lead to increased data inequality, and this is a big risk. 

We need to consider consequences in advance, not after the fact, because by acting in advance we may be able to influence the design and eventual outcomes of AI-related technologies.  

The risks of AI, especially at the frontier of development, require a new way of thinking

A relational view of risk is needed in envisioning the future of AI as an emerging technology, and in conceptualising risk as a duality that integrates both benefit and harm.   

An algorithm may perform really badly on a particular population or other group if the algorithm is not exposed to data from that population while the algorithm is being developed and people are trained on it – and data from marginalised patient groups is often underrepresented. So this leads to a vicious cycle in which algorithms don’t reflect healthcare data of certain groups, and those algorithms then deepen health inequalities affecting those communities. Synthetic data, although it has many problems of its own, has been used to counter algorithmic bias in some healthcare situations. 

While offshoring has existed in the data world for decades now, we see a particular new risk with regards to AI.

The term ‘data colonialism’ reflects a parallel with historical colonialism, but data has characteristics that go beyond the natural resources at the core of colonialism over many centuries. As we say in the editorial, data extraction by the Global North from the Global South “creates new human ‘data relations’ and a new social order; even social life” – and this poses the risk that the Global North can control data flows and tilt not only the data landscape but the entire digital economy. 

We have heard the argument that workers in the Global South benefit monetarily through AI-related work. We nevertheless suggest there is an imbalance because the short-term work (often in poorly paid and precarious work) can, over the long run, lead to greater inequality and further marginalisation. We conclude that academics and society at large need new ways of understanding and managing these risks. 

Policymakers who seek to create AI legislation need to account for its fast-moving fluidity alongside the digital maturity in individual countries and regions. These policies should also reflect the rights and ethics of those doing the data work and helping to develop these new algorithms and AI technologies, particularly those in the Global South who risk further marginalisation. 

Our editorial has 3 broad conclusions:

• first, policymakers must define the key risks from AI while simultaneously demystifying less severe risks within a broader context that may include benefits as well
• second, policy needs to respond to the evolving situation and resist becoming static
• third we need to revise current policies on data protection while at the same time holding AI technologies to account within the existing framework

We advise to be on guard that people and organisations not rely on their own professional judgment as they lean on AI to make judgments and set core values.  The paper also highlights the importance of multidisciplinary collaboration between important stakeholders, including innovators (academia or industry), policymakers, institutional organisations collecting the data and, importantly, the public.