AI without representation is just inequity at scale: on the exportation of unrepresentative artificial intelligence models to the Global South

A common assumption in discussions of artificial intelligence (AI) is that increasing demographic representation in training data is sufficient to mitigate bias. Under this view, failures in facial recognition, clinical classification, and language generation are treated primarily as problems of coverage that can be resolved through more inclusive sampling. While intuitively appealing, this assumption becomes problematic when models trained and validated within one sociotechnical context are deployed into others whose realities were never meaningfully represented during development. This limitation is particularly evident in health-related AI, where outcomes are shaped not only by biological variables but also by lifestyle and environmental factors that interact dynamically over time, challenging purely data-centric notions of representation (Tan et al., 2025).Emerging evidence suggests that the issue extends beyond coverage alone. Large-scale audits of biometric and healthcare datasets reveal widespread deficiencies in fairness, privacy, and regulatory compliance, underscoring structural weaknesses in how datasets are constructed and evaluated (Mittal et al., 2024). Even when models are optimized to satisfy formal definitions of algorithmic fairness, their real-world effects may remain inequitable. Simulation-based studies of AI-assisted healthcare systems demonstrate that achieving parity in model performance across groups does not necessarily translate into equitable outcomes, particularly when broader sociotechnical dynamics are taken into account (Stanley et al., 2025). More fundamentally, reducing fairness to measurable performance across predefined groups risks oversimplifying the structural and intersecting determinants that shape real-world outcomes, including the compounded effects of identity, environment, and access to care (Tan and Benos, 2025).These limitations are not confined to data alone. Modern machine learning systems frequently rely on spurious correlations that enable high performance within specific datasets but fail under distributional shifts, a phenomenon widely described as the "Clever Hans" effect (Kauffmann et al., 2025). At a deeper level, model representations themselves often diverge from human conceptual organization, lacking the hierarchical and multi-scale structure that supports robust generalization (Muttenthaler et al., 2025). In parallel, studies of large language models across different sociocultural contexts continue to document persistent biases shaped by underlying data distributions and societal structures (Jiang et al., 2025), while work on dataset construction highlights enduring challenges in achieving globally representative, ethically sourced data (Xiang et al., 2025). These biases are further amplified when models treat populations as discrete or homogeneous groups, ignoring the intersectional nature of identity and the contextual factors that mediate how data is generated, interpreted, and applied (Tan and Benos, 2025).Taken together, these findings suggest that representation in AI cannot be reduced to demographic inclusion alone. Instead, it must be understood across multiple interdependent dimensions: data (what is collected and how), context (the systems in which models are deployed, including environmental and lifestyle conditions), epistemology (how models internally structure knowledge), and infrastructure (the regulatory and operational frameworks governing their use). Failures along any one of these dimensions introduce risk; failures across all four, as often occurs when models developed in the Global North are exported to the Global South, can transform technical limitations into mechanisms of inequity at scale.As used here, the Global South refers less to a fixed geographic region than to positions of relative disadvantage in the global distribution of technological authority, resources, and control. It is not interchangeable with low-and middle-income countries, resource-constrained settings, or settings of limited regulatory capacity, since these categories may overlap yet often diverge, and it is not internally uniform, since countries and institutions may differ in technical capacity, regulatory maturity, data infrastructure, and ties to multinational technology providers. The concern is therefore not that a model moves between regions, but that it may cross between sociotechnical environments of unequal authority.This argument builds on a body of critical scholarship on AI and the Global South. Work on decolonial AI has shown how the design and deployment of systems may reproduce historical patterns of power when authority and value remain concentrated in high-income settings (Mohamed et al., 2020). Studies of algorithmic fairness in the Global South have further suggested that adjusting model outputs alone can act as window dressing when the technical, social, and physical distance between developers and affected communities is left intact, as analyses of deployments in India indicate (Sambasivan et al., 2021).Recent work reinforces this concern across regions and domains, with assessments of AI governance in Sub-Saharan Africa reporting limited local control and calling for data sovereignty (Ayana et al., 2024), studies of health AI treating colonial history as a sociostructural determinant that shapes fairness before a model is evaluated (Asiedu et al., 2024), and analyses of the political economy of AI locating inequity in ownership, infrastructure dependency, and labor rather than in model performance alone (Regilme, 2024).These perspectives tend to address data, fairness, participation, and governance through governance and process frameworks. The contribution of this article is to connect these stages within a single cumulative account, in which representation may fail at any point in the AI lifecycle and in which such failures can compound rather than remain isolated. This article argues that addressing inequity in AI requires an expanded notion of representation that integrates these dimensions. The sections that follow examine each in turn and contend that only a coordinated, systems-level response can prevent AI from reproducing, and amplifying, existing global asymmetries. Addressing these asymmetries therefore requires not only technical refinement but also transdisciplinary engagement across engineering, social science, and public health to ensure that AI systems reflect the complexity of the societies they are deployed in (Tan and Benos, 2025). The lifecycle framing of representation in this article, from data construction to evaluation, deployment feedback, generalization, and governance, is summarized in Fig. 1.Structural bias in AI systems often originates before model development, in the construction of datasets themselves. Representation at the data level is not limited to who is included, but extends to how data are captured, with what instruments, under which protocols, and what is excluded during preprocessing. These upstream decisions shape the statistical structure of the data and, consequently, the behavior of downstream models. Large-scale audits of biometric and healthcare datasets demonstrate that such design choices systematically affect fairness, privacy, and regulatory compliance, with widespread deficiencies observed across commonly used benchmarks (Mittal et al., 2024).One important source of structural bias lies in the technical parameters of data acquisition. In medical imaging, AI models trained on chest radiographs have been shown to predict patient race with high accuracy, not through clinically meaningful features, but through correlations with acquisition and processing characteristics embedded in the images. These include factors such as exposure, contrast, and positioning, which reflect underlying disparities in clinical practice and data collection rather than biological signal (Lotter, 2024).More broadly, machine learning systems are known to exploit low-level statistical regularities and shortcut features, reinforcing patterns that are incidental to the task but predictive within a given dataset. A similar phenomenon is observed in facial image analysis, where models can infer sensitive or unrelated personal attributes from non-demographic signals embedded in images, including camera artifacts, presentation choices, and contextual features. These findings highlight that datasets often contain far more information than intended, raising concerns about unintended information leakage and the difficulty of isolating meaningful signal from confounding factors (Tkachenko and Jedidi, 2023).These acquisition-level biases are compounded by geographic and structural asymmetries in dataset curation. Dominant benchmarks in computer vision continue to exhibit limited global representation, with many datasets lacking diversity, relying on non-consensual data collection practices, and failing to capture environmental and contextual variability across populations (Xiang et al., 2025). Even when efforts are made to construct more balanced datasets, achieving meaningful representation requires deliberate, resource-intensive design, underscoring that diversity does not emerge automatically from scale.In natural language processing, comparable patterns arise from the composition of large-scale pretraining corpora. These datasets are disproportionately derived from digitally active populations, often reflecting the linguistic patterns, cultural norms, and perspectives of urban, educated, and highly connected users.As a result, entire communities may be systematically underrepresented, leading to persistent biases and stereotypical outputs that mirror existing societal structures (Jiang et al., 2025). These gaps are not confined to digital datasets. Even within clinical data collected in high-resource settings, substantial heterogeneity exists across institutions, with differences in patient populations, disease prevalence, and data collection practices producing non-identically distributed datasets. Such heterogeneity poses challenges for standard machine learning which often and comparable data distributions across training and deployment environments et al., a broader limitations in physical and digital infrastructure what data can be collected in the In many regions, particularly across existing datasets on and of remain to capture local reflecting a of data for and 2025). In such settings, entire populations may remain within downstream effects of these asymmetries are medical for have been shown to different for clinical on geographic how data and deployment environments interact to outcomes et al., 2024).These structural deficiencies are to be resolved through data alone. the work that of into training data can to a process in which models information about the underlying particularly at et al., 2024). populations are in these distributional they are disproportionately to in training under of dataset a further the used to model performance may fail to capture the disparities that This limitation the from data representation to as the in machine learning often model performance into a single the that a for populations while systematically failing In medical imaging, this is particularly have been shown to and patient populations, with in intersectional by attributes such as and et al., Such findings a of a model can clinically at the level while producing patterns of that care for the groups at of medical can therefore as a of statistical clinically meaningful disparities that such as are for across but they by reveal a clinically point equitable or across This clinical decisions are not made across all they are made at specific that who or These disparities may be for intersectional where dimensions such as and can compound and less et al., and Benos, et al., broadly, formal fairness can also when treated as for about and et that of fairness definitions can that are from the they are to and may the groups they were to becomes deeper when performance is not by learning the clinical but by spurious correlations in the In models were to rely on confounding factors to image acquisition and dataset source rather than medical These systems yet fail under in systems or rely on that performance alone may not reveal et al., This is not an of datasets. More standard performance can be to a model has a or a shortcut that to work under the of the et al., not automatically this In medical imaging, models can demographic attributes such as and their and demographic has been with disparities in and that demographic can fairness and models that within the dataset. et that this local often to in where models for fairness may not remain of fairness during may be less a of the model than a of the This of is to clinical et that fairness in a source may not under distribution when are compound and in and clinical one a in performance gaps that were in the source in the Such findings are to the Global South, where models from high-resource settings may different disease prevalence, practices, patterns, infrastructure and the conceptual framing of may be to clinical fairness as statistical across groups is not clinically differences may reflect clinical rather than bias. et therefore that clinical AI fairness be rather than with sensitive fairness and by clinical context and rather than as technical together, these findings that the of AI can reproduce and the structural gaps in data mechanisms to and intersectional and and with clinical and social models may systematically fail the populations they are to while by standard This to a deeper structural the that deployed models to be are often as the structural biases in data construction and the failures described in the a common they in the systems that are an AI model is into a a or a public the mechanisms that to be and are to address of AI and while is on AI, is only limited work on what AI systems are deployed et al., 2024). A of frameworks for AI and that while of frameworks only and when or how AI be et al., 2024). This a structural in how machine learning systems are through to ignoring important and systems and and that are important for and over the remain and et al., The can remain a critical in computer vision for a in the model may not but when an image the et al., The the never deployed real-world and were not during language model and models not of difficulty in which the model does not or human can the et al., have been across including geographic and where models at and is the model when the these were in the et al., 2024). models more and their outputs more the they to This concern is particularly for large language models embedded in In such settings, the model is not a but of a sociotechnical that can what information is or and shape et this as an a deployed may such as and authority but the of or exported AI this the of may be amplified not only through but also through that outputs to or infrastructure therefore include and meaningful when systems are used in or algorithmic many biases are to their and are not many of or how they are and the systems public The of which systematically and to be how a can at before failures are when are yet systems as with for affected to or outcomes the at In in the has substantial which on the and whose are by or physical and which are compounded by and power in and The an assumption that does not where local infrastructure is limited and where a can a to at requires and yet this challenging to and social structures across developed and et al., 2025). In shape regulatory outcomes in that and bias in AI systems can in access to disproportionately populations et al., algorithmic the distributional data have been shown to be highly sensitive to model and for affected particularly when the for the the to address et al., the made by for and and are to than they to be that the of is compounded by a of the et al., dynamics are in In the an to of public and to for social and it has been for on the and the of mechanisms to an et al., 2024). A broader of public across the region that decisions often rely on for that and that the to a is far in the region than global fairness 2024). of these including on from countries, were with to local is also from of political economy and The of AI and infrastructure, and on technical and these where authority over a of the political economy of AI inequity in infrastructure dependency, and the of in high-income settings rather than in model performance alone (Regilme, 2024). of AI governance in Sub-Saharan Africa that limited local data infrastructure can institutions on with authority to or how systems are (Ayana et al., 2024). These a rather than an the model and the data during it has the to or the and who the of local and structural response is to design mechanisms into the deployment the human at the it becomes to for the of AI and the of information that may be human in et al., This model structures and a for and in settings where regulatory is has not been is in the Global South, where models from settings the contextual local infrastructure, or regulatory to when they are The of a is not a it systematically the populations and contexts represented in the environment, while populations and The is not generalization but a structural of existing a more in the These are not and the argument does not they biometric and under different different and different of In clinical AI, to be as biological or a in and systems the is an from a or a and the power between an and the such decisions to these is more than what a on systems and design in high-resource settings, a to treat social problems as technical with more data, and a common of representation in which the of the deployment context were never into the It is this rather than any single that the lifecycle account is to failures in data evaluation, and deployment a deeper structural AI systems are often validated under that are than the environments into which they are generalization as a technical to be in this is not a that a model has or it on the between the the deployment and the social or task the model is to In et of and that the is a machine learning is within the specific context in which it is applied et The more a model that in one may fail when exported into whose data, practices, infrastructure, and this is Modern machine learning can many that on data, yet under or deployment et demonstrate this across computer medical imaging, natural language processing, health and medical that models with similar performance can diverge in fairness, and et The is what to be a model may reflect the statistical of a rather than a of the an differences in model behavior may be by choices such as or training rather than by with deployment learning this training data contain acquisition models may these of the et across medical imaging, clinical and data that model performance can be models exploit data acquisition biases that not across This is not a of model It is a of how collected data the protocols, and practices of the that A model that these may during but when to a or practice that from the broader may these rather than not they et that many AI benchmarks and that benchmarks can of further may reflect to characteristics rather than that to data In natural language processing, et that high performance on does not robust generalization across domains, shifts, or of the et These findings for representation benchmarks whose data as evidence of are or from the of models can to while for the contexts where their failures are dynamics when models developed in high-income settings are deployed in low-and middle-income et the of a AI model to in and that models trained an context may when local infrastructure, data and clinical differ from of the work that is patient populations, healthcare local data and collection across In such is not by the model alone. It the assumption that a high-resource can in for settings with different clinical work on bias reinforces this et that models can performance across and settings, particularly when data are and that algorithmic bias can fairness while et This is but it also the fairness and not automatically with the must be evaluated and in the deployment distribution this et that in patient image and can affect the performance of medical AI systems the for of clinically distribution This for the Global South, where the technical and regulatory infrastructure for may be models are than local is the populations represented in the data may also the from model together, these patterns suggest that generalization is not primarily a but a structural It is in how are how benchmarks are whose contexts are treated as and deployment environments are given the authority and infrastructure to models on their such as deployment may in settings, but they also about groups may be excluded from systems models were not to in the et The deeper is not to models from populations for they but to AI that populations are represented in the data, evaluation, governance, and systems that a model is for This is the taken in the Fig. this from gaps to feedback, generalization, and inequity at argument developed across the sections on a single inequity in AI is not at any one of the but through the effects of all of data, by left by and to across contexts they never that are to and to inclusion alone cannot this contribution of this article is the argument that representation must be as infrastructure, not a representation but does contextual the social, and environmental of a deployment were during This becomes when AI systems cross developed in high-income settings about data infrastructure, capacity, disease prevalence, practices, and clinical these models are with does not is the that made regulatory the and the communities with the authority and to model behavior and The is a structural in which can to the developers who the and by the populations who is also a of and control. A model that the cannot or and whose data it does not has not been meaningfully represented in that of how balanced training data may representation as infrastructure therefore requires that ownership, and authority be data, evaluation, and this requires that each of the be to it beyond demographic composition to include acquisition practices, and of by and intersectional as a with in with affected communities rather than from be on the of infrastructure of performance over time, with for and These are not to AI development. are the under which of deployment can be made operational by for each of infrastructure in Fig. the evidence that be before a is for infrastructure of collection settings, acquisition and with an of the and context the data infrastructure intersectional and local at the used in rather than performance alone. infrastructure with and mechanisms through which affected can infrastructure local regulatory of the model and of data, and the authority to or the on this view, is not a of the model in but a of these are in the where it which representation as infrastructure to be rather than this representation as infrastructure also include and systems in deployment not be only at the model but also at the level of and mechanisms et al., mechanisms The of to algorithmic decisions that affect is not a it is one of the through which failures and at such that disproportionately affect populations remain to the systems to In contexts where broader regulatory infrastructure is mechanisms into deployment may one of the to Global South is not a in this public health and across low-and middle-income countries have developed in systems under of and high social This is not it is the of that AI have systematically and what is to models of across deployment design that Global South institutions as rather than are therefore ethically and the these institutions models continue to be validated that not the environments in which they are regulatory in the Global South is The to and, or AI systems requires infrastructure that cannot be to and cannot be and in local AI governance are therefore structural for equitable not to the and mechanisms for this article remain to the settings that understood as infrastructure, is a technical and a political It requires that the communities to be by AI failures be not only in training data, but also in the governance structures that how models are deployed, and failures this article has not arise at in the they and they one data collection is by that failures are by the of The of mechanisms generalization failures to in the contexts where they the The is not a of discrete technical problems but a structural summarized in Fig. these failures across the AI lifecycle and infrastructure at each is not distributed The communities represented in AI are also often the to the systems exported to that requires more than It requires that a model the to deployment through to the context it not through the of the data on which it representation is not exported into contexts it not to or it becomes inequity at