AIMSDistill Model Weights: distilled ModernBERT students and teacher logits

Read full text
Authors: Adriana-Eufrosina Bora, Richi Nayak, Duoyi Zhang, Md Abul Bashar, Kerrie Lee Mengersen
Cited by: 2

Abstract

Trained model weights and teacher logits accompanying AIMSDistill (Expert Systems with Applications 332:133691), which distils four specialised teacher models into a single lightweight ModernBERT student for cross-jurisdictional analysis of corporate modern slavery statements. A separate student is trained for each jurisdiction. FILES au.pth (1.47 GB) — Distilled student, Australian MSA. Macro-F1 0.7312 on the AIMS.au test set. Also serves as the Agent 1 evidence-extraction classifier in the AIMS-QA framework. uk.pth (1.47 GB) — Distilled student, UK. Macro-F1 0.7244 on the AIMS.uk test set. ca.pth (1.47 GB) — Distilled student, Canada. Macro-F1 0.7295 on the AIMS.ca test set. logits.zip (325 MB) — Output logits from the four specialised teacher models (targeted contrastive learning, domain-specific prompts, pseudo-labelling, context-enhanced learning), allowing the distillation step to be reproduced without retraining the teachers. RELATION TO EARLIER DEPOSIT The Australian student was first deposited by Duoyi Zhang as "student_model.pth", doi:10.6084/m9.figshare.29176703. The au.pth file in this record is byte-identical to it (MD5 73dba7a7918721a987629faea1175635). This record adds the UK and Canadian students and the teacher logits, so that all AIMSDistill artefacts are available together under one DOI. NOT INCLUDED The ModernBERT backbone is not distributed here. It is the unmodified public model answerdotai/ModernBERT-large, downloaded automatically by the transformers library. USE Sentence-level multi-label classification of modern slavery statements against the seven mandatory reporting criteria of the Australian Modern Slavery Act 2018, as also illustrated in AIMS.au (Training data: AIMS.au (doi:10.6084/m9.figshare.28489340)). Each student is trained for a specific jurisdiction and should be used on statements from that jurisdiction. Applying the Australian student to UK or Canadian statements will produce results that differ from those reported in the paper. These models are decision support for human reviewers, not automated compliance verdicts. Misclassifying a corporate disclosure carries legal and ethical consequences; the paper states that full automation is not recommended. CHECKSUMS au.pth SHA-256 a08ff4b5967688dbc7db18d439d5bcdd84f9068e0a8128bdcb3f50c8d3f7457dau.pth MD5 73dba7a7918721a987629faea1175635 CITE For the weights themselves, and for any use of the distillation method: Bora AE, Zhang D, Bashar MA, Nayak R, Mengersen K (2026). AIMSDistill: Distilling Knowledge from Specialised AI Teachers for Cross-Jurisdictional Compliance Analysis of Modern Slavery Statements. Expert Systems with Applications 332, 133691. doi:10.1016/j.eswa.2026.133691 If you are using au.pth as the Agent 1 classifier within the AIMS-QA question-answering framework, please also cite: Bora AE, Zhang D, Thinyane H, Bashar MA, Nayak R, Mengersen K (2026). AIMS-QA: A Framework for Scaling Beyond Compliance the Assessment of Corporate Modern Slavery Statements with AI-driven Question Answering. Data & Policy, Cambridge University Press. Code: https://github.com/mila-studios/ai4h_aims-au

Publication details

Journal / source
Figshare
Publication date
15 August 2026
Document type
Dataset