CowBase — A library for automatic milking system related data handling and curation in Python

Technology adoption on dairy farms allows for data-driven research on production, fertility, health, and welfare. However, collecting and preparing data for analysis is often tedious and time-consuming. Furthermore, there are substantial differences between dairy technology suppliers’ data standards and data structures. This paper presents CowBase, an open-source Python package that effectively streamlines data extraction and pre-processing from dairy farm milking and sensor systems. Specifically, data extraction for sensors connected to DeLaval DelPro (Tumba, Sweden) and Lely T4C or Horizon (Maassluis The Netherlands) was implemented, including milk meters, activity sensors, and reproduction analysis. CowBase was tested and validated on data collected from multiple projects within the Livestock Technology research group at KU Leuven, Belgium. This resulted in a database of 71 dairy farms, with data going back to 2005. Thorough data exploration was conducted to better understand data curation requirements, develop a database structure for efficient data retrieval, and create a toolbox to support analysis and visualization. A lactation metadata correction was developed to correct for 0.92% of incorrect, farmer-entered calving dates and 0.61% parity numbers. A smoothing algorithm was developed to correct for 8.68% of DeLaval milk recordings that were affected by a current or previous incomplete milking event. A quantitative analysis of multivariate sensor data and a small heat-stress study were included to showcase the use for future research. The use of CowBase enables greater use of dairy farm milking and sensor data, supports data sharing, facilitates collaborations, and enables the reproduction of research results in the dairy sector. • CowBase streamlines dairy farm sensor data collection and pre-processing. • Correction for lactation metadata and incomplete milk session improves data quality. • Enhances data sharing, collaboration, and reproducibility in PLF research. • Currently supports data from DeLaval or Lely automatic milking systems.