Machine learning approaches to dietary classification from dental microtexture in primates

Abstract

Abstract Dental microwear texture (DMT) analysis is a critical proxy for reconstructing the diets of extant and extinct mammals. While craniodental morphology reflects selective pressures across evolutionary timescales, microwear captures localized, short-term dietary signals over weeks to months. However, the high dimensionality of modern 3D surface microtexture data, often spanning disparate parameter sets (such as standards and scale-sensitive fractal analysis, ), complicates classification, particularly when working with limited paleontological datasets. To address this, we present a robust machine learning pipeline designed to automatically classify primate samples ( $$N = 99$$ ) across 6 dietary groups and 7 species. Our methodology leverages a nested leave-one-out cross-validation framework to evaluate multiple classifiers, including multinomial logistic regression (MLR) with various regularization approaches, Naive Bayes, and tree-based ensemble algorithms (e.g., Random forests, XGBoost). Our results demonstrate that Lasso-regularized MLR and Naive Bayes yield the highest predictive performance while enforcing strict feature selection to maintain interpretability. Crucially, models relying exclusively on parameters consistently outperformed those using , as variables better capture the microroughness localized mechanical abrasions generated by specific diets in our dataset. Furthermore, the integration of novel Fourier-based descriptors and isotropy variables significantly enhanced models’ discriminating power. By providing a mathematically rigorous framework to isolate precise ecological signals from noisy, high-dimensional data, this approach enables more accurate and reproducible dietary classifications. Ultimately, refining these dietary reconstructions is essential for resolving broader questions regarding niche partitioning, species evolution, and paleoecological dynamics.