A modified Fay-Herriot model with machine learning for small area estimation of per capita expenditure among the poor and agricultural households

¹ School of Data Science, Mathematics, and Informatics, IPB University, Jl. Meranti Wing, Dramaga Campus IPB, Bogor, West Java, 16680, Indonesia
² Badan Pusat Statistik (Statistics Indonesia), Jl. Dr. Sutomo No. 6–8, Central Jakarta 10710, Indonesia

^* Corresponding author: khairil@apps.ipb.ac.id

Abstract

Accurately measuring household welfare, particularly among poor and agricultural populations, is essential for effective and inclusive policy formulation. Per capita expenditure serves as a key welfare indicator, yet direct estimates at small-area levels are often unreliable due to limited sample sizes. Small Area Estimation (SAE) offers a cost-efficient alternative by leveraging auxiliary data, but standard models such as Fay-Herriot (FH) assume linearity and may perform poorly under nonlinear data structures. This study introduces a novel modification to the FH model by incorporating Random Forest (RF), a machine learning method, into the fixed effect estimation stage, following initial parameter estimation via the Expectation- Maximization (EM) algorithm. The resulting FH-RF model is designed to capture complex nonlinear patterns while maintaining compatibility with area-level auxiliary data. When applied to estimate per capita expenditure among agricultural households, the FH-RF model outperforms the FH-EM model in predictive accuracy. Results further reveal that agricultural households exhibit spending patterns more closely aligned with the general population than with poor households, indicating distinct welfare dynamics. The proposed model highlights the potential of machine learning to enhance SAE methodology and inform data-driven poverty and agricultural policy interventions.