How to Purchase Labels? A Cost-Effective Approach Using Active Learning Markets

We introduce and analyze active learning markets as a way to purchase labels, in situations where analysts aim to acquire additional data to improve model fitting, or to better train models for predictive analytics applications. This comes in contrast to the many proposals that already exist to purchase features and examples. By originally formalizing the market clearing as an optimization problem, we integrate budget constraints and improvement thresholds into the label acquisition process. We focus on a single-buyer-multiple-seller setup and propose the use of two active learning strategies (variance based and query-by-committee based), paired with distinct pricing mechanisms. They are compared with benchmark baselines including random sampling and a greedy knapsack heuristic. The proposed strategies are validated on real-world data sets from two critical application domains: real estate pricing and energy forecasting. Results demonstrate the robustness of our approach, consistently achieving superior performance with fewer labels acquired compared with conventional methods. Our proposal comprises an easy-to-implement practical solution for optimizing data acquisition in resource