A growing number of pretrained foundation models can provide auxiliary predictions
for downstream tasks in which labeled outcomes are scarce. These models, however,
differ in task-specific accuracy and query cost, so querying every model for every
input is rarely affordable. We develop a budget-constrained learning framework
that uses a small gold-standard sample to learn which models to query and how to
combine their outputs with freely available covariates. Within this framework, our
first algorithm queries a common affordable subset for all inputs, using a branch-andbound
search that finds the subset with the smallest empirical risk. Since the most
useful models may vary from one input to another, we further develop a second
algorithm that decides for each input which model to query next and when to stop,
using cross-fitted backward Q-learning. For the first algorithm, we establish oracle
inequalities and minimax-optimal rates for expected excess prediction risk when the
regression function admits an affordable representation. For the second algorithm,
we characterize when input-dependent acquisition strictly improves on the best
affordable fixed subset and establish finite-sample bounds on the prediction risk
and average cost of the learned rule. In simulations and a multimodal laptop price
application, both algorithms outperform benchmarks that correct the predictions of
one foundation model or combine those of two.