The expressive capacity of physical systems employed for learning is limited by the unavoidable presence of noise in their extracted outputs. Though present in physical systems across both the classical and quantum regimes, the precise impact of noise on learning remains poorly understood. Focusing on supervised learning, we present a mathematical framework for evaluating the resolvable expressive capacity (REC) of general physical systems under finite sampling noise and provide a methodology for extracting its extrema, the eigentasks. Eigentasks are a native set of functions that a given physical system can approximate with minimal error. We show that the REC of a quantum system is limited by the fundamental theory of quantum measurement and obtain a tight upper bound for the REC of any finitely sampled physical system. We then provide empirical evidence that extracting low-noise eigentasks can lead to improved performance for machine learning tasks such as classification, displaying robustness to overfitting. We present analyses suggesting that correlations in the measured quantum system enhance learning capacity by reducing noise in eigentasks. The applicability of these results in practice is demonstrated with experiments on superconducting quantum processors. Our findings have broad implications for quantum machine learning and sensing applications.