In XGBoost, the weak learners are decision trees. The algorithm builds these trees sequentially, with each new tree aiming to correct the errors of the combined previous trees. Recall that $r_i = -\frac{\partial R(H_{T-1}(x_i))}{\partial H_{T-1}(x_i)}$ are the pseudo-residuals. XGBoost formulates the optimization problem for finding the next weak learner as: $$ h_T = \arg\min_{h \in H} \sum_{i=1}^n (r_i \cdot h(x_i) + \frac{1}{2} h(x_i)^2) $$ We can rewrite this by defining $\hat{y}_i = -r_i$: $$ h_T = \arg\min_{h \in H} \sum_{i=1}^n (-\hat{y}_i \cdot h(x_i) + \frac{1}{2} h(x_i)^2) $$ where $\hat{y}i = \frac{\partial R(H{T-1}(x_i))}{\partial H_{T-1}(x_i)}$ This formulation is equivalent to: ...