Xgboost

XGBoost gradient boosting library. Use for tabular ML.

majiayu000 Updated 567 repo stars

File contents

XGBoost

XGBoost is the winningest algorithm in Kaggle history for tabular data. v2.1 (2025) brings native Blackwell GPU support and Polars integration.

When to Use

  • Tabular Data: It usually beats Deep Learning on structured tables.
  • Speed: Extremely optimized C++ backend.

Core Concepts

Gradient Boosting

Building extensive decision trees sequentially, each correcting the previous one's errors.

DMatrix

Internal optimized data structure.

Device Parameter

device="cuda" enables GPU acceleration.

Best Practices (2025)

Do:

  • Use device="cuda": GPU training is 10x faster.
  • Use Early Stopping: Stop training when validation error rises.
  • Pass Polars Dataframes: No need to convert to Pandas/NumPy first.

Don't:

  • Don't use one-hot encoding: Use native categorical support (enable_categorical=True).

References

majiayu000/claude-skill-registry-data/tree/main/ai-ml/xgboost commit ae5d221b8c

Frequently asked questions

npx skillmds@latest add majiayu000/xgboost