LightGBM Regression

Leaf-wise gradient boosting for large, wide tabular datasets

---

Overview

• Model ID: lightgbm-regression-gpu
• Framework: LightGBM
• Hardware: GPU (histogram training; small footprint)
• Multi-output: Yes (predicts multiple target columns simultaneously)
• Specialist type: tabular

LightGBM grows trees leaf-wise using a histogram-based algorithm, which makes it fast and memory-efficient on large, high-dimensional datasets. Use the exact ID lightgbm-regression-gpu as the model= argument.

For classification, the companion ID is lightgbm-classification (CPU). See the ML Classical index for the full model table.

---

When to Use

✅ Perfect for:

• Large datasets (100k+ rows)
• High-dimensional data (many features)
• Fast training iteration
• Multi-output regression (predict multiple targets at once)

❌ Not ideal for:

• Very small datasets — try XGBoost or Random Forest
• Problems better suited to deep learning (very large, unstructured)

---

Quick Start

Single-output

from colabhive import ColabHive

client = ColabHive(api_key="...", account_id="...")

dataset = client.datasets.upload(name="sales_data", file="./train.csv")

job = client.training.create(
    model="lightgbm-regression-gpu",
    dataset_id=dataset.id,
    hyperparameters={
        "n_estimators": 200,
        "num_leaves": 63,
        "learning_rate": 0.05,
        "target_column": "price",
    },
)
job.wait()
print(job.get_metrics())

# Register as an inference endpoint
endpoint = client.training.register_for_inference(
    run_id=job.id,
    name="lightgbm-sales-regression",
    description="LightGBM regression model",
    visibility="account",
)

# Run inference
result = client.endpoints.infer(
    endpoint_id=endpoint.endpoint_id,
    input_data={"instances": [{"feature1": 1.0, "feature2": 2.5}]},
)
print(result)

Multi-output

job = client.training.create(
    model="lightgbm-regression-gpu",
    dataset_id=dataset.id,
    hyperparameters={
        "n_estimators": 200,
        "num_leaves": 63,
        "learning_rate": 0.05,
        "target_columns": ["price", "quantity", "margin"],
    },
)

Multi-output

When using target_columns (plural), LightGBM trains one model per target (via MultiOutputRegressor). Metrics are reported per target and aggregated.

---

Hyperparameters

Parameter	Default	Range	Description
n_estimators	100	10-1000	Number of boosting iterations
num_leaves	31	2-255	Max leaves per tree
learning_rate	0.1	0.001-0.3	Step size for weight updates
max_depth	6	1-20	Max tree depth
feature_fraction	1.0	0.1-1.0	Fraction of features per tree
lambda_l2	0.0	0-10	L2 regularization
seed	null	0-2147483647	Random seed (null = unseeded)
target_column	—	—	Single target column (promoted to target_columns)
target_columns	—	—	Multiple target columns (multi-output)

---

Expected Results

Regression metrics reported after training:

• MAE (lower is better)
• RMSE (lower is better)
• R² (higher is better; 1.0 is perfect)

---

Tips

1. LightGBM vs XGBoost: LightGBM tends to be faster on large/wide data; XGBoost is a robust default on medium data.
2. Leaf-wise growth: grows the leaf with the largest loss reduction — powerful, but tune num_leaves/max_depth to avoid overfitting small data.
3. Memory efficient: histogram-based training keeps memory low.
4. Multi-output: use target_columns to predict related targets in one job.

---

Next Steps

• ML Classical index — all classical model IDs
• XGBoost Regression
• Random Forest Regression
• Hyperparameter Tuning Guide