Hyperparameter Optimization with Optuna or GridSearch

Why Hyperparameter Optimization Matters

A machine learning model is not only defined by the algorithm you choose. It is also shaped by the settings you give that algorithm. A Random Forest with shallow trees behaves very differently from one with deep trees. A support vector machine with a small regularization value can behave very differently from one with a large value.

That is why hyperparameter optimization matters. Two people can use the same dataset and the same algorithm, yet get very different results because their hyperparameters are different. Sometimes the default settings are good enough. Often, they are only a starting point.

The goal is not to chase the highest possible validation score blindly. The goal is to find a setting that generalizes well, trains within your budget, remains interpretable where needed, and fits the business problem. A slightly simpler model with stable performance is often better than a complex model that wins by 0.2% on one split.

Hyperparameters vs Parameters

The difference between parameters and hyperparameters is simple but important. Parameters are learned by the model. Hyperparameters are chosen by the practitioner or optimization algorithm.

In linear regression, coefficients are parameters. In a decision tree, split rules are learned parameters. However, the maximum tree depth, minimum samples per split, and criterion are hyperparameters. They influence how the learning process behaves.

Hyperparameters are powerful because they control model flexibility. If they are too restrictive, the model may underfit. If they are too flexible, the model may overfit. Good tuning finds a useful balance.

The Basic Hyperparameter Optimization Workflow

A reliable tuning workflow starts before you write any optimization code. First, define the business objective. Second, choose the evaluation metric. Third, create a validation strategy. Fourth, define a realistic search space. Only then should you run GridSearch, RandomizedSearch, or Optuna.

This order matters because optimization will chase whatever score you give it. If the metric is wrong, the tuned model will be wrong in a more systematic way. For example, optimizing accuracy on an imbalanced fraud dataset can produce a model that looks good but misses the rare cases that matter most.

After tuning, do not report only the best cross-validation score. Evaluate the final selected model on a holdout test set that was not used during tuning. This protects you from overfitting the validation process itself.

GridSearch: The Exhaustive Approach

GridSearch is the most straightforward hyperparameter optimization method. You define a grid of possible values, and the algorithm evaluates every combination. In scikit-learn, this is usually done using GridSearchCV.

The main advantage is clarity. You know exactly which combinations will be tested. This makes GridSearch easy to explain, reproduce, and audit. The main limitation is cost. If you add more parameters and more values, the number of combinations grows quickly.

Suppose you test 4 values for max_depth, 3 values for min_samples_split, and 5 values for n_estimators. That is 4 × 3 × 5 = 60 combinations. With 5-fold cross-validation, the model trains 300 times. This is manageable for small models, but expensive for larger pipelines.

from sklearn.datasets import load_breast_cancer
from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import GridSearchCV, train_test_split
from sklearn.metrics import classification_report

X, y = load_breast_cancer(return_X_y=True)

X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=42, stratify=y
)

model = RandomForestClassifier(random_state=42)

param_grid = {
    "n_estimators": [100, 200, 300],
    "max_depth": [None, 5, 10, 20],
    "min_samples_split": [2, 5, 10]
}

grid_search = GridSearchCV(
    estimator=model,
    param_grid=param_grid,
    scoring="f1",
    cv=5,
    n_jobs=-1,
    refit=True
)

grid_search.fit(X_train, y_train)

print("Best parameters:", grid_search.best_params_)
print("Best CV score:", grid_search.best_score_)

best_model = grid_search.best_estimator_
y_pred = best_model.predict(X_test)

print(classification_report(y_test, y_pred))

This example tunes a Random Forest classifier and selects the configuration with the best average cross-validation F1-score. The final model is then evaluated on the test set. That separation between tuning and final evaluation is essential.

When GridSearch Works Best

GridSearch is useful when the search space is small and you already have a strong idea of which values matter. It is also useful in teaching, benchmarking, and regulated environments where explainability of the tuning process matters.

For example, if you are tuning logistic regression, you may only need to compare a few values of regularization strength and penalty type. Exhaustive search is reasonable. Similarly, for a small decision tree model, checking a few depth values may be enough.

• Use GridSearch when the search space is small.
• Use GridSearch when you need a transparent tuning process.
• Use GridSearch when each model fit is cheap.
• Avoid GridSearch when many continuous hyperparameters are involved.
• Avoid GridSearch when training one model already takes a long time.

RandomizedSearch: The Practical Middle Ground

GridSearch checks every combination in a fixed grid. RandomizedSearch samples combinations from distributions or lists. In scikit-learn, this is done using RandomizedSearchCV.

RandomizedSearch is often more efficient because not every hyperparameter matters equally. If one parameter is highly influential and another barely matters, exhaustive grid combinations waste time. Random search can explore a wider area with a fixed budget.

The key control is n_iter. This decides how many sampled configurations are evaluated. If n_iter is 50 and cv is 5, the model trains 250 times. You can increase or decrease the budget depending on time and compute.

from scipy.stats import randint
from sklearn.model_selection import RandomizedSearchCV
from sklearn.ensemble import RandomForestClassifier

model = RandomForestClassifier(random_state=42)

param_distributions = {
    "n_estimators": randint(100, 600),
    "max_depth": [None, 5, 10, 20, 30],
    "min_samples_split": randint(2, 20),
    "min_samples_leaf": randint(1, 10)
}

random_search = RandomizedSearchCV(
    estimator=model,
    param_distributions=param_distributions,
    n_iter=40,
    scoring="f1",
    cv=5,
    random_state=42,
    n_jobs=-1,
    refit=True
)

random_search.fit(X_train, y_train)

print("Best parameters:", random_search.best_params_)
print("Best CV score:", random_search.best_score_)

RandomizedSearch is not the main focus of this article, but it is important because it sits between GridSearch and Optuna. If GridSearch is too slow but you are not ready for Bayesian optimization, RandomizedSearch is a strong practical baseline.

Optuna: Smarter Hyperparameter Optimization

Optuna is an automatic hyperparameter optimization framework designed to search intelligently. Instead of checking a fixed grid, Optuna runs trials. Each trial evaluates one set of hyperparameters. Over time, the sampler uses previous results to suggest better candidates.

Optuna is especially useful when your search space is large, continuous, conditional, or expensive. For example, if the best learning rate may be anywhere between 0.00001 and 0.1, a fixed grid is awkward. Optuna can sample values from a continuous range and adapt the search based on performance.

Another major advantage is pruning. In long training jobs, Optuna can stop weak trials early. This saves time because bad configurations do not need to run until completion. The official Optuna Trial API provides the interface for suggesting parameters, reporting intermediate values, and managing trial behavior.

import optuna
from sklearn.datasets import load_breast_cancer
from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import cross_val_score, train_test_split

X, y = load_breast_cancer(return_X_y=True)

X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=42, stratify=y
)

def objective(trial):
    n_estimators = trial.suggest_int("n_estimators", 100, 600)
    max_depth = trial.suggest_int("max_depth", 3, 30)
    min_samples_split = trial.suggest_int("min_samples_split", 2, 20)
    min_samples_leaf = trial.suggest_int("min_samples_leaf", 1, 10)

    model = RandomForestClassifier(
        n_estimators=n_estimators,
        max_depth=max_depth,
        min_samples_split=min_samples_split,
        min_samples_leaf=min_samples_leaf,
        random_state=42,
        n_jobs=-1
    )

    scores = cross_val_score(
        model,
        X_train,
        y_train,
        scoring="f1",
        cv=5,
        n_jobs=-1
    )

    return scores.mean()

study = optuna.create_study(direction="maximize")
study.optimize(objective, n_trials=50)

print("Best score:", study.best_value)
print("Best parameters:", study.best_params)

In Optuna, the objective function is the heart of the workflow. It receives a trial object, asks the trial to suggest hyperparameters, trains a model, evaluates it, and returns a score. The study then manages the full optimization process.

GridSearch vs Optuna: Which One Should You Use?

GridSearch and Optuna are not enemies. They solve the same broad problem at different levels of sophistication. GridSearch is simple and exhaustive. Optuna is flexible and adaptive.

If you are teaching hyperparameter tuning or tuning a small model, GridSearch is often enough. If you are tuning gradient boosting, neural networks, large pipelines, or expensive models, Optuna is usually better.

Designing a Good Search Space

Search space design is where many tuning projects succeed or fail. If the search space is too narrow, the optimizer cannot find strong configurations. If it is too wide, the optimizer wastes time exploring unrealistic values.

The best search space uses domain knowledge. For example, learning rates often work better on a logarithmic scale because values like 0.001 and 0.01 are very different. Regularization values may also need log-scale sampling. Tree depth, on the other hand, is usually a small integer range.

Avoid blindly tuning every hyperparameter. Start with parameters that strongly affect bias, variance, and learning behavior. For a Random Forest, tune n_estimators, max_depth, min_samples_split, and min_samples_leaf first. For gradient boosting, learning_rate, n_estimators, max_depth, subsample, and regularization parameters are often important.

Conditional Hyperparameters in Optuna

Conditional hyperparameters are one reason Optuna feels more natural than GridSearch for complex models. Some hyperparameters only matter when another hyperparameter has a certain value. For example, an SVM with a linear kernel does not need gamma, while an RBF kernel does.

GridSearch can handle this using a list of dictionaries, but Optuna often makes the logic easier to express directly in Python.

from sklearn.svm import SVC
from sklearn.model_selection import cross_val_score

def svm_objective(trial):
    kernel = trial.suggest_categorical("kernel", ["linear", "rbf"])
    C = trial.suggest_float("C", 1e-3, 100, log=True)

    if kernel == "rbf":
        gamma = trial.suggest_float("gamma", 1e-4, 10, log=True)
    else:
        gamma = "scale"

    model = SVC(kernel=kernel, C=C, gamma=gamma)

    scores = cross_val_score(
        model,
        X_train,
        y_train,
        scoring="f1",
        cv=5
    )

    return scores.mean()

This structure keeps the search space logical. The optimizer does not waste trials on irrelevant combinations. That becomes especially valuable when tuning deep learning models, pipelines, or models with many architecture choices.

Pruning: Stop Bad Trials Early

Pruning is one of Optuna’s most useful features for expensive training. During a trial, the model reports intermediate results. If the trial is clearly underperforming, Optuna can stop it early and move on.

This is especially useful for neural networks, gradient boosting, and iterative algorithms. If a configuration performs poorly after several epochs or boosting rounds, there may be little value in completing it.

Optuna provides pruners such as MedianPruner. A pruner compares intermediate results and decides whether a trial should continue. The Optuna MedianPruner documentation explains how median-based pruning uses intermediate values from completed trials.

import optuna

def objective_with_pruning(trial):
    score = 0.0

    for step in range(10):
        # Train or update model here.
        # This is a simplified pattern for demonstration.
        score += trial.suggest_float(f"step_value_{step}", 0.0, 1.0)

        trial.report(score, step)

        if trial.should_prune():
            raise optuna.TrialPruned()

    return score

pruner = optuna.pruners.MedianPruner(n_startup_trials=5)

study = optuna.create_study(
    direction="maximize",
    pruner=pruner
)

study.optimize(objective_with_pruning, n_trials=50)

Pruning should be used carefully. If early performance is not predictive of final performance, pruning may stop promising trials too soon. Always validate whether pruning makes sense for your model and metric.

Cross-Validation: The Foundation of Fair Tuning

Hyperparameter optimization depends on fair evaluation. If you tune on a single train-test split, your best configuration may be lucky rather than genuinely better. Cross-validation reduces this risk by evaluating each configuration across multiple splits.

For classification, stratified cross-validation is often useful because it preserves class proportions across folds. For time series, normal k-fold cross-validation is usually wrong because it can leak future information into the past. Use time-based splits instead.

The validation strategy should match the real prediction scenario. If your model will predict future sales, your validation should respect time. If your model will classify new customers, ensure the same customer does not appear in both train and validation data.

Choosing the Right Metric

The metric tells the optimizer what “good” means. If you choose the wrong metric, the optimizer will faithfully solve the wrong problem.

Accuracy is fine for balanced classification tasks, but weak for imbalanced problems. F1-score is useful when false positives and false negatives both matter. Precision is useful when false alarms are costly. Recall is useful when missing a positive case is costly. ROC-AUC and PR-AUC are useful for ranking quality, but they should be interpreted carefully.

Regression tasks have similar choices. RMSE penalizes large errors more heavily. MAE is easier to interpret because it is the average absolute error. MAPE can be useful for business forecasting, but it behaves poorly when true values are near zero.

For statistical testing and experimentation mindsets, Codeayan’s guide on A/B testing and hypothesis testing is a helpful related read. Hyperparameter tuning is not the same as A/B testing, but both require disciplined comparison and clear success criteria.

Overfitting the Validation Set

Hyperparameter optimization can overfit too. If you try hundreds or thousands of configurations, some may perform well on validation by chance. This is why a final untouched test set matters.

The risk increases when the dataset is small, the search space is huge, or the validation strategy is noisy. In these cases, the best validation score may not translate to better real-world performance.

A practical defense is to keep the search space reasonable, use cross-validation, compare against a baseline, and evaluate the final selected model once on a holdout test set. For critical projects, repeated cross-validation or nested cross-validation may be appropriate.

Hyperparameter Optimization for Different Model Types

Different model families have different tuning priorities. A decision tree is mostly controlled by depth and split rules. A Random Forest is controlled by number of trees, depth, feature sampling, and leaf settings. Gradient boosting is heavily influenced by learning rate, number of estimators, depth, subsampling, and regularization.

Neural networks introduce additional choices: learning rate, optimizer, batch size, dropout, architecture depth, hidden units, weight decay, scheduler, and number of epochs. This makes Optuna especially useful because the search space is large and continuous.

Building a Tuning Strategy That Saves Time

Do not begin with a massive search. Start with a baseline model and understand its errors. Then tune the most influential hyperparameters first. This makes the process faster and easier to interpret.

A useful strategy is coarse-to-fine tuning. First, search a wide range with fewer trials. Then narrow the range around promising values. Finally, run a smaller refined search. This works well with both RandomizedSearch and Optuna.

For GridSearch, avoid large grids at the beginning. Instead, use a small grid to understand sensitivity. If max_depth values of 5, 10, and 20 perform similarly, there is no need to test every integer between them.

• Start with a baseline before tuning.
• Tune the most important hyperparameters first.
• Use log scales for learning rates and regularization values.
• Limit the search budget based on model training cost.
• Save all tuning results for later comparison and audit.

Reading Tuning Results Correctly

The best parameter set is not the only useful output. The full search history tells you how sensitive the model is to different settings. If many configurations perform similarly, the model may be stable. If performance swings wildly, the model may be sensitive and require more careful validation.

With GridSearchCV, results are available in cv_results_. You can convert them into a DataFrame and inspect scores, ranks, and parameter values. With Optuna, the study object stores trial history, best values, and parameter importance tools.

import pandas as pd

results = pd.DataFrame(grid_search.cv_results_)

cols = [
    "rank_test_score",
    "mean_test_score",
    "std_test_score",
    "param_n_estimators",
    "param_max_depth",
    "param_min_samples_split"
]

print(results[cols].sort_values("rank_test_score").head(10))

Look at standard deviation as well as mean score. A configuration with slightly lower average performance but lower variance may be preferable in production. Stability matters when models are used in business workflows.

Production Concerns: Cost, Latency, and Maintainability

Hyperparameter optimization should not be judged only by validation score. A model with 1,000 trees may perform slightly better than a model with 200 trees, but it may also be slower and more expensive to serve.

In real systems, you should consider training time, inference latency, memory usage, explainability, and monitoring complexity. This is especially important for edge deployment, mobile applications, and real-time prediction APIs.

If model size and deployment efficiency matter, Codeayan’s article on model quantization and distillation is a useful companion topic. Optimization should include practical deployment constraints, not just offline metrics.

Common Hyperparameter Optimization Mistakes

The most common mistake is tuning before fixing the data. If features are poor, labels are noisy, or leakage exists, hyperparameter optimization will not solve the core problem. It may only hide it temporarily.

Another mistake is making the search space too large. A huge search space sounds ambitious, but it often wastes compute. Better search space design usually beats brute force.

A third mistake is changing the test set after seeing results. The test set should be treated like a final exam. Once you use it repeatedly for decisions, it becomes part of the tuning process and loses its value.

• Do not optimize before building a strong baseline.
• Do not use the test set during tuning.
• Do not tune too many hyperparameters at once.
• Do not optimize the wrong metric for the business problem.
• Do not ignore training cost, inference speed, and model complexity.

Optuna or GridSearch: A Practical Decision Guide

Use GridSearch when you want simplicity, transparency, and a small search space. It is excellent for beginners and still useful in professional workflows when the number of combinations is manageable.

Use Optuna when the search space is large, continuous, conditional, or expensive. It gives you more control, better search efficiency, and optional pruning. It is especially useful for advanced model tuning and deep learning experiments.

A sensible real-world approach is progressive. Start with defaults. Build a baseline. Try a small GridSearch or RandomizedSearch. If the model is promising and the search space becomes complex, move to Optuna.

Key Takeaways

Conclusion

Further reading: Review the scikit-learn GridSearchCV documentation, RandomizedSearchCV documentation, Optuna official site, and the Optuna Study API documentation.