MicroLIA.optimization

Created on Sat Feb 25 10:39:23 2023

@author: daniel

Module Contents

Classes

objective_xgb	Optuna objective class for optimizing an XGBoost classifier using cross-validation.
objective_nn	Optuna objective class for optimizing an MLP classifier using cross-validation.
objective_rf	Optuna objective class for optimizing a RF classifier using cross-validation.

Functions

hyper_opt([data_x, data_y, clf, n_iter, opt_cv, ...])	Optimizes hyperparameters using a k-fold cross-validation splitting strategy.
borutashap_opt(data_x, data_y[, boruta_trials, model, ...])	Applies a combination of the Boruta algorithm and SHAP values for feature selection.
standardize_data(data_x[, method, return_scaler])	Normalizes the data using the specified method.
impute_missing_values(data[, imputer, strategy, k, ...])	Impute missing values in the input data array using various imputation strategies.
Strawman_imputation(data)	Perform Strawman imputation, a time-efficient algorithm in which missing data values

class MicroLIA.optimization.objective_xgb(data_x, data_y, limit_search=False, opt_cv=3, scoring_metric='f1', SEED_NO=1909)

Bases: object

Optuna objective class for optimizing an XGBoost classifier using cross-validation.

This class defines the optimization logic for tuning XGBoost hyperparameters using the Optuna framework. It supports limited or broad search spaces depending on the limit_search flag, and returns the cross-validated performance metric for each trial.

Parameters:

• data_x (ndarray) – Feature matrix of shape (n_samples, n_features).

• data_y (ndarray or array-like) – Corresponding class labels of shape (n_samples,).

• limit_search (bool, optional) – If True, restricts the hyperparameter search space to a narrower range. Defaults to False (broad search).

• opt_cv (int, optional) – Number of cross-validation folds. Must be >= 2. Default is 3.

• scoring_metric (str, optional) – Evaluation metric used during optimization. Options are: [‘accuracy’, ‘f1’, ‘precision’, ‘recall’, ‘roc_auc’]. Default is ‘f1’.

• SEED_NO (int, optional) – Random seed for reproducibility. Default is 1909.

Returns:

Cross-validated score (mean across folds) for the given trial configuration.

Return type:

float

__call__(trial)

Run a single optimization trial by training the XGBoost model on cross-validation folds and returning the mean performance metric.

Parameters:

trial (optuna.Trial) – A trial object provided by Optuna to suggest hyperparameters.

Returns:

Mean cross-validated score for the trial.

Return type:

float

class MicroLIA.optimization.objective_nn(data_x, data_y, opt_cv, scoring_metric='f1', SEED_NO=1909)

Bases: object

Optuna objective class for optimizing an MLP classifier using cross-validation.

This class defines the optimization logic for tuning XGBoost hyperparameters using the Optuna framework. It supports limited or broad search spaces depending on the limit_search flag, and returns the cross-validated performance metric for each trial.

Parameters:

• data_x (ndarray) – Feature matrix of shape (n_samples, n_features).

• data_y (ndarray or array-like) – Corresponding class labels of shape (n_samples,).

• limit_search (bool, optional) – If True, restricts the hyperparameter search space to a narrower range. Defaults to False (broad search).

• opt_cv (int, optional) – Number of cross-validation folds. Must be >= 2. Default is 3.

• scoring_metric (str, optional) – Evaluation metric used during optimization. Options are: [‘accuracy’, ‘f1’, ‘precision’, ‘recall’, ‘roc_auc’]. Default is ‘f1’.

• SEED_NO (int, optional) – Random seed for reproducibility. Default is 1909.

Returns:

Cross-validated score (mean across folds) for the given trial configuration.

Return type:

float

__call__(trial)

Run a single optimization trial by training the XGBoost model on cross-validation folds and returning the mean performance metric.

Parameters:

trial (optuna.Trial) – A trial object provided by Optuna to suggest hyperparameters.

Returns:

Mean cross-validated score for the trial.

Return type:

float

class MicroLIA.optimization.objective_rf(data_x, data_y, opt_cv, scoring_metric='f1', SEED_NO=1909)

Bases: object

Optuna objective class for optimizing a RF classifier using cross-validation.

This class defines the optimization logic for tuning XGBoost hyperparameters using the Optuna framework. It supports limited or broad search spaces depending on the limit_search flag, and returns the cross-validated performance metric for each trial.

Parameters:

• data_x (ndarray) – Feature matrix of shape (n_samples, n_features).

• data_y (ndarray or array-like) – Corresponding class labels of shape (n_samples,).

• limit_search (bool, optional) – If True, restricts the hyperparameter search space to a narrower range. Defaults to False (broad search).

• opt_cv (int, optional) – Number of cross-validation folds. Must be >= 2. Default is 3.

• scoring_metric (str, optional) – Evaluation metric used during optimization. Options are: [‘accuracy’, ‘f1’, ‘precision’, ‘recall’, ‘roc_auc’]. Default is ‘f1’.

• SEED_NO (int, optional) – Random seed for reproducibility. Default is 1909.

Returns:

Cross-validated score (mean across folds) for the given trial configuration.

Return type:

float

__call__(trial)

Run a single optimization trial by training the XGBoost model on cross-validation folds and returning the mean performance metric.

Parameters:

trial (optuna.Trial) – A trial object provided by Optuna to suggest hyperparameters.

Returns:

Mean cross-validated score for the trial.

Return type:

float

MicroLIA.optimization.hyper_opt(data_x=None, data_y=None, clf='rf', n_iter=25, opt_cv=3, scoring_metric='f1', balance=True, limit_search=True, return_study=True, SEED_NO=1909)

Optimizes hyperparameters using a k-fold cross-validation splitting strategy.

This function constructs a classification engine based on the input classifier (clf) and tunes its hyperparameters using Optuna. If return_study=True, the Optuna Study object will be returned for further analysis or visualization.

Parameters:

• data_x (ndarray, optional) – Feature matrix of shape (n_samples, n_features).

• data_y (ndarray or list of str, optional) – Corresponding class labels of shape (n_samples,).

• clf (str, optional) – Classifier to optimize. Options are: ‘rf’ (Random Forest), ‘nn’ (Neural Network), ‘xgb’ (XGBoost). Default is ‘rf’.

• n_iter (int, optional) – Maximum number of optimization iterations (trials). Default is 25.

• opt_cv (int, optional) – Number of cross-validation folds used per trial. Must be >= 2. Default is 3.

• scoring_metric (str, optional) – Evaluation metric used during optimization. Options are: [‘accuracy’, ‘f1’, ‘precision’, ‘recall’, ‘roc_auc’]. Default is ‘f1’.

• balance (bool, optional) – If True, class weights will be computed and applied to help address class imbalance. Only applies to binary classification. Default is True.

• limit_search (bool, optional) – If True, restricts the hyperparameter search space for quicker optimization. Default is True.

• return_study (bool, optional) – If True, also returns the Optuna Study object used during optimization. Default is True.

• SEED_NO (int, optional) – Random seed for reproducibility. Default is 1909.

Returns:

• model (BaseEstimator) – Trained classifier with optimal hyperparameters.

• params (dict) – Dictionary of the best hyperparameter combination found during optimization.

• study (optuna.study.Study, optional) – Only returned if return_study=True. The Optuna study object used for optimization.

MicroLIA.optimization.borutashap_opt(data_x, data_y, boruta_trials=50, model='rf', importance_type='gain', SEED_NO=1909)

Applies a combination of the Boruta algorithm and SHAP values for feature selection.

This method was developed by Eoghan Keany (2020) and integrates model-based feature selection with Shapley values to yield a stable, interpretable set of features.

See: https://doi.org/10.5281/zenodo.4247618

Parameters:

• data_x (ndarray) – Feature matrix of shape (n_samples, n_features).

• data_y (ndarray or list of str) – Corresponding class labels of shape (n_samples,).

• boruta_trials (int, optional) – Number of trials to run. A higher value increases the robustness of feature selection. Defaults to 50.

• model (str, optional) – Model to use for computing feature importance. Options are: ‘rf’ (Random Forest) or ‘xgb’ (XGBoost). Defaults to ‘rf’.

• importance_type (str, optional) – XGBoost-specific feature importance metric. Options are: [‘gain’, ‘weight’, ‘cover’, ‘total_gain’, ‘total_cover’]. Default is ‘gain’.

• SEED_NO (int, optional) – Random seed for reproducibility. Default is 1909.

Returns:

• selected_indices (ndarray) – 1D array of indices corresponding to selected features.

• feat_selector (BorutaSHAP) – The feature selection object, containing selection history and plotting methods.

MicroLIA.optimization.standardize_data(data_x, method='min-max', return_scaler=True)

Normalizes the data using the specified method.

Tree-based ensembles do not require standardized inputs, but methods such as neural networks or PCA (which are sensitive to feature ranges) benefit from standardization.

Parameters:

• data_x (ndarray) – Training data feature matrix of shape (n_samples, n_features).

• method (str, optional) – Normalization method. Options are: ‘min-max’ (default), ‘robust’, or ‘standard’.

• return_scaler (bool, optional) – If True, returns both the normalized data and the fitted scaler. If False, returns only the normalized data. Default is True.

Returns:

• norm_data_x (ndarray) – Normalized feature matrix.

• scaler (MinMaxScaler or RobustScaler or StandardScaler, optional) – The fitted scaler object. Only returned if return_scaler is True.

Raises:

ValueError – If an unknown method is specified.

MicroLIA.optimization.impute_missing_values(data, imputer=None, strategy='knn', k=3, constant_value=0, nan_threshold=0.5)

Impute missing values in the input data array using various imputation strategies.

This function identifies columns with a high fraction of NaNs (as defined by nan_threshold) and replaces them with zeros before applying imputation. This avoids issues where imputation algorithms would otherwise remove those columns.

Notes

• KNN imputation is sensitive to outliers and performs worse when features are highly correlated. Tang & Ishwaran (2017) report that in such cases, Random Forest-based methods may be superior.

Parameters:

• data (ndarray) – Input data array with missing values. Shape (n_samples, n_features).

• imputer (SimpleImputer or KNNImputer, optional) – A pre-configured imputer object. If provided, only transformation is applied. If None, a new imputer is created and returned. Default is None.

• strategy (str, optional) – Strategy to use for imputation. Options are: ‘mean’, ‘median’, ‘mode’, ‘constant’, or ‘knn’. Default is ‘knn’.

• k (int, optional) – Number of neighbors for k-Nearest Neighbor imputation. Only used if strategy=’knn’. Default is 3.

• constant_value (float or int, optional) – Value to use if strategy=’constant’. Default is 0.

• nan_threshold (float, optional) – Columns with NaN ratios above this threshold will be filled with zeros before imputation. Default is 0.9.

Returns:

• imputed_data (ndarray) – Data with missing values filled in.

• imputer (SimpleImputer or KNNImputer) – The fitted imputer used for the transformation. Only returned if imputer was None at input.

Raises:

ValueError – If an invalid strategy is given or required parameters are missing.

MicroLIA.optimization.Strawman_imputation(data)

Perform Strawman imputation, a time-efficient algorithm in which missing data values are replaced with the median value of the entire non-NaN sample.

If the data is one-hot encoded boolean (e.g., 0/1), the median will correspond to the most frequent value, which is sufficient for random forests that do not accept True/False input.

This is the baseline imputation algorithm used in: Tang & Ishwaran (2017), https://arxiv.org/pdf/1701.05305.pdf

Notes

• This function assumes each row corresponds to a sample and missing values are encoded as either np.nan or np.inf.

• For 1D arrays, the overall median of finite values is used.

• For 2D arrays, the median is computed independently for each column.

Parameters:

data (ndarray or list) – Input array of shape (n,) or (n_samples, n_features) with missing values encoded as NaN or Inf.

Returns:

imputed_data – The input data with missing values replaced using median-based imputation.

Return type:

ndarray