MicroLIA.ensemble_model

Created on Sat Jan 21 23:59:14 2017

@author: danielgodinez

Module Contents

Classes

Classifier

Creates a machine learning classifier object. The built-in methods can be used to optimize the engine and output visualizations.

Functions

format_labels(→ list)	Takes a list of labels and returns the list with all words capitalized and underscores removed.
evaluate_model(classifier, data_x, data_y[, ...])	Cross-checks model accuracy and outputs both the predicted
generate_matrix(predicted_labels_list, actual_targets, ...)	Generates the confusion matrix using the output from the evaluate_model() function.
generate_plot(conf_matrix, classes[, normalize, ...])	Generates the confusion matrix figure object, but does not plot.
min_max_norm(data_x)	Normalizes the data to be between 0 and 1. NaN values are ignored.
_set_style_()	Function to configure the matplotlib.pyplot style. This function is called before any images are saved,

class MicroLIA.ensemble_model.Classifier(data_x=None, data_y=None, clf='rf', optimize=False, opt_cv=10, scoring_metric='f1', limit_search=True, impute=False, imp_method='knn', n_iter=25, boruta_trials=50, boruta_model='rf', balance=True, training_data=None, SEED_NO=1909)

Creates a machine learning classifier object. The built-in methods can be used to optimize the engine and output visualizations.

Parameters:

• data_x (ndarray) – 2D array of size (n x m), where n is the number of samples, and m the number of features.

• data_y (ndarray, str) – 1D array containing the corresponing labels.

• clf (str) – The machine learning classifier to optimize. Can either be ‘rf’ for Random Forest, ‘nn’ for Neural Network, or ‘xgb’ for Extreme Gradient Boosting. Defaults to ‘rf’.

• optimize (bool) – If True the Boruta algorithm will be run to identify the features that contain useful information (if boruta_trials > 0), after which the optimal engine hyperparameters will be determined using Bayesian optimization (if n_iter > 0).

• opt_cv (int) – Cross-validations to perform when assesing the performance during the hyperparameter optimization. For example, if cv=3, then each optimization trial will be assessed according to the 3-fold cross validation accuracy. Defaults to 10. If set to None then it will default to 5-fold CV. Cannot be disabled, therefore must be greater than 1. NOTE: The higher this value, the longer the optimization will take.

• scoring_metric (str) – Evaluation metric used during optimization. Options are: [‘accuracy’, ‘f1’, ‘precision’, ‘recall’, ‘roc_auc’]. Default is ‘f1’.

• limit_search (bool) – If False, the search space for the parameters will be expanded, as there are some hyperparameters that can range from 0 to inf. Defaults to True to limit the search and speed up the optimization routine.

• impute (bool) – If True data imputation will be performed to replace NaN values. Defaults to False. If set to True, the imputer attribute will be saved for future transformations.

• imp_method (str, optional) – Imputation strategy to use if impute is set to True. Defaults to ‘knn’. The imputation methods supported include: (‘knn’): Fill missing values using k-Nearest Neighbor imputation. (‘mean’): Fill missing values with the mean of the non-missing values in the same column. (‘median’): Fill missing values with the median of the non-missing values in the same column. (‘mode’): Fill missing values with the mode (most frequent value) of the non-missing values in the same column.

• n_iter (int) – The maximum number of iterations to perform during the hyperparameter search. Defaults to 25. Can be set to 0 to avoid this optimization routine.

• boruta_trials (int) – The number of trials to run when running Boruta for feature selection. Can be set to 0 for no feature selection. Defaults to 50.

• boruta_model (str) – The ensemble algorithm to use when calculating the feature importance metrics for the features, which is utilized by the Boruta algorithm to construct the distributions. Can either be ‘rf’ or ‘xgb’. In practice setting this to ‘xgb’ will result in a more agressive feature selection. Defaults to ‘rf’.

• balance (bool, optional) – If True, a weights array will be calculated and used when fitting the classifier. This can improve classification when classes are imbalanced and is ignored otherwise. This is only applied if the classification is a binary task. Defaults to True.

• training_data (DataFrame, optional) – A dataframe that represents the output from generating the training set. This can be input in lieu of the data_x and data_y arguments. Note that the dataframe must have a “label” column, and is intended to be used after executing the MicroLIA.training_set routine.

• SEED_NO (int) – The random seed to initialize all processes.

create(overwrite_training=False)

Creates the machine learning engine, current options are either a Random Forest, XGBoost, or a Neural Network classifier.

overwrite_training (bool): Whether to replace the original input data_x with the pre-processed

data_x. Defaults to False.

Returns:

Trained and optimized classifier.

save(dirname=None, path=None, overwrite=False)

Saves the trained classifier in a new directory named ‘MicroLIA_ensemble_model’, as well as the imputer and the features to use attributes, if not None.

Parameters:

• dirname (str) – The name of the directory where the model folder will be saved. This directory will be created, and therefore if it already exists in the system an error will appear.

• path (str) – Absolute path where the data folder will be saved Defaults to None, in which case the directory is saved to the local home directory.

• overwrite (bool, optional) – If True the ‘MicroLIA_ensemble_model’ folder this function creates in the specified path will be deleted if it exists and created anew to avoid duplicate files.

load(path=None)

Loads the model, imputer, and feats to use, if created and saved. This function will look for a folder named ‘MicroLIA_models’ in the local home directory, unless a path argument is set.

Parameters:

path (str) – Path where the directory ‘MicroLIA_models’ is saved. Defaults to None, in which case the folder is assumed to be in the local home directory.

predict(time, mag, magerr, convert=True, apply_weights=True, zp=24)

Predics the class label of new, unseen data.

Parameters:

• time (ndarray) – Array of observation timestamps.

• mag (ndarray) – Array of observed magnitudes.

• magerr (ndarray) – Array of corresponding magnitude errors.

• convert (bool) – If False the features are computed with the input magnitudes. Defaults to True to convert and compute in flux.

• apply_weights (bool) – Whether to apply the photometric errors when calculating the features. Defaults to True. Note that this assumes that the erros are Gaussian and uncorrelated.

• zp (float) – Zeropoint of the instrument, used to convert from magnitude to flux. Defaults to 24.

Returns:

Array containing the classes and the corresponding probability predictions.

loo_training()

This method performs leave-one-out cross validation to assess the training set. This effectively simulates the probability prediction of each training instance during a blind search.

plot_tsne(data_y=None, highlight_class=None, norm=True, norm_method='min-max', pca=False, learning_rate=1000, perplexity=35, scale_feature=None, log_feature=False, scale_proba_class=None, cmap='viridis', xlim=None, ylim=None, legend_loc='upper center', title='Feature Parameter Space', savefig=False, fname='tSNE_Projection.png', hdu=None)

Plots a t-SNE projection using the sklearn.manifold.TSNE() method.

Running this method will assign the tsne_x and tsne_y class attributes, which are the scatter point positions from the t-SNE projection. These x,y positions will correspond with the ordering of the data in the data_x feature matrix.

Note

To highlight individual samples, use the data_y optional input and set that sample’s data_y value to a unique name, and set that same label in the highlight_class variable so that it can be highlighted clearly in the plot.

Parameters:

• data_y (ndarray, optional) – A custom labels array, that coincides with the labels in model.data_y. Defaults to None, in which case the model.data_y labels are used.

• highlight_class (optional) – The class label that you wish to highlight, setting this optional parameter will increase the size and alpha parameter for these points in the plot.

• norm (bool) – If True the data will be min-max normalized. Defaults to True.

• norm_method (bool) – Normalization method, if norm is True. Options are: ‘min-max’ (default), ‘robust’, or ‘standard’.

• pca (bool) – If True the data will be fit to a Principal Component Analysis and all of the corresponding principal components will be used to generate the t-SNE plot. Defaults to False.

• learning_rate (float) – The learning rate for t-SNE, usually between 10 and 1000. Default is 1000, can also be set to ‘auto’.

• perplexity (float) – Related to the number of nearest neighbors, with larger datasets requiring a larger perplexity. Default is 35.

• scale_feature (optional, str) – Whether to scale the t-SNE points by a feature value. If None the standard projection is plotted. Currently this only works if the training_data csv has been input. The input feature label must be a column in the csv. Note that the features computed in derivative space have the _deriv suffix at the end (e.g., vonNeumannRatio_deriv)

• log_feature (bool) – Whether to log-scale the feature value, only if scale_feature is not None.

• scale_proba_class (str, int, float) – Used to scale the t-SNE points by probability prediction. The input must be the class label for the particular probability prediction to show. Cannot be applied if scale_feature has been enabled.

• cmap (str) – Colormap to use when scaling the points by either feature value or probability prediction.

• legend_loc (str) – Location of legend, using matplotlib style.

• title (str) – Title of the figure.

• savefig (bool) – If True the figure will not disply but will be saved instead. Defaults to False.

• fname (str) – Filename to be used if savefig is True. Can also include the path to where figure should be saved. Defaults to tSNE_Projection.png which will be saved to the local working directory.

Returns:

AxesImage.

plot_conf_matrix(data_y=None, norm=False, norm_method='min-max', pca=False, k_fold=10, normalize=True, title='Confusion Matrix', savefig=False, fname='Ensemble_Confusion_Matrix.png')

plot_roc_curve(k_fold=10, pca=False, title='Receiver Operating Characteristic Curve', savefig=False)

Plots ROC curve with k-fold cross-validation, as such the standard deviation variations are also plotted.

Parameters:

• k_fold (int, optional) – The number of cross-validations to perform. The output confusion matrix will display the mean accuracy across all k_fold iterations. Defaults to 10.

• pca (bool) – If True the data will be fit to a Principal Component Analysis and all of the corresponding principal components will be used to evaluate the classifier and construct the matrix. Defaults to False.

• title (str, optional) – The title of the output plot.

• savefig (bool) – If True the figure will not disply but will be saved instead. Defaults to False.

Returns:

AxesImage

plot_hyper_opt(baseline=None, xlim=None, ylim=None, xlog=True, ylog=False, ylabel=None, savefig=False)

Plots the hyperparameter optimization history.

Note

The Optuna API has its own plot function: plot_optimization_history(self.optimization_results)

Parameters:

• baseline (float) – Baseline accuracy achieved when using only the default engine hyperparameters. If input a vertical line will be plot to indicate this baseline accuracy. Defaults to None.

• xlim (tuple) – Limits for the x-axis, e.g. xlim = (0, 1000)

• ylim (tuple) – Limits for the y-axis. e.g. ylim = (0.9, 0.94)

• xlog (bool) – If True the x-axis will be log-scaled. Defaults to True.

• ylog (bool) – If True the y-axis will be log-scaled. Defaults to False.

• ylabel (str) – The ylabel of the plot, optional.

• savefig (bool) – If True the figure will not disply but will be saved instead. Defaults to False.

Returns:

AxesImage

plot_feature_opt(feat_names=None, top='all', include_other=True, include_shadow=True, include_rejected=False, flip_axes=True, title='Feature Importance', save_data=False, savefig=False)

Returns plot displaying the z-score distribution of each feature across all trials.

Note

The following can be used to output the plot from the original BorutaShap API.

model.feature_history.plot(which_features=’accepted’, X_size=14)

Can designate to display either ‘all’, ‘accepted’, or ‘tentative’

Parameters:

• feat_names (ndarry, optional) – A list or array containing the names of the features in the data_x matrix, in order. Defaults to None, in which case the respective indices will appear instead. Can be set to ‘default’ which will the features in MicroLIA.features module, so only to be used if all the features were extracted using MicroLIA.extract_features.

• top (float, optional) – Designates how many features to plot. If set to 3, it will plot the top 3 performing features. Can be ‘all’ in which casee all features that were accepted are plotted. Defaults to ‘all’.

• include_other (bool) – Whether to include the features that are not in the top designation, if True these features will be averaged out and displayed. Defaults to True.

• include_shadow (bool) – Whether to include the mean shadow feature that was used as a baseline for ‘random’ behavior. Defaults to True.

• include_rejected (bool) – Whether to include the rejected features, if False these features will not be shown. If set to True or ‘all’, all the rejected features will show. If set to a number, only the designated top rejected features will show. Defaults to False.

• flip_axes (bool) – Whether transpose the figure. Defaults to True.

• save_data (bool) – Whether to save the feature importances as a csv file, defaults to False.

• savefig (bool) – If True the figure will not disply but will be saved instead. Defaults to False.

Returns:

AxesImage

plot_hyper_param_importance(plot_time=True, savefig=False)

Plots the hyperparameter optimization history.

Note

The Optuna API provides its own plotting function: plot_param_importances(self.optimization_results)

Parameters:

• plot_tile (bool) – If True, the importance on the duration will also be included. Defaults to True.

• savefig (bool) – If True the figure will not disply but will be saved instead. Defaults to False.

Returns:

AxesImage

save_hyper_importance()

Calculates and saves binary files containing dictionaries with importance information, one for the importance and one for the duration importance

Note

This procedure is time-consuming but must be run once before plotting the importances. This function will save two files in the model folder for future use.

Returns:

Saves two binary files, importance and duration importance.

MicroLIA.ensemble_model.format_labels(labels: list) → list

Takes a list of labels and returns the list with all words capitalized and underscores removed. Also replaces ‘eta’ with ‘Learning Rate’ and ‘n_estimators’ with ‘Number of Trees’.

Parameters:

labels (list) – A list of strings.

Returns:

Reformatted list, of same lenght.

MicroLIA.ensemble_model.evaluate_model(classifier, data_x, data_y, normalize=True, k_fold=10)

Cross-checks model accuracy and outputs both the predicted and the true class labels.

Parameters:

• classifier – The machine learning classifier to optimize.

• data_x (ndarray) – 2D array of size (n x m), where n is the number of samples, and m is the number of features.

• data_y (ndarray, str) – 1D array containing the corresponding labels.

• normalize (bool, optional) – If False, the confusion matrix will display the total number of objects in the sample. Defaults to True, in which case the values are normalized between 0 and 1.

• k_fold (int, optional) – The number of cross-validations to perform. The output confusion matrix will display the mean accuracy across all k_fold iterations. Defaults to 10.

Returns:

The first output is the 1D array of the true class labels. The second output is the 1D array of the predicted class labels.

MicroLIA.ensemble_model.generate_matrix(predicted_labels_list, actual_targets, classes, normalize=True, title='Confusion Matrix', savefig=False, fname='Ensemble_Confusion_Matrix.png')

Generates the confusion matrix using the output from the evaluate_model() function.

Parameters:

• predicted_labels_list – 1D array containing the predicted class labels.

• actual_targets – 1D array containing the actual class labels.

• classes (list) – A list containing the label of the two training bags. This will be used to set the axis. Ex) classes = [‘ML’, ‘OTHER’]

• normalize (bool, optional) – If True the matrix accuracy will be normalized and displayed as a percentage accuracy. Defaults to True.

• title (str, optional) – The title of the output plot.

• savefig (bool) – If True the figure will not disply but will be saved instead. Defaults to False.

• fname (str) – Filename to be used if savefig is True. Can also include the path to where figure should be saved. Defaults to Ensemble_Confusion_Matrix.png which will be saved to the local working directory.

Returns:

AxesImage.

MicroLIA.ensemble_model.generate_plot(conf_matrix, classes, normalize=False, title='Confusion Matrix', savefig=False)

Generates the confusion matrix figure object, but does not plot.

Parameters:

• conf_matrix – The confusion matrix generated using the generate_matrix() function.

• classes (list) – A list containing the label of the two training bags. This will be used to set the axis. Defaults to a list containing ‘ML’ & ‘OTHER’.

• normalize (bool, optional) – If True the matrix accuracy will be normalized and displayed as a percentage accuracy. Defaults to True.

• title (str, optional) – The title of the output plot.

• savefig (bool) – If True the figure will not disply but will be saved instead. Defaults to False.

Returns:

AxesImage object.

MicroLIA.ensemble_model.min_max_norm(data_x)

Normalizes the data to be between 0 and 1. NaN values are ignored. The transformation matrix will be returned as it will be needed to consitently normalize new data.

Parameters:

data_x (ndarray) – 2D array of size (n x m), where n is the number of samples, and m the number of features.

Returns:

Normalized data array.

MicroLIA.ensemble_model._set_style_()

Function to configure the matplotlib.pyplot style. This function is called before any images are saved, after which the style is reset to the default.