Hyperparameters:

all classifiers from scikit-learn (sklearn) module
The scikit-learn version is 0.23.2.

If hyperparameters not specified, default values from sklearn module were used.

--------------------------------------------------------------------------
LogisticRegression
https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html

solver='lbfgs'

Parameter definition:

solver{‘newton-cg’, ‘lbfgs’, ‘liblinear’, ‘sag’, ‘saga’}, default=’lbfgs’
Algorithm to use in the optimization problem.
--------------------------------------------------------------------------
    KNeighborsClassifier
https://scikit-learn.org/stable/modules/generated/sklearn.neighbors.KNeighborsClassifier.html?highlight=kneighborsclassifier#sklearn.neighbors.KNeighborsClassifier

'n_neighbors'=5, metric='minkowski', p=2

n_neighbors: int, default=5
Number of neighbors to use by default for kneighbors queries.

p: int, default=2
Power parameter for the Minkowski metric. When p = 1, this is equivalent to using manhattan_distance (l1), and euclidean_distance (l2) for p = 2. For arbitrary p, minkowski_distance (l_p) is used.

metric: str or callable, default=’minkowski’
the distance metric to use for the tree. The default metric is minkowski, and with p=2 is equivalent to the standard Euclidean metric. See the documentation of DistanceMetric for a list of available metrics. If metric is “precomputed”, X is assumed to be a distance matrix and must be square during fit. X may be a sparse graph, in which case only “nonzero” elements may be considered neighbors.
--------------------------------------------------------------------------
    SVC - linear
https://scikit-learn.org/0.23/modules/generated/sklearn.svm.SVC.html?highlight=svc#sklearn.svm.SVC

kernel="linear", C=0.025


C: float, default=1.0
Regularization parameter. The strength of the regularization is inversely proportional to C. Must be strictly positive. The penalty is a squared l2 penalty.

kernel: {‘linear’, ‘poly’, ‘rbf’, ‘sigmoid’, ‘precomputed’}, default=’rbf’
Specifies the kernel type to be used in the algorithm. It must be one of ‘linear’, ‘poly’, ‘rbf’, ‘sigmoid’, ‘precomputed’ or a callable. If none is given, ‘rbf’ will be used. If a callable is given it is used to pre-compute the kernel matrix from data matrices; that matrix should be an array of shape (n_samples, n_samples).

--------------------------------------------------------------------------
    SVC - poly

kernel="poly", degree=3, C=30, gamma='scale'

C: float, default=1.0
Regularization parameter. The strength of the regularization is inversely proportional to C. Must be strictly positive. The penalty is a squared l2 penalty.

gamma: {‘scale’, ‘auto’} or float, default=’scale’
Kernel coefficient for ‘rbf’, ‘poly’ and ‘sigmoid’.

	if gamma='scale' (default) is passed then it uses 1 / (n_features * 	X.var()) as value of gamma,

	if ‘auto’, uses 1 / n_features.

kernel: {‘linear’, ‘poly’, ‘rbf’, ‘sigmoid’, ‘precomputed’}, default=’rbf’
Specifies the kernel type to be used in the algorithm. It must be one of ‘linear’, ‘poly’, ‘rbf’, ‘sigmoid’, ‘precomputed’ or a callable. If none is given, ‘rbf’ will be used. If a callable is given it is used to pre-compute the kernel matrix from data matrices; that matrix should be an array of shape (n_samples, n_samples).

degree: int, default=3
Degree of the polynomial kernel function (‘poly’). Ignored by all other kernels.
--------------------------------------------------------------------------
    SVC - RBF

gamma=2, C=1

C: float, default=1.0
Regularization parameter. The strength of the regularization is inversely proportional to C. Must be strictly positive. The penalty is a squared l2 penalty.

gamma: {‘scale’, ‘auto’} or float, default=’scale’
Kernel coefficient for ‘rbf’, ‘poly’ and ‘sigmoid’.

	if gamma='scale' (default) is passed then it uses 1 / (n_features * 	X.var()) as value of gamma,

	if ‘auto’, uses 1 / n_features.

kernel: {‘linear’, ‘poly’, ‘rbf’, ‘sigmoid’, ‘precomputed’}, default=’rbf’
Specifies the kernel type to be used in the algorithm. It must be one of ‘linear’, ‘poly’, ‘rbf’, ‘sigmoid’, ‘precomputed’ or a callable. If none is given, ‘rbf’ will be used. If a callable is given it is used to pre-compute the kernel matrix from data matrices; that matrix should be an array of shape (n_samples, n_samples).

--------------------------------------------------------------------------
    GaussianNB

https://scikit-learn.org/0.23/modules/generated/sklearn.naive_bayes.GaussianNB.html?highlight=gaussiannb#sklearn.naive_bayes.GaussianNB

default values
--------------------------------------------------------------------------
    DecisionTreeClassifier
https://scikit-learn.org/0.23/modules/generated/sklearn.tree.DecisionTreeClassifier.html?highlight=decisiontreeclassifier#sklearn.tree.DecisionTreeClassifier

max_depth=5

max_dept: hint, default=None
The maximum depth of the tree. If None, then nodes are expanded until all leaves are pure or until all leaves contain less than min_samples_split samples.

--------------------------------------------------------------------------
    RandomForestClassifier
https://scikit-learn.org/0.23/modules/generated/sklearn.ensemble.RandomForestClassifier.html?highlight=randomforestclassifier#sklearn.ensemble.RandomForestClassifier

max_depth=5, n_estimators=10, max_features=1

n_estimators: int, default=100
The number of trees in the forest.

max_features: int 
The number of features to consider when looking for the best split:
	If int, then consider max_features features at each split.

max_dept: hint, default=None
The maximum depth of the tree. If None, then nodes are expanded until all leaves are pure or until all leaves contain less than min_samples_split samples.

--------------------------------------------------------------------------
    MLPClassifier

https://scikit-learn.org/0.23/modules/generated/sklearn.neural_network.MLPClassifier.html?highlight=mlpclassifier#sklearn.neural_network.MLPClassifier

alpha=1, max_iter=1000

max_iter:int, default=200
Maximum number of iterations. The solver iterates until convergence (determined by ‘tol’) or this number of iterations. For stochastic solvers (‘sgd’, ‘adam’), note that this determines the number of epochs (how many times each data point will be used), not the number of gradient steps.

alpha: float, default=0.0001
L2 penalty (regularization term) parameter.
--------------------------------------------------------------------------
    AdaBoostClassifier

https://scikit-learn.org/0.23/modules/generated/sklearn.ensemble.AdaBoostClassifier.html?highlight=adaboostclassifier#sklearn.ensemble.AdaBoostClassifier

default values
--------------------------------------------------------------------------
Hyperparameters (GRID-SEARCH best params):

all classifiers from scikit-learn (sklearn) module
The scikit-learn version is 0.23.2.

If hyperparameters not specified, default values from sklearn module were used.

--------------------------------------------------------------------------
LogisticRegression
https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html

GRID-SEARCH best params:
{'C': 11.288378916846883, 'penalty': 'l1', 'solver': 'liblinear'}

Parameter definition:
C: float, default=1.0
Inverse of regularization strength; must be a positive float. Like in support vector machines, smaller values specify stronger regularization.

penalty{‘l1’, ‘l2’, ‘elasticnet’, ‘none’}, default=’l2’
Used to specify the norm used in the penalization. The ‘newton-cg’, ‘sag’ and ‘lbfgs’ solvers support only l2 penalties. ‘elasticnet’ is only supported by the ‘saga’ solver. If ‘none’ (not supported by the liblinear solver), no regularization is applied.

solver{‘newton-cg’, ‘lbfgs’, ‘liblinear’, ‘sag’, ‘saga’}, default=’lbfgs’
Algorithm to use in the optimization problem.
--------------------------------------------------------------------------
    KNeighborsClassifier
https://scikit-learn.org/stable/modules/generated/sklearn.neighbors.KNeighborsClassifier.html?highlight=kneighborsclassifier#sklearn.neighbors.KNeighborsClassifier

GRID-SEARCH best params:
{'metric': 'minkowski', 'n_jobs': -1, 'n_neighbors': 5, 'p': 2}

n_neighbors: int, default=5
Number of neighbors to use by default for kneighbors queries.

p: int, default=2
Power parameter for the Minkowski metric. When p = 1, this is equivalent to using manhattan_distance (l1), and euclidean_distance (l2) for p = 2. For arbitrary p, minkowski_distance (l_p) is used.

n_jobs: int, default=None
The number of parallel jobs to run for neighbors search. None means 1 unless in a joblib.parallel_backend context. -1 means using all processors. See Glossary for more details. Doesn’t affect fit method.

metric: str or callable, default=’minkowski’
the distance metric to use for the tree. The default metric is minkowski, and with p=2 is equivalent to the standard Euclidean metric. See the documentation of DistanceMetric for a list of available metrics. If metric is “precomputed”, X is assumed to be a distance matrix and must be square during fit. X may be a sparse graph, in which case only “nonzero” elements may be considered neighbors.
--------------------------------------------------------------------------
    SVC
https://scikit-learn.org/0.23/modules/generated/sklearn.svm.SVC.html?highlight=svc#sklearn.svm.SVC

GRID-SEARCH best params:
{'C': 0.08858667904100823,
  'gamma': 'scale',
  'kernel': 'linear',
  'random_state': 0},


C: float, default=1.0
Regularization parameter. The strength of the regularization is inversely proportional to C. Must be strictly positive. The penalty is a squared l2 penalty.

gamma: {‘scale’, ‘auto’} or float, default=’scale’
Kernel coefficient for ‘rbf’, ‘poly’ and ‘sigmoid’.

if gamma='scale' (default) is passed then it uses 1 / (n_features * X.var()) as value of gamma,

if ‘auto’, uses 1 / n_features.

kernel: {‘linear’, ‘poly’, ‘rbf’, ‘sigmoid’, ‘precomputed’}, default=’rbf’
Specifies the kernel type to be used in the algorithm. It must be one of ‘linear’, ‘poly’, ‘rbf’, ‘sigmoid’, ‘precomputed’ or a callable. If none is given, ‘rbf’ will be used. If a callable is given it is used to pre-compute the kernel matrix from data matrices; that matrix should be an array of shape (n_samples, n_samples).

random_state: int or RandomState instance, default=None
Controls the pseudo random number generation for shuffling the data for probability estimates. Ignored when probability is False. Pass an int for reproducible output across multiple function calls. See Glossary.

--------------------------------------------------------------------------
    SVC

GRID-SEARCH best params:
{'C': 1.623776739188721,
  'degree': 1,
  'gamma': 'scale',
  'kernel': 'poly',
  'random_state': 0}

C: float, default=1.0
Regularization parameter. The strength of the regularization is inversely proportional to C. Must be strictly positive. The penalty is a squared l2 penalty.

gamma: {‘scale’, ‘auto’} or float, default=’scale’
Kernel coefficient for ‘rbf’, ‘poly’ and ‘sigmoid’.

if gamma='scale' (default) is passed then it uses 1 / (n_features * X.var()) as value of gamma,

if ‘auto’, uses 1 / n_features.

kernel: {‘linear’, ‘poly’, ‘rbf’, ‘sigmoid’, ‘precomputed’}, default=’rbf’
Specifies the kernel type to be used in the algorithm. It must be one of ‘linear’, ‘poly’, ‘rbf’, ‘sigmoid’, ‘precomputed’ or a callable. If none is given, ‘rbf’ will be used. If a callable is given it is used to pre-compute the kernel matrix from data matrices; that matrix should be an array of shape (n_samples, n_samples).

random_state: int or RandomState instance, default=None
Controls the pseudo random number generation for shuffling the data for probability estimates. Ignored when probability is False. Pass an int for reproducible output across multiple function calls. See Glossary.

degree: int, default=3
Degree of the polynomial kernel function (‘poly’). Ignored by all other kernels.
--------------------------------------------------------------------------
    SVC

GRID-SEARCH best params:
{'C': 11.288378916846883,
  'gamma': 0.004832930238571752,
  'kernel': 'rbf',
  'random_state': 0}

C: float, default=1.0
Regularization parameter. The strength of the regularization is inversely proportional to C. Must be strictly positive. The penalty is a squared l2 penalty.

gamma: {‘scale’, ‘auto’} or float, default=’scale’
Kernel coefficient for ‘rbf’, ‘poly’ and ‘sigmoid’.

if gamma='scale' (default) is passed then it uses 1 / (n_features * X.var()) as value of gamma,

if ‘auto’, uses 1 / n_features.

kernel: {‘linear’, ‘poly’, ‘rbf’, ‘sigmoid’, ‘precomputed’}, default=’rbf’
Specifies the kernel type to be used in the algorithm. It must be one of ‘linear’, ‘poly’, ‘rbf’, ‘sigmoid’, ‘precomputed’ or a callable. If none is given, ‘rbf’ will be used. If a callable is given it is used to pre-compute the kernel matrix from data matrices; that matrix should be an array of shape (n_samples, n_samples).

random_state: int or RandomState instance, default=None
Controls the pseudo random number generation for shuffling the data for probability estimates. Ignored when probability is False. Pass an int for reproducible output across multiple function calls. See Glossary.
--------------------------------------------------------------------------
    GaussianNB

https://scikit-learn.org/0.23/modules/generated/sklearn.naive_bayes.GaussianNB.html?highlight=gaussiannb#sklearn.naive_bayes.GaussianNB

GRID-SEARCH best params:
default values
--------------------------------------------------------------------------
    DecisionTreeClassifier
https://scikit-learn.org/0.23/modules/generated/sklearn.tree.DecisionTreeClassifier.html?highlight=decisiontreeclassifier#sklearn.tree.DecisionTreeClassifier

GRID-SEARCH best params:
{'criterion': 'entropy', 'max_depth': 4, 'random_state': 0}

criterion: {“gini”, “entropy”}, default=”gini”
The function to measure the quality of a split. Supported criteria are “gini” for the Gini impurity and “entropy” for the information gain.

max_dept: hint, default=None
The maximum depth of the tree. If None, then nodes are expanded until all leaves are pure or until all leaves contain less than min_samples_split samples.

random_state: int, RandomState instance, default=None
Controls the randomness of the estimator. The features are always randomly permuted at each split, even if splitter is set to "best". When max_features < n_features, the algorithm will select max_features at random at each split before finding the best split among them. But the best found split may vary across different runs, even if max_features=n_features. That is the case, if the improvement of the criterion is identical for several splits and one split has to be selected at random. To obtain a deterministic behaviour during fitting, random_state has to be fixed to an integer. See Glossary for details.
--------------------------------------------------------------------------
    RandomForestClassifier
https://scikit-learn.org/0.23/modules/generated/sklearn.ensemble.RandomForestClassifier.html?highlight=randomforestclassifier#sklearn.ensemble.RandomForestClassifier

GRID-SEARCH best params:
{'max_features': 51, 'n_estimators': 11}

n_estimators: int, default=100
The number of trees in the forest.

max_features: int 
The number of features to consider when looking for the best split:

If int, then consider max_features features at each split.

--------------------------------------------------------------------------
    MLPClassifier

https://scikit-learn.org/0.23/modules/generated/sklearn.neural_network.MLPClassifier.html?highlight=mlpclassifier#sklearn.neural_network.MLPClassifier

GRID-SEARCH best params:
{'alpha': 0.002154434690031882, 'max_iter': 1000, 'solver': 'lbfgs'}

solver{‘lbfgs’, ‘sgd’, ‘adam’}, default=’adam’
	The solver for weight optimization.

	‘lbfgs’ is an optimizer in the family of quasi-Newton methods.

	‘sgd’ refers to stochastic gradient descent.

	Note: The default solver ‘adam’ works pretty well on relatively large datasets (with thousands of training samples or more) in terms of both training time and validation score. For small datasets, however, ‘lbfgs’ can converge faster and perform better.

max_iter:int, default=200
Maximum number of iterations. The solver iterates until convergence (determined by ‘tol’) or this number of iterations. For stochastic solvers (‘sgd’, ‘adam’), note that this determines the number of epochs (how many times each data point will be used), not the number of gradient steps.

alpha: float, default=0.0001
L2 penalty (regularization term) parameter.
--------------------------------------------------------------------------
    AdaBoostClassifier

https://scikit-learn.org/0.23/modules/generated/sklearn.ensemble.AdaBoostClassifier.html?highlight=adaboostclassifier#sklearn.ensemble.AdaBoostClassifier

GRID-SEARCH best params:
{'n_estimators': 11, 'random_state': 0}

n_estimators: int, default=50
The maximum number of estimators at which boosting is terminated. In case of perfect fit, the learning procedure is stopped early.

random_state: int or RandomState, default=None
Controls the random seed given at each base_estimator at each boosting iteration. Thus, it is only used when base_estimator exposes a random_state. Pass an int for reproducible output across multiple function calls. See Glossary.
