* using log directory ‘/data/blackswan/ripley/R/packages/tests-devel/GAparsimony.Rcheck’ * using R Under development (unstable) (2023-01-31 r83739) * using platform: x86_64-pc-linux-gnu (64-bit) * R was compiled by gcc (GCC) 11.3.1 20220421 (Red Hat 11.3.1-2) GNU Fortran (GCC) 11.3.1 20220421 (Red Hat 11.3.1-2) * running under: Fedora 34 (Workstation Edition) * using session charset: UTF-8 * checking for file ‘GAparsimony/DESCRIPTION’ ... OK * checking extension type ... Package * this is package ‘GAparsimony’ version ‘0.9.4’ * checking package namespace information ... OK * checking package dependencies ... OK * checking if this is a source package ... OK * checking if there is a namespace ... OK * checking for executable files ... OK * checking for hidden files and directories ... OK * checking for portable file names ... OK * checking for sufficient/correct file permissions ... OK * checking whether package ‘GAparsimony’ can be installed ... OK * checking package directory ... OK * checking DESCRIPTION meta-information ... OK * checking top-level files ... OK * checking for left-over files ... OK * checking index information ... OK * checking package subdirectories ... OK * checking R files for non-ASCII characters ... OK * checking R files for syntax errors ... OK * checking whether the package can be loaded ... OK * checking whether the package can be loaded with stated dependencies ... OK * checking whether the package can be unloaded cleanly ... OK * checking whether the namespace can be loaded with stated dependencies ... OK * checking whether the namespace can be unloaded cleanly ... OK * checking loading without being on the library search path ... OK * checking dependencies in R code ... OK * checking S3 generic/method consistency ... OK * checking replacement functions ... OK * checking foreign function calls ... OK * checking R code for possible problems ... OK * checking Rd files ... OK * checking Rd metadata ... OK * checking Rd line widths ... OK * checking Rd cross-references ... OK * checking for missing documentation entries ... OK * checking for code/documentation mismatches ... OK * checking Rd \usage sections ... OK * checking Rd contents ... OK * checking for unstated dependencies in examples ... OK * checking examples ... OK * checking examples with --run-donttest ... [29s/29s] ERROR Running examples in ‘GAparsimony-Ex.R’ failed The error most likely occurred in: > ### Name: ga_parsimony > ### Title: GA-PARSIMONY > ### Aliases: ga_parsimony show,ga_parsimony-method > ### print,ga_parsimony-method print,ga_parsimony-method numericOrNA > ### Keywords: optimize > > ### ** Examples > > > ################################# > ### Example 1: Classification ### > ################################# > > # This a toy example that shows how to search, for the *iris* database, > # a parsimony classification NNET model with 'GAparsimony' > # and 'caret' packages. Validation errors and iterations have been > # reduced to speedup the process > > library(GAparsimony) > # Training and testing Datasets > library(caret) Loading required package: ggplot2 Loading required package: lattice > > data(iris) > # Z-score of input features > iris_esc <- data.frame(scale(iris[,1:4]),Species=iris[,5]) > > # Define an 70%/30% train_val/test split of the dataset > set.seed(1234) > inTraining <- createDataPartition(iris_esc$Species, p=.70, list=FALSE) > data_train <- iris_esc[ inTraining,] > data_test <- iris_esc[-inTraining,] > > # Function to evaluate each SVM individual > # ---------------------------------------- > fitness_SVM <- function(chromosome, ...) + { + # First two values in chromosome are 'C' & 'sigma' of 'svmRadial' method + tuneGrid <- data.frame(C=chromosome[1],sigma=chromosome[2]) + + # Next values of chromosome are the selected features (TRUE if > 0.50) + selec_feat <- chromosome[3:length(chromosome)]>0.50 + + # Return -Inf if there is not selected features + if (sum(selec_feat)<1) return(c(kappa_val=-Inf,kappa_test=-Inf,complexity=Inf)) + + # Extract features from the original DB plus response (last column) + data_train_model <- data_train[,c(selec_feat,TRUE)] + data_test_model <- data_test[,c(selec_feat,TRUE)] + + # Validate each individual with only a 2-CV + # Yo obtain a robust validation measure + # use 'repeatedcv' with more folds and times + # (see 2nd and 3rd examples...) + train_control <- trainControl(method = "cv",number = 5) + + # train the model + set.seed(1234) + model <- train(Species ~ ., data=data_train_model, + trControl=train_control, + method="svmRadial", metric="Kappa", + tuneGrid=tuneGrid, verbose=FALSE) + + # Extract validation and test accuracy + accuracy_val <- model$results$Accuracy + accuracy_test <- postResample(pred=predict(model, data_test_model), + obs=data_test_model[,ncol(data_test_model)])[2] + + # Obtain Complexity = Num_Features*1E6+Number of support vectors + complexity <- sum(selec_feat)*1E6+model$finalModel@nSV + + # Return(validation accuracy, testing accuracy, model_complexity) + vect_errors <- c(accuracy_val=accuracy_val, + accuracy_test=accuracy_test,complexity=complexity) + return(vect_errors) + } > > > # --------------------------------------------------------------------------------- > # Search the best parsimonious model with GA-PARSIMONY by using Feature Selection, > # Parameter Tuning and Parsimonious Model Selection > # --------------------------------------------------------------------------------- > library(GAparsimony) > > # Ranges of size and decay > min_param <- c(0.0001, 0.00001) > max_param <- c(0.9999, 0.99999) > names_param <- c("C","sigma") > > # ga_parsimony can be executed with a different set of 'rerank_error' values > rerank_error <- 0.001 > > ## No test: > > GAparsimony_model <- ga_parsimony(fitness=fitness_SVM, + min_param=min_param, + max_param=max_param, + names_param=names_param, + nFeatures=ncol(data_train)-1, + names_features=colnames(data_train)[-ncol(data_train)], + keep_history = TRUE, + rerank_error = rerank_error, + popSize = 20, + maxiter = 20, + early_stop=7, + feat_thres=0.90,# Perc selec features in first iter + feat_mut_thres=0.10,# Prob. feature to be 1 in mutation + not_muted=1, + parallel = FALSE, # speedup with 'n' cores or all with TRUE + seed_ini = 1234) > > > print(paste0("Best Parsimonious SVM with C=", + GAparsimony_model@bestsolution['C'], + " sigma=", + GAparsimony_model@bestsolution['sigma'], + " -> ", + " AccuracyVal=", + round(GAparsimony_model@bestsolution['fitnessVal'],6), + " AccuracyTest=", + round(GAparsimony_model@bestsolution['fitnessTst'],6), + " Num Features=", + round(GAparsimony_model@bestsolution['complexity']/1E6,0), + " Complexity=", + round(GAparsimony_model@bestsolution['complexity'],2))) [1] "Best Parsimonious SVM with C=0.931742658534227 sigma=0.575766682755514 -> AccuracyVal=0.971429 AccuracyTest=0.9 Num Features=1 Complexity=1000035" > > print(summary(GAparsimony_model)) +------------------------------------+ | GA-PARSIMONY | +------------------------------------+ GA-PARSIMONY settings: Number of Parameters = 2 Number of Features = 4 Population size = 20 Maximum of generations = 20 Number of early-stop gen. = 7 Elitism = 4 Crossover probability = 0.8 Mutation probability = 0.1 Max diff(error) to ReRank = 0.001 Perc. of 1s in first popu.= 0.9 Prob. to be 1 in mutation = 0.1 Search domain = C sigma Sepal.Length Sepal.Width Petal.Length Petal.Width Min_param 0.0001 0.00001 0 0 0 0 Max_param 0.9999 0.99999 1 1 1 1 GA-PARSIMONY results: Iterations = 8 Best validation score = 0.9714286 Solution with the best validation score in the whole GA process = [,1] fitnessVal 9.714286e-01 fitnessTst 9.000000e-01 complexity 3.000050e+06 C 5.139881e-01 sigma 2.225027e-01 Sepal.Length 1.000000e+00 Sepal.Width 0.000000e+00 Petal.Length 1.000000e+00 Petal.Width 1.000000e+00 Results of the best individual at the last generation = Best indiv's validat.cost = 0.9714286 Best indiv's testing cost = 0.9 Best indiv's complexity = 1000035 Elapsed time in minutes = 0.427122 BEST SOLUTION = [,1] fitnessVal 9.714286e-01 fitnessTst 9.000000e-01 complexity 1.000035e+06 C 9.317427e-01 sigma 5.757667e-01 Sepal.Length 0.000000e+00 Sepal.Width 0.000000e+00 Petal.Length 0.000000e+00 Petal.Width 1.000000e+00 > > print(parsimony_importance(GAparsimony_model)) Petal.Width Petal.Length Sepal.Width Sepal.Length 100.00 37.50 18.75 12.50 > > > > ################################# > ### Example 2: Classification ### > ################################# > > #This example shows how to search, for the *Sonar* database, > #a parsimony classification SVM model with 'GAparsimony' and 'caret' packages. > > # Training and testing Datasets > library(caret) > library(GAparsimony) > library(mlbench) > data(Sonar) > > set.seed(1234) > inTraining <- createDataPartition(Sonar$Class, p=.80, list=FALSE) > data_train <- Sonar[ inTraining,] > data_test <- Sonar[-inTraining,] > > # Function to evaluate each SVM individual > # ---------------------------------------- > fitness_SVM <- function(chromosome, ...) + { + # First two values in chromosome are 'C' & 'sigma' of 'svmRadial' method + tuneGrid <- data.frame(C=chromosome[1],sigma=chromosome[2]) + + # Next values of chromosome are the selected features (TRUE if > 0.50) + selec_feat <- chromosome[3:length(chromosome)]>0.50 + + # Return -Inf if there is not selected features + if (sum(selec_feat)<1) return(c(kappa_val=-Inf,kappa_test=-Inf,complexity=Inf)) + + # Extract features from the original DB plus response (last column) + data_train_model <- data_train[,c(selec_feat,TRUE)] + data_test_model <- data_test[,c(selec_feat,TRUE)] + + # How to validate each individual + # 'repeats' could be increased to obtain a more robust validation metric. Also, + # 'number' of folds could be adjusted to improve the measure. + train_control <- trainControl(method = "repeatedcv",number = 10,repeats = 10) + + # train the model + set.seed(1234) + model <- train(Class ~ ., data=data_train_model, trControl=train_control, + method="svmRadial", metric="Kappa", + tuneGrid=tuneGrid, verbose=FALSE) + + # Extract kappa statistics (repeated k-fold CV and testing kappa) + kappa_val <- model$results$Kappa + kappa_test <- postResample(pred=predict(model, data_test_model), + obs=data_test_model[,ncol(data_test_model)])[2] + # Obtain Complexity = Num_Features*1E6+Number of support vectors + complexity <- sum(selec_feat)*1E6+model$finalModel@nSV + + # Return(validation error, testing error, model_complexity) + vect_errors <- c(kappa_val=kappa_val,kappa_test=kappa_test,complexity=complexity) + return(vect_errors) + } > > # --------------------------------------------------------------------------------- > # Search the best parsimonious model with GA-PARSIMONY by using Feature Selection, > # Parameter Tuning and Parsimonious Model Selection > # --------------------------------------------------------------------------------- > library(GAparsimony) > > # Ranges of size and decay > min_param <- c(00.0001, 0.00001) > max_param <- c(99.9999, 0.99999) > names_param <- c("C","sigma") > > # ga_parsimony can be executed with a different set of 'rerank_error' values > rerank_error <- 0.001 > > > # 40 individuals per population, 100 max generations with an early stopping > # of 10 generations (CAUTION! 7.34 minutes with 8 cores)!!!!! > GAparsimony_model <- ga_parsimony(fitness=fitness_SVM, + min_param=min_param, + max_param=max_param, + names_param=names_param, + nFeatures=ncol(data_train)-1, + names_features=colnames(data_train)[-ncol(data_train)], + keep_history = TRUE, + rerank_error = rerank_error, + popSize = 40, + maxiter = 100, + early_stop=10, + feat_thres=0.90,# Perc selec features in first iter + feat_mut_thres=0.10,# Prob. feature to be 1 in mutation + parallel = TRUE, seed_ini = 1234) Error in .check_ncores(length(names)) : 32 simultaneous processes spawned Calls: ga_parsimony ... startParallel -> -> makePSOCKcluster -> .check_ncores Execution halted * checking PDF version of manual ... OK * checking for non-standard things in the check directory ... OK * checking for detritus in the temp directory ... OK * checking for new files in some other directories ... OK * DONE Status: 1 ERROR See ‘/data/blackswan/ripley/R/packages/tests-devel/GAparsimony.Rcheck/00check.log’ for details. Command exited with non-zero status 1 Time 0:55.34, 49.91 + 4.94