* using log directory ‘/data/blackswan/ripley/R/packages/tests-devel/CopyDetect.Rcheck’ * using R Under development (unstable) (2022-04-08 r82134) * using platform: x86_64-pc-linux-gnu (64-bit) * using session charset: UTF-8 * checking for file ‘CopyDetect/DESCRIPTION’ ... OK * this is package ‘CopyDetect’ version ‘1.3’ * checking package namespace information ... OK * checking package dependencies ... OK * checking if this is a source package ... OK * checking if there is a namespace ... OK * checking for executable files ... OK * checking for hidden files and directories ... OK * checking for portable file names ... OK * checking for sufficient/correct file permissions ... OK * checking whether package ‘CopyDetect’ can be installed ... [14s/14s] OK * checking package directory ... OK * checking DESCRIPTION meta-information ... OK * checking top-level files ... OK * checking for left-over files ... OK * checking index information ... OK * checking package subdirectories ... OK * checking R files for non-ASCII characters ... OK * checking R files for syntax errors ... OK * checking whether the package can be loaded ... OK * checking whether the package can be loaded with stated dependencies ... OK * checking whether the package can be unloaded cleanly ... OK * checking whether the namespace can be loaded with stated dependencies ... OK * checking whether the namespace can be unloaded cleanly ... OK * checking loading without being on the library search path ... OK * checking dependencies in R code ... OK * checking S3 generic/method consistency ... OK * checking replacement functions ... OK * checking foreign function calls ... OK * checking R code for possible problems ... [20s/20s] OK * checking Rd files ... OK * checking Rd metadata ... OK * checking Rd line widths ... OK * checking Rd cross-references ... OK * checking for missing documentation entries ... OK * checking for code/documentation mismatches ... OK * checking Rd \usage sections ... OK * checking Rd contents ... OK * checking for unstated dependencies in examples ... OK * checking contents of ‘data’ directory ... OK * checking data for non-ASCII characters ... OK * checking data for ASCII and uncompressed saves ... OK * checking examples ... OK * checking examples with --run-donttest ... [12s/13s] ERROR Running examples in ‘CopyDetect-Ex.R’ failed The error most likely occurred in: > ### Name: similarity1 > ### Title: Response Similarity for Dcihotomously Scored Items > ### Aliases: similarity1 > > ### ** Examples > > > data(form1) > dim(form1) [1] 1636 172 > head(form1) EID cent_id iraw.A1 iraw.A2 iraw.A3 iraw.A4 iraw.A5 iraw.A6 iraw.A7 1 e100001 59 1 1 1 0 1 0 0 2 e100002 67 1 0 1 0 0 0 0 3 e100003 41 0 0 0 0 1 0 1 4 e100004 4214 1 1 0 0 1 0 0 5 e100005 2305 1 1 0 0 1 1 0 6 e100006 5880 0 1 0 0 1 0 1 iraw.A8 iraw.A9 iraw.A10 iraw.A11 iraw.A12 iraw.A13 iraw.A14 iraw.A15 1 0 0 1 0 0 0 0 0 2 0 0 0 0 0 0 0 1 3 0 0 0 0 1 1 1 0 4 0 1 0 0 0 1 0 0 5 0 0 1 1 0 0 0 0 6 1 0 0 0 0 0 1 0 iraw.A16 iraw.A17 iraw.A18 iraw.A19 iraw.A20 iraw.A21 iraw.A22 iraw.A23 1 1 1 1 0 0 1 1 0 2 0 1 0 0 0 1 1 1 3 1 1 1 0 0 0 1 0 4 0 1 0 0 0 0 1 1 5 0 0 0 0 0 1 1 1 6 0 0 0 1 0 1 1 1 iraw.A24 iraw.A25 iraw.A26 iraw.A27 iraw.A28 iraw.A29 iraw.A30 iraw.A31 1 0 0 0 0 1 0 0 0 2 0 0 0 0 1 0 0 0 3 0 1 0 0 0 1 0 0 4 0 0 0 0 0 0 0 0 5 1 1 0 0 1 0 0 1 6 0 0 0 1 1 1 0 0 iraw.A32 iraw.A33 iraw.A34 iraw.A35 iraw.A36 iraw.A37 iraw.A38 iraw.A39 1 1 0 0 0 1 0 0 0 2 0 0 1 0 0 1 1 0 3 0 0 0 1 1 0 1 1 4 0 1 1 1 1 1 0 0 5 1 0 1 0 1 1 0 0 6 0 0 1 0 0 1 0 0 iraw.A40 iraw.A41 iraw.A42 iraw.A43 iraw.A44 iraw.A45 iraw.A46 iraw.A47 1 0 0 0 0 0 0 0 0 2 0 1 1 0 0 0 1 1 3 0 1 1 0 0 0 0 1 4 0 0 0 1 0 1 0 1 5 1 0 0 0 0 0 0 0 6 0 1 1 0 0 1 1 1 iraw.A48 iraw.A49 iraw.A50 iraw.A51 iraw.A52 iraw.A53 iraw.A54 iraw.A55 1 0 1 1 0 1 0 1 0 2 1 0 0 0 0 0 0 0 3 1 1 0 1 0 0 1 0 4 0 0 0 1 0 0 0 0 5 0 1 0 1 1 1 1 1 6 0 0 0 0 0 0 1 0 iraw.A56 iraw.A57 iraw.A58 iraw.A59 iraw.A60 iraw.A61 iraw.A62 iraw.A63 1 0 1 0 1 0 1 0 0 2 1 1 0 0 0 0 1 0 3 0 1 1 0 0 1 0 0 4 0 0 0 1 0 0 0 0 5 0 1 0 0 0 0 1 0 6 0 0 0 1 0 0 0 0 iraw.A64 iraw.A65 iraw.A66 iraw.A67 iraw.A68 iraw.A69 iraw.A70 iraw.A71 1 0 1 1 0 0 0 0 1 2 0 1 0 1 0 0 0 0 3 0 0 0 0 0 0 0 0 4 1 0 1 0 0 0 1 0 5 0 0 0 1 0 0 0 1 6 1 0 0 0 1 0 0 0 iraw.A72 iraw.A73 iraw.A74 iraw.A75 iraw.A76 iraw.A77 iraw.A78 iraw.A79 1 1 0 1 0 0 0 0 1 2 0 0 0 0 1 0 0 0 3 0 1 1 1 0 0 0 1 4 0 1 1 0 0 1 1 0 5 1 0 0 0 0 0 0 1 6 0 0 0 1 0 0 1 1 iraw.A80 iraw.A81 iraw.A82 iraw.A83 iraw.A84 iraw.A85 iraw.A86 iraw.A87 1 0 0 0 0 1 0 0 0 2 0 1 1 0 1 0 0 0 3 0 1 1 0 0 1 0 0 4 0 1 0 0 0 1 0 0 5 0 0 1 0 1 0 0 0 6 0 1 0 0 1 1 0 0 iraw.A88 iraw.A89 iraw.A90 iraw.A91 iraw.A92 iraw.A93 iraw.A94 iraw.A95 1 1 1 0 0 0 0 0 0 2 0 1 1 0 0 0 0 0 3 1 0 0 1 0 0 1 1 4 1 0 1 0 1 0 1 0 5 1 0 0 0 0 0 0 0 6 1 0 1 1 0 1 1 1 iraw.A96 iraw.A97 iraw.A98 iraw.A99 iraw.A100 iraw.A101 iraw.A102 iraw.A103 1 1 0 0 1 0 1 0 0 2 0 0 0 0 0 1 0 1 3 1 0 0 0 0 0 0 1 4 0 1 0 0 0 1 1 1 5 0 1 0 1 0 0 0 0 6 0 0 1 1 0 1 0 1 iraw.A104 iraw.A105 iraw.A106 iraw.A107 iraw.A108 iraw.A109 iraw.A110 1 0 1 0 1 1 0 0 2 0 0 0 0 1 1 0 3 0 0 0 1 0 1 0 4 0 1 0 1 0 0 1 5 0 0 0 0 0 0 0 6 1 1 1 0 1 1 1 iraw.A111 iraw.A112 iraw.A113 iraw.A114 iraw.A115 iraw.A116 iraw.A117 1 0 0 0 0 1 0 0 2 0 1 1 1 1 1 0 3 1 0 0 1 0 0 0 4 1 0 0 1 1 0 0 5 1 0 0 1 0 0 1 6 0 1 1 1 0 0 1 iraw.A118 iraw.A119 iraw.A120 iraw.A121 iraw.A122 iraw.A123 iraw.A124 1 0 1 0 0 0 0 0 2 0 0 0 0 0 1 1 3 0 1 1 0 0 0 0 4 0 0 0 1 0 1 0 5 1 0 0 1 0 1 0 6 0 0 0 1 1 1 0 iraw.A125 iraw.A126 iraw.A127 iraw.A128 iraw.A129 iraw.A130 iraw.A131 1 0 1 0 0 0 0 0 2 1 0 1 0 1 0 0 3 0 0 0 1 0 0 0 4 0 1 0 0 0 0 1 5 0 1 1 1 1 0 1 6 1 0 1 1 1 0 0 iraw.A132 iraw.A133 iraw.A134 iraw.A135 iraw.A136 iraw.A137 iraw.A138 1 0 0 1 1 0 0 0 2 0 0 1 0 0 1 0 3 0 1 1 1 0 0 0 4 0 1 1 0 0 0 0 5 0 1 1 0 1 0 1 6 0 1 0 0 0 0 0 iraw.A139 iraw.A140 iraw.A141 iraw.A142 iraw.A143 iraw.A144 iraw.A145 1 0 0 0 1 0 0 1 2 0 0 1 1 0 0 0 3 0 0 0 1 0 0 1 4 0 0 1 1 0 0 1 5 0 1 0 0 1 1 0 6 0 0 0 0 0 0 0 iraw.A146 iraw.A147 iraw.A148 iraw.A149 iraw.A150 iraw.A151 iraw.A152 1 1 0 1 1 1 1 0 2 0 0 0 0 0 0 1 3 1 0 0 1 1 1 0 4 0 0 0 0 0 0 1 5 0 1 0 0 1 0 0 6 0 0 0 0 0 0 0 iraw.A153 iraw.A154 iraw.A155 iraw.A156 iraw.A157 iraw.A158 iraw.A159 1 0 1 1 1 0 0 0 2 0 0 0 1 0 1 1 3 1 1 0 1 1 1 0 4 0 1 1 1 1 0 0 5 0 0 0 1 1 1 0 6 1 1 0 0 0 1 0 iraw.A160 iraw.A161 iraw.A162 iraw.A163 iraw.A164 iraw.A165 iraw.A166 1 0 0 1 1 0 0 1 2 1 1 0 1 1 0 1 3 1 0 0 0 0 0 1 4 0 0 0 1 1 0 1 5 0 1 0 1 1 0 0 6 0 0 0 1 0 1 0 iraw.A167 iraw.A168 iraw.A169 iraw.A170 1 0 1 0 0 2 0 0 1 0 3 0 0 0 0 4 0 1 1 1 5 0 0 0 1 6 0 1 1 1 > > # the first column of this dataset is unique individual IDs > # the second column of this dataset is unique center IDs > # From Column 3 to Column 172, dichotomous item responses > > > # For the sake of reducing the computational time, > # I will analyze a subset of this dataset (first 20 items) > > subset <- form1[1:1000,1:22] > > dim(subset) [1] 1000 22 > head(subset) EID cent_id iraw.A1 iraw.A2 iraw.A3 iraw.A4 iraw.A5 iraw.A6 iraw.A7 1 e100001 59 1 1 1 0 1 0 0 2 e100002 67 1 0 1 0 0 0 0 3 e100003 41 0 0 0 0 1 0 1 4 e100004 4214 1 1 0 0 1 0 0 5 e100005 2305 1 1 0 0 1 1 0 6 e100006 5880 0 1 0 0 1 0 1 iraw.A8 iraw.A9 iraw.A10 iraw.A11 iraw.A12 iraw.A13 iraw.A14 iraw.A15 1 0 0 1 0 0 0 0 0 2 0 0 0 0 0 0 0 1 3 0 0 0 0 1 1 1 0 4 0 1 0 0 0 1 0 0 5 0 0 1 1 0 0 0 0 6 1 0 0 0 0 0 1 0 iraw.A16 iraw.A17 iraw.A18 iraw.A19 iraw.A20 1 1 1 1 0 0 2 0 1 0 0 0 3 1 1 1 0 0 4 0 1 0 0 0 5 0 0 0 0 0 6 0 0 0 1 0 > > > # Computing similarity for a single pair > > a <- similarity1(data = subset, + model = "1PL", + person.id = "EID", + item.loc = 3:22, + single.pair= c("e100287","e100869")) Iteration: 1, Log-Lik: -10572.872, Max-Change: 0.14716 Iteration: 2, Log-Lik: -10531.998, Max-Change: 0.07004 Iteration: 3, Log-Lik: -10521.677, Max-Change: 0.03528 Iteration: 4, Log-Lik: -10518.905, Max-Change: 0.01800 Iteration: 5, Log-Lik: -10518.177, Max-Change: 0.00869 Iteration: 6, Log-Lik: -10518.001, Max-Change: 0.00427 Iteration: 7, Log-Lik: -10517.955, Max-Change: 0.00240 Iteration: 8, Log-Lik: -10517.942, Max-Change: 0.00119 Iteration: 9, Log-Lik: -10517.939, Max-Change: 0.00060 Iteration: 10, Log-Lik: -10517.938, Max-Change: 0.00031 Iteration: 11, Log-Lik: -10517.938, Max-Change: 0.00016 Iteration: 12, Log-Lik: -10517.938, Max-Change: 0.00008 > > print(a) ************************************************************************ CopyDetect - An R Package to Compute Response Similarity Indices for Multiple-Choice Tests Version 1.3, released on October 2018 Cengiz Zopluoglu Assistant Professor University of Miami - Department of Educational and Psychological Studies Research, Measurement, and Evaluation Program c.zopluoglu@miami.edu ************************************************************************* Processing Date: Sun Apr 10 09:34:37 2022 Suspected Copier: Examinee ID e100287 Response Vector: 1 1 0 1 0 1 1 1 0 1 1 0 1 1 0 0 1 0 1 0 Suspected Source: Examinee ID e100869 Response Vector: 1 1 1 1 0 1 1 1 1 1 1 1 1 1 0 1 1 0 1 0 Number-Correct Score for Examinee ID e100287 : 12 Number-Correct Score for Examinee ID e100869 : 16 MLE Ability Estimate for Examinee ID e100287 : -1.1203 MLE Ability Estimate for Examinee ID e100869 : 0.5512 Number of Identical Incorrect Responses: 4 Number of Identical Correct Responses: 12 Number of Identical Responses: 16 Alpha Level Index 0.05 0.01 0.001 W Not Flagged Not Flagged Not Flagged M4 Flagged Not Flagged Not Flagged GBT Not Flagged Not Flagged Not Flagged K Not Flagged Not Flagged Not Flagged K1 Not Flagged Not Flagged Not Flagged K2 Not Flagged Not Flagged Not Flagged S1 Not Flagged Not Flagged Not Flagged S2 Not Flagged Not Flagged Not Flagged ******************************* W Index ********************************* Estimated Probabilities of Correct Response: Examinee 287 Examinee 869 Item 1 0.788 0.905 Item 2 0.693 0.852 Item 3 0.692 0.852 Item 4 0.585 0.783 Item 5 0.502 0.721 Item 6 0.811 0.917 Item 7 0.742 0.880 Item 8 0.626 0.811 Item 9 0.383 0.614 Item 10 0.819 0.921 Item 11 0.534 0.746 Item 12 0.243 0.451 Item 13 0.628 0.812 Item 14 0.601 0.794 Item 15 0.522 0.737 Item 16 0.681 0.845 Item 17 0.835 0.929 Item 18 0.371 0.602 Item 19 0.890 0.954 Item 20 0.452 0.679 Expected Number of Identical Responses = 12.703 Standard Deviation of The Expected Number of Identical Responses: = 2.034 W index value = 1.6205 Likelihood of Agreemet (p-value) = 0.05256 ********************** Generalized Binomial Test ************************ Probability of Matching on Each Item: Prob. of Matching Item 1 0.733 Item 2 0.636 Item 3 0.635 Item 4 0.548 Item 5 0.501 Item 6 0.760 Item 7 0.684 Item 8 0.578 Item 9 0.473 Item 10 0.768 Item 11 0.517 Item 12 0.525 Item 13 0.580 Item 14 0.560 Item 15 0.511 Item 16 0.625 Item 17 0.787 Item 18 0.474 Item 19 0.854 Item 20 0.483 Exact Probability Distribution for Number of Matches: Probability of 0 Match 0.000 Probability of 1 Match 0.000 Probability of 2 Match 0.000 Probability of 3 Match 0.000 Probability of 4 Match 0.000 Probability of 5 Match 0.001 Probability of 6 Match 0.003 Probability of 7 Match 0.010 Probability of 8 Match 0.027 Probability of 9 Match 0.059 Probability of 10 Match 0.106 Probability of 11 Match 0.155 Probability of 12 Match 0.184 Probability of 13 Match 0.177 Probability of 14 Match 0.137 Probability of 15 Match 0.084 Probability of 16 Match 0.040 Probability of 17 Match 0.014 Probability of 18 Match 0.003 Probability of 19 Match 0.001 Probability of 20 Match 0.000 Probability of Observing 16 or More Matches = 0.05751 *************************** K Index ******************************* Number-Incorrect Score for Examinee 287 (suspected copier): 8 Number-Incorrect Score for Examinee 869 (suspected source): 4 Number of Examinees in the Subgroup of Number-Incorrect Score 8 : 65 Number of Identical Incorrect Responses between Each Examinee in This Subgroup and Source Examinee: 3 2 4 3 1 2 2 3 2 1 1 4 3 1 2 3 3 2 4 2 1 2 2 2 1 1 2 2 4 1 2 3 2 1 1 2 1 4 2 2 0 4 1 3 3 2 3 2 4 2 3 0 1 1 2 1 2 4 4 2 3 2 3 3 4 Mean: 2.2308 Binomial Probability of Matching on an Identical Incorrect Response with Source Examinee for This Number-Incorrect Subgroup: 2.2308 / 4 = 0.5577 Probability of Observing 4 or More Identical Incorrect Matches Using Binomial Distribution = 0.0967 *************************** K Variants(K1,K2,S1) ******************************* Number Incorrect Number Observed Average Number of Predicted Average Number of Score Group of Examinees Identical Incorrect Matches Identical Incorrect Matches with Source Examinee with Source Examinee Linear Quadratic Loglinear Model Model Model 0 11 0.000 0.340 0.021 0.728 1 21 0.381 0.550 0.351 0.813 2 82 0.585 0.760 0.664 0.907 3 147 0.986 0.970 0.962 1.013 4 149 1.275 1.180 1.244 1.131 5 159 1.535 1.390 1.510 1.262 6 140 1.764 1.600 1.760 1.409 7 96 2.052 1.811 1.994 1.572 8 65 2.231 2.021 2.212 1.755 9 50 2.360 2.231 2.414 1.959 10 33 2.758 2.441 2.600 2.187 11 18 2.611 2.651 2.771 2.441 12 14 2.929 2.861 2.925 2.724 13 5 2.600 3.071 3.064 3.041 14 7 3.286 3.282 3.186 3.394 15 1 4.000 3.492 3.293 3.788 16 1 3.000 3.702 3.384 4.229 17 0 NA NA NA NA 18 0 NA NA NA NA 19 0 NA NA NA NA 20 0 NA NA NA NA Number-Incorrect Score for Examinee 287 : 8 Number-Incorrect Score for Examinee 869 : 4 Binomial Probability of Matching on an Identical Incorrect Response with Source Examinee for Number-Incorrect Subgroup 8 : for K1 index: 2.021 / 4 = 0.505 for K2 index: 2.212 / 4 = 0.553 Predicted Average Number of Identical Incorrect Response with Source Examinee for Number-Incorrect Subgroup 8 (parameter to be used for S1 index in Poisson Distribution): 1.755 Probability of Observing 4 or More Identical Incorrect Matches Using Binomial Distribution --- K1 index p value: 0.065 --- K2 index p value: 0.093 Probability of Observing 4 or More Identical Incorrect Matches Using Poisson Distribution --- S1 index p value: 0.068 *************************** S2 Index ******************************* Number Incorrect Number Observed Average Number of Observed Average Weighted Number of Score Group of Examinees Identical Incorrect Matches Identical Correct Matches with Source Examinee with Source Examinee 0 11 0.000 0.003 1 21 0.381 0.005 2 82 0.585 0.009 3 147 0.986 0.016 4 149 1.275 0.032 5 159 1.535 0.040 6 140 1.764 0.050 7 96 2.052 0.066 8 65 2.231 0.092 9 50 2.360 0.107 10 33 2.758 0.120 11 18 2.611 0.202 12 14 2.929 0.239 13 5 2.600 0.188 14 7 3.286 0.374 15 1 4.000 0.001 16 1 3.000 0.001 17 0 NA NA 18 0 NA NA 19 0 NA NA 20 0 NA NA Predicted Total Number of Incorrect and Weighted Correct Matches Using Loglinear Model: Number Incorrect Predicted Score Group Values 0 1.608 1 1.727 2 1.855 3 1.992 4 2.139 5 2.297 6 2.466 7 2.649 8 2.844 9 3.054 10 3.280 11 3.522 12 3.782 13 4.062 14 4.362 15 4.684 16 5.030 17 NA 18 NA 19 NA 20 NA Predicted Average Number of Identical Response with Source Examinee for Number-Incorrect Subgroup 8 : 3 (rounded to the nearest integer) S2 index p value: 0.159 *************************** M4 Index ******************************* The detailed results for the M4 index is not printed here due to a large size of matrix for the exact probability distribution. Please type 'x$M4.index$exact.prob.dist' where 'x' should be replaced with the object name you assign for the results of the function. M4 index p value: 0.046 > > > ## No test: > > # Computing for multiple pairs > > pairs <- matrix(as.character(sample(subset$EID,20)),nrow=10,ncol=2) > > a <- similarity1(data = subset, + model = "1PL", + person.id = "EID", + item.loc = 3:20, + many.pairs = pairs) Iteration: 1, Log-Lik: -9715.566, Max-Change: 0.17315 Iteration: 2, Log-Lik: -9665.199, Max-Change: 0.08029 Iteration: 3, Log-Lik: -9652.656, Max-Change: 0.04074 Iteration: 4, Log-Lik: -9649.290, Max-Change: 0.02024 Iteration: 5, Log-Lik: -9648.432, Max-Change: 0.01022 Iteration: 6, Log-Lik: -9648.209, Max-Change: 0.00539 Iteration: 7, Log-Lik: -9648.142, Max-Change: 0.00224 Iteration: 8, Log-Lik: -9648.131, Max-Change: 0.00116 Iteration: 9, Log-Lik: -9648.128, Max-Change: 0.00060 Iteration: 10, Log-Lik: -9648.127, Max-Change: 0.00029 Iteration: 11, Log-Lik: -9648.127, Max-Change: 0.00016 Iteration: 12, Log-Lik: -9648.127, Max-Change: 0.00008 > > print(a) ************************************************************************ CopyDetect - An R Package to Compute Response Similarity Indices for Multiple-Choice Tests Version 1.3, released on October 2018 Cengiz Zopluoglu Assistant Professor University of Miami - Department of Educational and Psychological Studies Research, Measurement, and Evaluation Program c.zopluoglu@miami.edu ************************************************************************* Processing Date: Sun Apr 10 09:34:44 2022 Probability Values Obtained from Various Response Similarity Indices Error in Math.data.frame(list(W.pvalue = c(0.432825512711739, 0.079195249651785, : non-numeric-alike variable(s) in data frame: K2.pvalue, S1.pvalue, S2.pvalue Calls: print -> print.CopyDetectMany -> Math.data.frame Execution halted * checking PDF version of manual ... OK * checking for non-standard things in the check directory ... OK * checking for detritus in the temp directory ... OK * checking for new files in some other directories ... OK * DONE Status: 1 ERROR See ‘/data/blackswan/ripley/R/packages/tests-devel/CopyDetect.Rcheck/00check.log’ for details. Command exited with non-zero status 1 Time 1:56.30, 102.53 + 9.83