Orange: Test and Score - Revision history

Onnowpurbo: /* Output */

2020-04-08T04:07:48Z

Output

Onnowpurbo at 04:04, 8 April 2020

2020-04-08T04:04:12Z

Onnowpurbo: /* Contoh */

2020-04-08T04:03:02Z

Contoh

Onnowpurbo: /* Contoh */

2020-04-08T04:02:52Z

Contoh

Onnowpurbo: /* Contoh */

2020-02-27T02:14:22Z

Contoh

Onnowpurbo at 02:13, 27 February 2020

2020-02-27T02:13:43Z

Onnowpurbo at 00:54, 29 January 2020

2020-01-29T00:54:21Z

Onnowpurbo at 05:46, 23 January 2020

2020-01-23T05:46:59Z

Onnowpurbo: Created page with "Sumber: https://docs.biolab.si//3/visual-programming/widgets/evaluate/testandscore.html Tests learning algorithms on data. Inputs Data: input dataset Test Data:..."

2020-01-16T23:46:18Z

Created page with "Sumber: https://docs.biolab.si//3/visual-programming/widgets/evaluate/testandscore.html Tests learning algorithms on data. Inputs Data: input dataset Test Data:..."

New page

Sumber: https://docs.biolab.si//3/visual-programming/widgets/evaluate/testandscore.html

Tests learning algorithms on data.

Inputs

Data: input dataset

Test Data: separate data for testing

Learner: learning algorithm(s)

Outputs

Evaluation Results: results of testing classification algorithms

The widget tests learning algorithms. Different sampling schemes are available, including using separate test data. The widget does two things. First, it shows a table with different classifier performance measures, such as classification accuracy and area under the curve. Second, it outputs evaluation results, which can be used by other widgets for analyzing the performance of classifiers, such as ROC Analysis or Confusion Matrix.

The Learner signal has an uncommon property: it can be connected to more than one widget to test multiple learners with the same procedures.

../../_images/TestLearners-stamped.png

The widget supports various sampling methods.

Cross-validation splits the data into a given number of folds (usually 5 or 10). The algorithm is tested by holding out examples from one fold at a time; the model is induced from other folds and examples from the held out fold are classified. This is repeated for all the folds.

Leave-one-out is similar, but it holds out one instance at a time, inducing the model from all others and then classifying the held out instances. This method is obviously very stable, reliable… and very slow.

Random sampling randomly splits the data into the training and testing set in the given proportion (e.g. 70:30); the whole procedure is repeated for a specified number of times.

Test on train data uses the whole dataset for training and then for testing. This method practically always gives wrong results.

Test on test data: the above methods use the data from Data signal only. To input another dataset with testing examples (for instance from another file or some data selected in another widget), we select Separate Test Data signal in the communication channel and select Test on test data.

For classification, Target class can be selected at the bottom of the widget. When Target class is (Average over classes), methods return scores that are weighted averages over all classes. For example, in case of the classifier with 3 classes, scores are computed for class 1 as a target class, class 2 as a target class, and class 3 as a target class. Those scores are averaged with weights based on the class size to retrieve the final score.

Produce a report.

The widget will compute a number of performance statistics:

Classification ../../_images/TestLearners.png

Area under ROC is the area under the receiver-operating curve.

Classification accuracy is the proportion of correctly classified examples.

F-1 is a weighted harmonic mean of precision and recall (see below).

Precision is the proportion of true positives among instances classified as positive, e.g. the proportion of Iris virginica correctly identified as Iris virginica.

Recall is the proportion of true positives among all positive instances in the data, e.g. the number of sick among all diagnosed as sick.

Regression ../../_images/TestLearners-regression.png

MSE measures the average of the squares of the errors or deviations (the difference between the estimator and what is estimated).

RMSE is the square root of the arithmetic mean of the squares of a set of numbers (a measure of imperfection of the fit of the estimator to the data)

MAE is used to measure how close forecasts or predictions are to eventual outcomes.

R2 is interpreted as the proportion of the variance in the dependent variable that is predictable from the independent variable.

Example

In a typical use of the widget, we give it a dataset and a few learning algorithms and we observe their performance in the table inside the Test & Score widget and in the ROC. The data is often preprocessed before testing; in this case we did some manual feature selection (Select Columns widget) on Titanic dataset, where we want to know only the sex and status of the survived and omit the age.

../../_images/TestLearners-example-classification.png

Another example of using this widget is presented in the documentation for the Confusion Matrix widget.

==Referensi==

* https://docs.biolab.si//3/visual-programming/widgets/evaluate/testandscore.html

==Pranala Menarik==

* [[Orange]]

@@ Line 19: / Line 19: @@
 Signal Learner mempunyai property uncommon: ini bisa di sambungkan ke lebih dari satu widget untuk men-test banyak learner dengan prosedur yang sama.
-[[File:TestLearners-stamped.png|center|200px|thumb]]
+[[File:TestLearners-stamped.png|center|600px|thumb]]
 * The widget supports various sampling methods.

@@ Line 3: / Line 3: @@
-Test algoritma learning pada data.
+Widget Test & Score men-test algoritma learning pada data.
 ==Input==
@@ Line 15: / Line 15: @@
   Evaluation Results: results of testing classification algorithms
-Widget Test & Score akan menguji algoritma learning. Skema pengambilan sampel yang berbeda tersedia, termasuk menggunakan test data yang terpisah. Widget melakukan dua hal. Pertama, widget akan menunjukkan tabel dengan ukuran kinerja classifier yang berbeda, seperti akurasi klasifikasi dan area di bawah kurva. Kedua, output hasil evaluasi, yang dapat digunakan oleh widget lain untuk menganalisis kinerja pengklasifikasi, seperti ROC Analysis atau Confusion Matrix.
+Widget Test & Score akan menguji algoritma learning. Skema pengambilan sampel yang berbeda tersedia, termasuk menggunakan test data yang terpisah. Widget Test & Score melakukan dua hal. Pertama, widget Test & Score akan menunjukkan tabel dengan ukuran kinerja classifier yang berbeda, seperti akurasi klasifikasi dan area di bawah kurva. Kedua, output hasil evaluasi, yang dapat digunakan oleh widget lain untuk menganalisis kinerja pengklasifikasi, seperti widget ROC Analysis atau widget Confusion Matrix.
 Signal Learner mempunyai property uncommon: ini bisa di sambungkan ke lebih dari satu widget untuk men-test banyak learner dengan prosedur yang sama.

@@ Line 55: / Line 55: @@
 Dalam penggunaan widget Test & Score yang biasa, kita akan memasukan set data dan beberapa algoritma learning dan kita mengamati kinerjanya dalam tabel di dalam widget Test & Score dan di widget ROC Analysis. Data sering diproses sebelumnya sebelum pengujian; dalam hal ini kami melakukan beberapa pilihan feature manual (widget Select Columns) pada dataset Titanic, di mana kita hanya ingin mengetahui jenis kelamin dan status orang yang selamat dan mengabaikan age (usia).
-[[File:TestLearners-example-classification.png|center|200px|thumb]]
+[[File:TestLearners-example-classification.png|center|600px|thumb]]
 Contoh lain penggunaan widget ini di jelaskan di dokumentasi [[Orange: Confusion Matrix | Confusion Matrix widget]].

@@ Line 53: / Line 53: @@
 ==Contoh==
-Dalam penggunaan widget yang tipikal, kita akan memasukan set data dan beberapa algoritma learning dan kita mengamati kinerjanya dalam tabel di dalam widget Test & Score dan di ROC. Data sering diproses sebelumnya sebelum pengujian; dalam hal ini kami melakukan beberapa pilihan feature manual (Select Columns widget) pada dataset Titanic, di mana kita hanya ingin mengetahui jenis kelamin dan status orang yang selamat dan mengabaikan age (usia).
+Dalam penggunaan widget Test & Score yang biasa, kita akan memasukan set data dan beberapa algoritma learning dan kita mengamati kinerjanya dalam tabel di dalam widget Test & Score dan di widget ROC Analysis. Data sering diproses sebelumnya sebelum pengujian; dalam hal ini kami melakukan beberapa pilihan feature manual (widget Select Columns) pada dataset Titanic, di mana kita hanya ingin mengetahui jenis kelamin dan status orang yang selamat dan mengabaikan age (usia).
 [[File:TestLearners-example-classification.png|center|200px|thumb]]

@@ Line 57: / Line 57: @@
 [[File:TestLearners-example-classification.png|center|200px|thumb]]
-Contoh lain penggunaan widget ini di jelaskan di dokumentasi Confusion Matrix widget.
+Contoh lain penggunaan widget ini di jelaskan di dokumentasi [[Orange: Confusion Matrix | Confusion Matrix widget]].
 ==Referensi==

@@ Line 5: / Line 5: @@
 Tests learning algorithms on data.
-Inputs
+==Input==
-    Data: input dataset
+ Data: input dataset
-    Test Data: separate data for testing
+ Test Data: separate data for testing
-    Learner: learning algorithm(s)
+ Learner: learning algorithm(s)
-Outputs
+==Output==
-    Evaluation Results: results of testing classification algorithms
+ Evaluation Results: results of testing classification algorithms
 The widget tests learning algorithms. Different sampling schemes are available, including using separate test data. The widget does two things. First, it shows a table with different classifier performance measures, such as classification accuracy and area under the curve. Second, it outputs evaluation results, which can be used by other widgets for analyzing the performance of classifiers, such as ROC Analysis or Confusion Matrix.
@@ Line 21: / Line 21: @@
 [[File:TestLearners-stamped.png|center|200px|thumb]]
-    The widget supports various sampling methods.
+* The widget supports various sampling methods.
-        Cross-validation splits the data into a given number of folds (usually 5 or 10). The algorithm is tested by holding out examples from one fold at a time; the model is induced from other folds and examples from the held out fold are classified. This is repeated for all the folds.
+** Cross-validation splits the data into a given number of folds (usually 5 or 10). The algorithm is tested by holding out examples from one fold at a time; the model is induced from other folds and examples from the held out fold are classified. This is repeated for all the folds.
-        Leave-one-out is similar, but it holds out one instance at a time, inducing the model from all others and then classifying the held out instances. This method is obviously very stable, reliable… and very slow.
+** Leave-one-out is similar, but it holds out one instance at a time, inducing the model from all others and then classifying the held out instances. This method is obviously very stable, reliable… and very slow.
-        Random sampling randomly splits the data into the training and testing set in the given proportion (e.g. 70:30); the whole procedure is repeated for a specified number of times.
+** Random sampling randomly splits the data into the training and testing set in the given proportion (e.g. 70:30); the whole procedure is repeated for a specified number of times.
-        Test on train data uses the whole dataset for training and then for testing. This method practically always gives wrong results.
+** Test on train data uses the whole dataset for training and then for testing. This method practically always gives wrong results.
-        Test on test data: the above methods use the data from Data signal only. To input another dataset with testing examples (for instance from another file or some data selected in another widget), we select Separate Test Data signal in the communication channel and select Test on test data.
+** Test on test data: the above methods use the data from Data signal only. To input another dataset with testing examples (for instance from another file or some data selected in another widget), we select Separate Test Data signal in the communication channel and select Test on test data.
-    For classification, Target class can be selected at the bottom of the widget. When Target class is (Average over classes), methods return scores that are weighted averages over all classes. For example, in case of the classifier with 3 classes, scores are computed for class 1 as a target class, class 2 as a target class, and class 3 as a target class. Those scores are averaged with weights based on the class size to retrieve the final score.
+* For classification, Target class can be selected at the bottom of the widget. When Target class is (Average over classes), methods return scores that are weighted averages over all classes. For example, in case of the classifier with 3 classes, scores are computed for class 1 as a target class, class 2 as a target class, and class 3 as a target class. Those scores are averaged with weights based on the class size to retrieve the final score.
-    Produce a report.
+* Produce a report.
-    The widget will compute a number of performance statistics:
-        Classification
+* The widget will compute a number of performance statistics:
 [[File:TestLearners.png|center|200px|thumb]]
-            Area under ROC is the area under the receiver-operating curve.
+*** Area under ROC is the area under the receiver-operating curve.
-            Classification accuracy is the proportion of correctly classified examples.
+*** Classification accuracy is the proportion of correctly classified examples.
-            F-1 is a weighted harmonic mean of precision and recall (see below).
+*** F-1 is a weighted harmonic mean of precision and recall (see below).
-            Precision is the proportion of true positives among instances classified as positive, e.g. the proportion of Iris virginica correctly identified as Iris virginica.
+*** Precision is the proportion of true positives among instances classified as positive, e.g. the proportion of Iris virginica correctly identified as Iris virginica.
-            Recall is the proportion of true positives among all positive instances in the data, e.g. the number of sick among all diagnosed as sick.
+*** Recall is the proportion of true positives among all positive instances in the data, e.g. the number of sick among all diagnosed as sick.
-        Regression
 [[File:TestLearners-regression.png|center|200px|thumb]]
-            MSE measures the average of the squares of the errors or deviations (the difference between the estimator and what is estimated).
+*** MSE measures the average of the squares of the errors or deviations (the difference between the estimator and what is estimated).
-            RMSE is the square root of the arithmetic mean of the squares of a set of numbers (a measure of imperfection of the fit of the estimator to the data)
+*** RMSE is the square root of the arithmetic mean of the squares of a set of numbers (a measure of imperfection of the fit of the estimator to the data)
-            MAE is used to measure how close forecasts or predictions are to eventual outcomes.
+*** MAE is used to measure how close forecasts or predictions are to eventual outcomes.
-            R2 is interpreted as the proportion of the variance in the dependent variable that is predictable from the independent variable.
+*** R2 is interpreted as the proportion of the variance in the dependent variable that is predictable from the independent variable.
 ==Contoh==