Orange: ROC Analysis - Revision history

Onnowpurbo: /* Contoh */

2020-04-08T03:35:21Z

Contoh

← Older revision		Revision as of 03:35, 8 April 2020
Line 44:		Line 44:
	==Contoh==		==Contoh==

	Saat ini, satu-satunya widget yang memberikan jenis sinyal yang tepat yang dibutuhkan oleh widget ROC Analysis adalah widget Test & Score. Di bawah ini, kita membandingkan dua classifier, yaitu widget Tree dan widget Naive Bayes, dalam widget Test&Score dan kemudian membandingkan kinerjanya dalam widget ROC Analysis, widget Life Curve dan widget Calibration Plot.		Saat ini, satu-satunya widget yang memberikan jenis sinyal yang tepat yang dibutuhkan oleh widget ROC Analysis adalah widget Test & Score.

			Di bawah ini, kita membandingkan dua classifier, yaitu widget Tree dan widget Naive Bayes, dalam widget Test&Score dan kemudian membandingkan kinerjanya dalam widget ROC Analysis, widget Life Curve dan widget Calibration Plot.

	[[File:ROCAnalysis-example.png\|center\|600px\|thumb]]		[[File:ROCAnalysis-example.png\|center\|600px\|thumb]]

Onnowpurbo: /* Contoh */

2020-04-08T03:30:28Z

Contoh

← Older revision		Revision as of 03:30, 8 April 2020
Line 44:		Line 44:
	==Contoh==		==Contoh==

	Saat ini, satu-satunya widget yang memberikan jenis sinyal yang tepat yang dibutuhkan oleh ROC Analysis adalah Test & Score. Di bawah ini, kita membandingkan dua classifier, yaitu Tree dan Naive Bayes, dalam Test&Score dan kemudian membandingkan kinerjanya dalam ROC		Saat ini, satu-satunya widget yang memberikan jenis sinyal yang tepat yang dibutuhkan oleh widget ROC Analysis adalah widget Test & Score. Di bawah ini, kita membandingkan dua classifier, yaitu widget Tree dan widget Naive Bayes, dalam widget Test&Score dan kemudian membandingkan kinerjanya dalam widget ROC Analysis, widget Life Curve dan widget Calibration Plot.
	Analysis, Life Curve dan Calibration Plot.

	[[File:ROCAnalysis-example.png\|center\|~~200px~~\|thumb]]		[[File:ROCAnalysis-example.png\|center\|600px\|thumb]]

	==Referensi==		==Referensi==

Onnowpurbo: /* Input */

2020-04-08T03:28:23Z

Input

← Older revision		Revision as of 03:28, 8 April 2020
Line 9:		Line 9:
	Widget ROC Analysis menunjukkan ROC curve untuk model yang diuji dan kecembungan yang sesuai. Ini berfungsi sebagai rata-rata perbandingan antara classification model. Kurva memplot false positive rate pada sumbu x (1-spesifisitas; probabilitas target = 1 ketika nilai true = 0) terhadap true positive rate pada sumbu y (sensitivitas; probabilitas target itu = 1 ketika nilai sebenarnya = 1). Semakin dekat kurva mengikuti batas kiri dan kemudian batas atas ruang ROC, semakin akurat classifier tersebut. Melalui cost dari false positive dan false negative, widget ROC Analysis juga dapat menentukan classifer dan threshold yang optimal.		Widget ROC Analysis menunjukkan ROC curve untuk model yang diuji dan kecembungan yang sesuai. Ini berfungsi sebagai rata-rata perbandingan antara classification model. Kurva memplot false positive rate pada sumbu x (1-spesifisitas; probabilitas target = 1 ketika nilai true = 0) terhadap true positive rate pada sumbu y (sensitivitas; probabilitas target itu = 1 ketika nilai sebenarnya = 1). Semakin dekat kurva mengikuti batas kiri dan kemudian batas atas ruang ROC, semakin akurat classifier tersebut. Melalui cost dari false positive dan false negative, widget ROC Analysis juga dapat menentukan classifer dan threshold yang optimal.

	[[File:ROCAnalysis-basic-stamped.png\|center\|~~200px~~\|thumb]]		[[File:ROCAnalysis-basic-stamped.png\|center\|600px\|thumb]]

	* Choose the desired Target Class. The default class is chosen alphabetically.		* Choose the desired Target Class. The default class is chosen alphabetically.

Onnowpurbo: /* Input */

2020-04-08T03:28:15Z

Input

← Older revision		Revision as of 03:28, 8 April 2020
Line 7:		Line 7:
	Evaluation Results: results of testing classification algorithms		Evaluation Results: results of testing classification algorithms

	Widget ROC Analysis menunjukkan ROC curve untuk model yang diuji dan kecembungan yang sesuai. Ini berfungsi sebagai rata-rata perbandingan antara classification model. Kurva memplot false positive rate pada sumbu x (1-spesifisitas; probabilitas target = 1 ketika nilai true = 0) terhadap true positive rate pada sumbu y (sensitivitas; probabilitas target itu = 1 ketika nilai sebenarnya = 1). Semakin dekat kurva mengikuti batas kiri dan kemudian batas atas ruang ROC, semakin akurat classifier tersebut. Melalui cost dari false positive dan false negative, widget juga dapat menentukan classifer dan threshold yang optimal.		Widget ROC Analysis menunjukkan ROC curve untuk model yang diuji dan kecembungan yang sesuai. Ini berfungsi sebagai rata-rata perbandingan antara classification model. Kurva memplot false positive rate pada sumbu x (1-spesifisitas; probabilitas target = 1 ketika nilai true = 0) terhadap true positive rate pada sumbu y (sensitivitas; probabilitas target itu = 1 ketika nilai sebenarnya = 1). Semakin dekat kurva mengikuti batas kiri dan kemudian batas atas ruang ROC, semakin akurat classifier tersebut. Melalui cost dari false positive dan false negative, widget ROC Analysis juga dapat menentukan classifer dan threshold yang optimal.

	[[File:ROCAnalysis-basic-stamped.png\|center\|200px\|thumb]]		[[File:ROCAnalysis-basic-stamped.png\|center\|200px\|thumb]]

Onnowpurbo at 03:26, 8 April 2020

2020-04-08T03:26:53Z

← Older revision		Revision as of 03:26, 8 April 2020
Line 1:		Line 1:
	Sumber: https://docs.biolab.si//3/visual-programming/widgets/evaluate/rocanalysis.html		Sumber: https://docs.biolab.si//3/visual-programming/widgets/evaluate/rocanalysis.html

	~~Plot~~ true positive rate terhadap false positive rate di ~~sebuahtest~~.		Widget ROC Analysis mem-plot true positive rate terhadap false positive rate di sebuah test.

	==Input==		==Input==
Line 7:		Line 7:
	Evaluation Results: results of testing classification algorithms		Evaluation Results: results of testing classification algorithms

	Widget menunjukkan ROC curve untuk model yang diuji dan kecembungan yang sesuai. Ini berfungsi sebagai rata-rata perbandingan antara classification model. Kurva ~~memplotfalse~~ positive rate pada sumbu x (1-spesifisitas; probabilitas target = 1 ketika nilai true = 0) terhadap true positive rate pada sumbu y (sensitivitas; probabilitas target itu = 1 ketika nilai sebenarnya = 1). Semakin dekat kurva mengikuti batas kiri dan kemudian batas atas ruang ROC, semakin akurat classifier tersebut. Melalui cost dari false positive dan false negative, widget juga dapat menentukan classifer dan threshold yang optimal.		Widget ROC Analysis menunjukkan ROC curve untuk model yang diuji dan kecembungan yang sesuai. Ini berfungsi sebagai rata-rata perbandingan antara classification model. Kurva memplot false positive rate pada sumbu x (1-spesifisitas; probabilitas target = 1 ketika nilai true = 0) terhadap true positive rate pada sumbu y (sensitivitas; probabilitas target itu = 1 ketika nilai sebenarnya = 1). Semakin dekat kurva mengikuti batas kiri dan kemudian batas atas ruang ROC, semakin akurat classifier tersebut. Melalui cost dari false positive dan false negative, widget juga dapat menentukan classifer dan threshold yang optimal.

	[[File:ROCAnalysis-basic-stamped.png\|center\|200px\|thumb]]		[[File:ROCAnalysis-basic-stamped.png\|center\|200px\|thumb]]

Onnowpurbo at 00:10, 10 March 2020

2020-03-10T00:10:23Z

← Older revision		Revision as of 00:10, 10 March 2020
Line 1:		Line 1:
	Sumber: https://docs.biolab.si//3/visual-programming/widgets/evaluate/rocanalysis.html		Sumber: https://docs.biolab.si//3/visual-programming/widgets/evaluate/rocanalysis.html

	~~Plots a~~ true positive rate ~~against a~~ false positive rate ~~of a test~~.		Plot true positive rate terhadap false positive rate di sebuahtest.

	==Input==		==Input==
Line 7:		Line 7:
	Evaluation Results: results of testing classification algorithms		Evaluation Results: results of testing classification algorithms

	~~The widget shows~~ ROC ~~curves for the tested models and the corresponding convex hull~~. ~~It serves as a mean of comparison between~~ classification ~~models~~. ~~The curve plots a false~~ positive rate ~~on an~~ x~~-axis~~ (1-~~specificity~~; ~~probability that~~ target=1 ~~when~~ true ~~value~~=0) ~~against a~~ true positive rate ~~on a~~ y~~-axis~~ (~~sensitivity~~; ~~probability that~~ target=1 ~~when true value~~=1). ~~The closer the curve follows the left-hand border and then the top border of the~~ ROC ~~space~~, ~~the more accurate the~~ classifier. ~~Given the costs of~~ false ~~positives and~~ false ~~negatives~~, ~~the~~ widget ~~can also determine the~~ optimal ~~classifier and threshold~~.		Widget menunjukkan ROC curve untuk model yang diuji dan kecembungan yang sesuai. Ini berfungsi sebagai rata-rata perbandingan antara classification model. Kurva memplotfalse positive rate pada sumbu x (1-spesifisitas; probabilitas target = 1 ketika nilai true = 0) terhadap true positive rate pada sumbu y (sensitivitas; probabilitas target itu = 1 ketika nilai sebenarnya = 1). Semakin dekat kurva mengikuti batas kiri dan kemudian batas atas ruang ROC, semakin akurat classifier tersebut. Melalui cost dari false positive dan false negative, widget juga dapat menentukan classifer dan threshold yang optimal.

	[[File:ROCAnalysis-basic-stamped.png\|center\|200px\|thumb]]		[[File:ROCAnalysis-basic-stamped.png\|center\|200px\|thumb]]
Line 26:		Line 26:
	[[File:ROCAnalysis-AUC.png\|center\|200px\|thumb]]		[[File:ROCAnalysis-AUC.png\|center\|200px\|thumb]]

	~~The~~ diagonal ~~dotted line represents the behavior of a~~ random classifier. ~~The full~~ diagonal ~~line represents iso~~-performance. ~~A black~~ “A” ~~symbol at the bottom of the~~ graph ~~proportionally readjusts the~~ graph.		Garis putus-putus diagonal mewakili perilaku random classifier. Garis diagonal penuh mewakiliiso-performance. Simbol “A” hitam di bagian bawah graph secara proporsional menyesuaikan graph.

	* The final box is dedicated to the analysis of the curve. The user can specify the cost of false positives (FP) and false negatives (FN), and the prior target class probability.		* The final box is dedicated to the analysis of the curve. The user can specify the cost of false positives (FP) and false negatives (FN), and the prior target class probability.
Line 44:		Line 44:
	==Contoh==		==Contoh==

	~~At the moment~~, ~~the only~~ widget ~~which gives the right type of signal needed by the~~ ROC Analysis is Test & Score. ~~Below~~, ~~we compare two classifiers~~, ~~namely~~ Tree ~~and~~ Naive Bayes, in Test&Score ~~and then compare their performance in~~ ROC Analysis, Life Curve ~~and~~ Calibration Plot.		Saat ini, satu-satunya widget yang memberikan jenis sinyal yang tepat yang dibutuhkan oleh ROC Analysis adalah Test & Score. Di bawah ini, kita membandingkan dua classifier, yaitu Tree dan Naive Bayes, dalam Test&Score dan kemudian membandingkan kinerjanya dalam ROC
			Analysis, Life Curve dan Calibration Plot.

	[[File:ROCAnalysis-example.png\|center\|200px\|thumb]]		[[File:ROCAnalysis-example.png\|center\|200px\|thumb]]

Onnowpurbo at 00:50, 29 January 2020

2020-01-29T00:50:28Z

← Older revision		Revision as of 00:50, 29 January 2020
Line 3:		Line 3:
	Plots a true positive rate against a false positive rate of a test.		Plots a true positive rate against a false positive rate of a test.

	~~Inputs~~		==Input==

	Evaluation Results: results of testing classification algorithms		Evaluation Results: results of testing classification algorithms

	The widget shows ROC curves for the tested models and the corresponding convex hull. It serves as a mean of comparison between classification models. The curve plots a false positive rate on an x-axis (1-specificity; probability that target=1 when true value=0) against a true positive rate on a y-axis (sensitivity; probability that target=1 when true value=1). The closer the curve follows the left-hand border and then the top border of the ROC space, the more accurate the classifier. Given the costs of false positives and false negatives, the widget can also determine the optimal classifier and threshold.		The widget shows ROC curves for the tested models and the corresponding convex hull. It serves as a mean of comparison between classification models. The curve plots a false positive rate on an x-axis (1-specificity; probability that target=1 when true value=0) against a true positive rate on a y-axis (sensitivity; probability that target=1 when true value=1). The closer the curve follows the left-hand border and then the top border of the ROC space, the more accurate the classifier. Given the costs of false positives and false negatives, the widget can also determine the optimal classifier and threshold.
Line 11:		Line 11:
	[[File:ROCAnalysis-basic-stamped.png\|center\|200px\|thumb]]		[[File:ROCAnalysis-basic-stamped.png\|center\|200px\|thumb]]

	Choose the desired Target Class. The default class is chosen alphabetically.		* Choose the desired Target Class. The default class is chosen alphabetically.
	If test results contain more than one classifier, the user can choose which curves she or he wants to see plotted. Click on a classifier to select or deselect it.		* If test results contain more than one classifier, the user can choose which curves she or he wants to see plotted. Click on a classifier to select or deselect it.
	When the data comes from multiple iterations of training and testing, such as k-fold cross validation, the results can be (and usually are) averaged.		* When the data comes from multiple iterations of training and testing, such as k-fold cross validation, the results can be (and usually are) averaged.

	[[File:ROC-Comparison.png\|center\|200px\|thumb]]		[[File:ROC-Comparison.png\|center\|200px\|thumb]]

	The averaging options are:		* The averaging options are:
	Merge predictions from folds (top left), which treats all the test data as if they came from a single iteration		** Merge predictions from folds (top left), which treats all the test data as if they came from a single iteration
	Mean TP rate (top right) averages the curves vertically, showing the corresponding confidence intervals		** Mean TP rate (top right) averages the curves vertically, showing the corresponding confidence intervals
	Mean TP and FP at threshold (bottom left) traverses over threshold, averages the positions of curves and shows horizontal and vertical confidence intervals		** Mean TP and FP at threshold (bottom left) traverses over threshold, averages the positions of curves and shows horizontal and vertical confidence intervals
	Show individual curves (bottom right) does not average but prints all the curves instead		** Show individual curves (bottom right) does not average but prints all the curves instead
	Option Show convex ROC curves refers to convex curves over each individual classifier (the thin lines positioned over curves). Show ROC convex hull plots a convex hull combining all classifiers (the gray area below the curves). Plotting both types of convex curves makes sense since selecting a threshold in a concave part of the curve cannot yield optimal results, disregarding the cost matrix. Besides, it is possible to reach any point on the convex curve by combining the classifiers represented by the points on the border of the concave region.		* Option Show convex ROC curves refers to convex curves over each individual classifier (the thin lines positioned over curves). Show ROC convex hull plots a convex hull combining all classifiers (the gray area below the curves). Plotting both types of convex curves makes sense since selecting a threshold in a concave part of the curve cannot yield optimal results, disregarding the cost matrix. Besides, it is possible to reach any point on the convex curve by combining the classifiers represented by the points on the border of the concave region.

	[[File:ROCAnalysis-AUC.png\|center\|200px\|thumb]]		[[File:ROCAnalysis-AUC.png\|center\|200px\|thumb]]

	The diagonal dotted line represents the behavior of a random classifier. The full diagonal line represents iso-performance. A black “A” symbol at the bottom of the graph proportionally readjusts the graph.		The diagonal dotted line represents the behavior of a random classifier. The full diagonal line represents iso-performance. A black “A” symbol at the bottom of the graph proportionally readjusts the graph.
	The final box is dedicated to the analysis of the curve. The user can specify the cost of false positives (FP) and false negatives (FN), and the prior target class probability.
	Default threshold (0.5) point shows the point on the ROC curve achieved by the classifier if it predicts the target class if its probability equals or exceeds 0.5.		* The final box is dedicated to the analysis of the curve. The user can specify the cost of false positives (FP) and false negatives (FN), and the prior target class probability.
	Show performance line shows iso-performance in the ROC space so that all the points on the line give the same profit/loss. The line further to the upper left is better than the one down and right. The direction of the line depends upon costs and probabilities. This gives a recipe for depicting the optimal threshold for the given costs: this is the point where the tangent with the given inclination touches the curve and it is marked in the plot. If we push the iso-performance higher or more to the left, the points on the iso-performance line cannot be reached by the learner. Going down or to the right, decreases the performance.		** Default threshold (0.5) point shows the point on the ROC curve achieved by the classifier if it predicts the target class if its probability equals or exceeds 0.5.
	The widget allows setting the costs from 1 to 1000. Units are not important, as are not the magnitudes. What matters is the relation between the two costs, so setting them to 100 and 200 will give the same result as 400 and 800. Defaults: both costs equal (500), Prior target class probability 50%(from the data).		** Show performance line shows iso-performance in the ROC space so that all the points on the line give the same profit/loss. The line further to the upper left is better than the one down and right. The direction of the line depends upon costs and probabilities. This gives a recipe for depicting the optimal threshold for the given costs: this is the point where the tangent with the given inclination touches the curve and it is marked in the plot. If we push the iso-performance higher or more to the left, the points on the iso-performance line cannot be reached by the learner. Going down or to the right, decreases the performance.
			** The widget allows setting the costs from 1 to 1000. Units are not important, as are not the magnitudes. What matters is the relation between the two costs, so setting them to 100 and 200 will give the same result as 400 and 800. Defaults: both costs equal (500), Prior target class probability 50%(from the data).

	[[File:ROCAnalysis-Plain.png\|center\|200px\|thumb]]		[[File:ROCAnalysis-Plain.png\|center\|200px\|thumb]]

	False positive cost: 830, False negative cost 650, Prior target class probability 73%.		* False positive cost: 830, False negative cost 650, Prior target class probability 73%.

	[[File:ROCAnalysis.png\|center\|200px\|thumb]]		[[File:ROCAnalysis.png\|center\|200px\|thumb]]

			* Press Save Image if you want to save the created image to your computer in a .svg or .png format.
	Press Save Image if you want to save the created image to your computer in a .svg or .png format.		* Produce a report.
	Produce a report.

	==Contoh==		==Contoh==

Onnowpurbo at 05:41, 23 January 2020

2020-01-23T05:41:44Z

← Older revision		Revision as of 05:41, 23 January 2020
Line 9:		Line 9:
	The widget shows ROC curves for the tested models and the corresponding convex hull. It serves as a mean of comparison between classification models. The curve plots a false positive rate on an x-axis (1-specificity; probability that target=1 when true value=0) against a true positive rate on a y-axis (sensitivity; probability that target=1 when true value=1). The closer the curve follows the left-hand border and then the top border of the ROC space, the more accurate the classifier. Given the costs of false positives and false negatives, the widget can also determine the optimal classifier and threshold.		The widget shows ROC curves for the tested models and the corresponding convex hull. It serves as a mean of comparison between classification models. The curve plots a false positive rate on an x-axis (1-specificity; probability that target=1 when true value=0) against a true positive rate on a y-axis (sensitivity; probability that target=1 when true value=1). The closer the curve follows the left-hand border and then the top border of the ROC space, the more accurate the classifier. Given the costs of false positives and false negatives, the widget can also determine the optimal classifier and threshold.

	~~../../_images/~~ROCAnalysis-basic-stamped.png		[[File:ROCAnalysis-basic-stamped.png\|center\|200px\|thumb]]

	Choose the desired Target Class. The default class is chosen alphabetically.		Choose the desired Target Class. The default class is chosen alphabetically.

	If test results contain more than one classifier, the user can choose which curves she or he wants to see plotted. Click on a classifier to select or deselect it.		If test results contain more than one classifier, the user can choose which curves she or he wants to see plotted. Click on a classifier to select or deselect it.
			When the data comes from multiple iterations of training and testing, such as k-fold cross validation, the results can be (and usually are) averaged.

	~~When the data comes from multiple iterations of training and testing, such as k-fold cross validation, the results can be (and usually are) averaged. ../../_images/~~ROC-Comparison.png ~~The averaging options are:~~		[[File:ROC-Comparison.png\|center\|200px\|thumb]]

			The averaging options are:
	Merge predictions from folds (top left), which treats all the test data as if they came from a single iteration		Merge predictions from folds (top left), which treats all the test data as if they came from a single iteration

	Mean TP rate (top right) averages the curves vertically, showing the corresponding confidence intervals		Mean TP rate (top right) averages the curves vertically, showing the corresponding confidence intervals

	Mean TP and FP at threshold (bottom left) traverses over threshold, averages the positions of curves and shows horizontal and vertical confidence intervals		Mean TP and FP at threshold (bottom left) traverses over threshold, averages the positions of curves and shows horizontal and vertical confidence intervals

	Show individual curves (bottom right) does not average but prints all the curves instead		Show individual curves (bottom right) does not average but prints all the curves instead
			Option Show convex ROC curves refers to convex curves over each individual classifier (the thin lines positioned over curves). Show ROC convex hull plots a convex hull combining all classifiers (the gray area below the curves). Plotting both types of convex curves makes sense since selecting a threshold in a concave part of the curve cannot yield optimal results, disregarding the cost matrix. Besides, it is possible to reach any point on the convex curve by combining the classifiers represented by the points on the border of the concave region.

	Option Show convex ROC curves refers to convex curves over each individual classifier (the thin lines positioned over curves). Show ROC convex hull plots a convex hull combining all classifiers (the gray area below the curves). Plotting both types of convex curves makes sense since selecting a threshold in a concave part of the curve cannot yield optimal results, disregarding the cost matrix. Besides, it is possible to reach any point on the convex curve by combining the classifiers represented by the points on the border of the concave region. ../../_images/ROCAnalysis-AUC.png The diagonal dotted line represents the behavior of a random classifier. The full diagonal line represents iso-performance. A black “A” symbol at the bottom of the graph proportionally readjusts the graph.		[[File:ROCAnalysis-AUC.png\|center\|200px\|thumb]]

			The diagonal dotted line represents the behavior of a random classifier. The full diagonal line represents iso-performance. A black “A” symbol at the bottom of the graph proportionally readjusts the graph.
	The final box is dedicated to the analysis of the curve. The user can specify the cost of false positives (FP) and false negatives (FN), and the prior target class probability.		The final box is dedicated to the analysis of the curve. The user can specify the cost of false positives (FP) and false negatives (FN), and the prior target class probability.
			Default threshold (0.5) point shows the point on the ROC curve achieved by the classifier if it predicts the target class if its probability equals or exceeds 0.5.
			Show performance line shows iso-performance in the ROC space so that all the points on the line give the same profit/loss. The line further to the upper left is better than the one down and right. The direction of the line depends upon costs and probabilities. This gives a recipe for depicting the optimal threshold for the given costs: this is the point where the tangent with the given inclination touches the curve and it is marked in the plot. If we push the iso-performance higher or more to the left, the points on the iso-performance line cannot be reached by the learner. Going down or to the right, decreases the performance.
			The widget allows setting the costs from 1 to 1000. Units are not important, as are not the magnitudes. What matters is the relation between the two costs, so setting them to 100 and 200 will give the same result as 400 and 800. Defaults: both costs equal (500), Prior target class probability 50%(from the data).

	~~Default threshold (0~~.~~5) point shows the point on the ROC curve achieved by the classifier if it predicts the~~ target class ~~if its~~ probability ~~equals or exceeds 0.5~~.		[[File:ROCAnalysis-Plain.png\|center\|200px\|thumb]]

			False positive cost: 830, False negative cost 650, Prior target class probability 73%.

	Show performance line shows iso-performance in the ROC space so that all the points on the line give the same profit/loss. The line further to the upper left is better than the one down and right. The direction of the line depends upon costs and probabilities. This gives a recipe for depicting the optimal threshold for the given costs: this is the point where the tangent with the given inclination touches the curve and it is marked in the plot. If we push the iso-performance higher or more to the left, the points on the iso-performance line cannot be reached by the learner. Going down or to the right, decreases the performance.		[[File:ROCAnalysis.png\|center\|200px\|thumb]]

	The widget allows setting the costs from 1 to 1000. Units are not important, as are not the magnitudes. What matters is the relation between the two costs, so setting them to 100 and 200 will give the same result as 400 and 800. Defaults: both costs equal (500), Prior target class probability 50%(from the data). ../../_images/ROCAnalysis-Plain.png False positive cost: 830, False negative cost 650, Prior target class probability 73%. ../../_images/ROCAnalysis.png

	Press Save Image if you want to save the created image to your computer in a .svg or .png format.		Press Save Image if you want to save the created image to your computer in a .svg or .png format.

	Produce a report.		Produce a report.

	~~Example~~		==Contoh==

	At the moment, the only widget which gives the right type of signal needed by the ROC Analysis is Test & Score. Below, we compare two classifiers, namely Tree and Naive Bayes, in Test&Score and then compare their performance in ROC Analysis, Life Curve and Calibration Plot.		At the moment, the only widget which gives the right type of signal needed by the ROC Analysis is Test & Score. Below, we compare two classifiers, namely Tree and Naive Bayes, in Test&Score and then compare their performance in ROC Analysis, Life Curve and Calibration Plot.

	~~../../_images/~~ROCAnalysis-example.png		[[File:ROCAnalysis-example.png\|center\|200px\|thumb]]




	==Referensi==		==Referensi==

Onnowpurbo: Created page with "Sumber: https://docs.biolab.si//3/visual-programming/widgets/evaluate/rocanalysis.html Plots a true positive rate against a false positive rate of a test. Inputs Evalua..."

2020-01-16T07:32:17Z

Created page with "Sumber: https://docs.biolab.si//3/visual-programming/widgets/evaluate/rocanalysis.html Plots a true positive rate against a false positive rate of a test. Inputs Evalua..."

New page

Sumber: https://docs.biolab.si//3/visual-programming/widgets/evaluate/rocanalysis.html

Plots a true positive rate against a false positive rate of a test.

Inputs

Evaluation Results: results of testing classification algorithms

The widget shows ROC curves for the tested models and the corresponding convex hull. It serves as a mean of comparison between classification models. The curve plots a false positive rate on an x-axis (1-specificity; probability that target=1 when true value=0) against a true positive rate on a y-axis (sensitivity; probability that target=1 when true value=1). The closer the curve follows the left-hand border and then the top border of the ROC space, the more accurate the classifier. Given the costs of false positives and false negatives, the widget can also determine the optimal classifier and threshold.

../../_images/ROCAnalysis-basic-stamped.png

Choose the desired Target Class. The default class is chosen alphabetically.

If test results contain more than one classifier, the user can choose which curves she or he wants to see plotted. Click on a classifier to select or deselect it.

When the data comes from multiple iterations of training and testing, such as k-fold cross validation, the results can be (and usually are) averaged. ../../_images/ROC-Comparison.png The averaging options are:

Merge predictions from folds (top left), which treats all the test data as if they came from a single iteration

Mean TP rate (top right) averages the curves vertically, showing the corresponding confidence intervals

Mean TP and FP at threshold (bottom left) traverses over threshold, averages the positions of curves and shows horizontal and vertical confidence intervals

Show individual curves (bottom right) does not average but prints all the curves instead

Option Show convex ROC curves refers to convex curves over each individual classifier (the thin lines positioned over curves). Show ROC convex hull plots a convex hull combining all classifiers (the gray area below the curves). Plotting both types of convex curves makes sense since selecting a threshold in a concave part of the curve cannot yield optimal results, disregarding the cost matrix. Besides, it is possible to reach any point on the convex curve by combining the classifiers represented by the points on the border of the concave region. ../../_images/ROCAnalysis-AUC.png The diagonal dotted line represents the behavior of a random classifier. The full diagonal line represents iso-performance. A black “A” symbol at the bottom of the graph proportionally readjusts the graph.

The final box is dedicated to the analysis of the curve. The user can specify the cost of false positives (FP) and false negatives (FN), and the prior target class probability.

Default threshold (0.5) point shows the point on the ROC curve achieved by the classifier if it predicts the target class if its probability equals or exceeds 0.5.

Show performance line shows iso-performance in the ROC space so that all the points on the line give the same profit/loss. The line further to the upper left is better than the one down and right. The direction of the line depends upon costs and probabilities. This gives a recipe for depicting the optimal threshold for the given costs: this is the point where the tangent with the given inclination touches the curve and it is marked in the plot. If we push the iso-performance higher or more to the left, the points on the iso-performance line cannot be reached by the learner. Going down or to the right, decreases the performance.

The widget allows setting the costs from 1 to 1000. Units are not important, as are not the magnitudes. What matters is the relation between the two costs, so setting them to 100 and 200 will give the same result as 400 and 800. Defaults: both costs equal (500), Prior target class probability 50%(from the data). ../../_images/ROCAnalysis-Plain.png False positive cost: 830, False negative cost 650, Prior target class probability 73%. ../../_images/ROCAnalysis.png

Press Save Image if you want to save the created image to your computer in a .svg or .png format.

Produce a report.

Example

At the moment, the only widget which gives the right type of signal needed by the ROC Analysis is Test & Score. Below, we compare two classifiers, namely Tree and Naive Bayes, in Test&Score and then compare their performance in ROC Analysis, Life Curve and Calibration Plot.

../../_images/ROCAnalysis-example.png

==Referensi==

* https://docs.biolab.si//3/visual-programming/widgets/evaluate/rocanalysis.html

==Pranala Menarik==

* [[Orange]]