Monday, November 30, 2009

How to apply accuracy assessment using error matrix?

I'm trying to analyse the data from satellite image classification. I found out that error matrix is the best way to find accuracy of the results. can anyone tell me how exactly can i find the values inside the error matrix cells. the calculation is easy once you find the values of the cells as most of my sources has guided me through it. my problem is to exactly find the cell values. could anyone help me with this. thx in advanceHow to apply accuracy assessment using error matrix?
I did something similar (may be the same!). I call it 'confusion' matrix arising out of 2-class 'pattern recognition' (PR) task. Most of the interpretation of satellite data is also a kind of a PR.


When you say 'accuracy' you are comparing one set of data against a 'set' picture considered accurate. That way the machine classifier's accuracy is established.


It is a 2 class problem, where the picture that is already 'set' classifies (your data points) it as belonging to one or the other class : 'Class I? - Yes' or 'Class 2? - Yes' '. Obviously each statement implies that the other class is a 'no' and we are only considering 'yes' es.


Now, your machine classifier moves in and makes its own classification. Its result is compared against the 'set' picture, point by point. For every data point ('datum'), both the opinions are compared.


There are 4 possibilities and one of them is chosen for every data point;


'Set' says Class1 (Yes), machine says Class1 (yes);


.....Class1, Class2;


.....Class2, Class1;


.....Class2, Class2.


After running out of points, the results of the points (as number of points) are arranged in a 'confusion matrix'


| (1/1) (1/2)|


| (2/1) (2/2)|.


The off-diagonal elements (disagreements) should be as near to zero as possible and diagonal elements (agreements) should contain maximum number of data points, ideally.


If n= no.of total data points, all the elements should add upto it.


Your design of the decision criterion should be refined till you get


| p.......0|


| 0.. (n-p)|.


I hope you can adapt it.

No comments:

Post a Comment

 
albinism