*rank of randomly selected positive response*divided by sample size. This equivalence is presented in the following code:

library(ROCR

**)**

gain.chart

**<-****function****(**n**)****{** score

**<-**runif**(**n**)** y

**<-****(**runif**(**n**)****<**score**)** plot

**(**performance**(**prediction**(**score, y**)**, "tpr", "rpp"**)**, lwd

**=**7, main = paste**(**"N =", n**))** lines

**(**ecdf**((**rank**(-**score**)[**y**==**T**])****/**n**)**, verticals

**=**T, do.points**=**F, col**=**"red", lwd**=**3**)****}**

set.seed

**(**1**)**par

**(**mfrow**=**c**(**1, 2**))**gain.chart

**(**10**)**gain.chart

**(**10000**)**The code plots the following gain charts:

For small samples the two methods do not produce identical plots as ecdf returns step function and ROCR plot provides linear interpolation at jumps.

Have you used Area Under Lift (AUL), analogous to AUC, for model evaluation/comparison? Do you have suggestions on an approach for calculating AUL?

ReplyDeleteIf I understand your question correctly you are asking about the Gini coefficient.

ReplyDeleteI did find this: http://stats.stackexchange.com/questions/24325/lorenz-curve-and-gini-coefficient-for-measuring-classifier-performance

DeleteI will also be looking at calculating the area under the Gain curve as well... (http://www.saedsayad.com/model_evaluation_c.htm)

And what is your question?

DeleteI am looking for methods to calculate the area under the Gain and Lift curves. The flux package has a method using the trapezoid rule (http://artax.karlin.mff.cuni.cz/r-help/library/flux/html/auc.html). What method would you use to add it to your Gain code above?

DeleteI would use trapezoid rule for approximation of definite integral with data points distributed uniformly (http://en.wikipedia.org/wiki/Trapezoidal_rule).

DeleteNote that there you should set x_1=f(x_1)=0 in order to get the left boundary of the integral.

Thanks for the info and reply.

DeleteUse ROCR package in R

Deleteuse function performance

example

library(ROCR)

pred<-prediction(actual,predicted)

perf<-performance(pred,"tpr","fpr")

plot(perf,col="red")

abline(0,1, lty = 8, col = "grey")

auc<-performance(pred,"auc")

unlist(auc@y.values)