Mr Morgan Mr Morgan - 7 months ago 74
Java Question

Which Weka and LibSVM .jar files to use in Java code for SVM classification

If I use Weka Explorer to run some training data against testing data using SVM with a linear kernel, everything is fine.

But I need to do this programmatically in my own Java and my current code looks like this:

Instances train = new Instances (...);
train.setClassIndex(train.numAttributes() - 1);
Instances test = new Instances (...) +
ClassificationType classificationType = ClassificationTypeDAO.get(6);
LibSVM libsvm = new LibSVM();
String options = (classificationType.getParameters());
String[] optionsArray = options.split(" ");
libsvm.setOptions(optionsArray);
String[] pars = libsvm.getOptions();
Evaluation eval = new Evaluation(train);
libsvm.buildClassifier(train);
eval.evaluateModel(libsvm, test);

System.out.println(eval.toSummaryString("\nResults\n======\n", false));


However, an exception is being thrown at line:

eval.evaluateModel(libsvm, test);


And despite numerous attempts at
try...catch
blocks around this code, the exception occurring is simply reported as
null
(which is really helpful) as per full stack trace below.

I don't believe this issue is due to my own code because other classifiers have run successfully with it. I am working on the theory that the cause of the problem is environmental. But where and what? I am running my application through NetBeans 8 using Tomcat and have recent versions of
weka.jar
and
LibSVM.jar
in the application's
.lib
folder.

But do I need
libsvm.jar
as provided by the download from:

http://www.csie.ntu.edu.tw/~cjlin/libsvm/

If the latter is the case, how can I resolve naming conflicts in Windows where
LibSVM.jar
and
libsvm.jar
are treated as the same file?

This has been really confusing me for the last few hours. I have tried adding both
LibSVM.jar
and
libsvm.jar
files into the
.lib
folder, renaming them both, putting them into a newly defined
CLASSPATH
, but nothing works.

The full stack trace for the Java exception is:


null
weka.classifiers.functions.LibSVM.distributionForInstance(LibSVM.java:1489)
weka.classifiers.Evaluation.evaluationForSingleInstance(Evaluation.java:1560)
weka.classifiers.Evaluation.evaluateModelOnceAndRecordPrediction(Evaluation.java:1597)
weka.classifiers.Evaluation.evaluateModel(Evaluation.java:1477)
visualRSS.test.Weka_LibSVM_Test.classify(Weka_LibSVM_Test.java:48)
visualRSS.initialisation.TestProgram_Context_Listener.contextInitialized(TestProgram_Context_Listener.java:29)
org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:3972)
org.apache.catalina.core.StandardContext.start(StandardContext.java:4467)
org.apache.catalina.core.StandardContext.reload(StandardContext.java:3228)
org.apache.catalina.manager.ManagerServlet.reload(ManagerServlet.java:943)
org.apache.catalina.manager.ManagerServlet.doGet(ManagerServlet.java:361)
javax.servlet.http.HttpServlet.service(HttpServlet.java:617)
javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:558)
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
org.apache.coyote.http11.Http11AprProcessor.process(Http11AprProcessor.java:859)
org.apache.coyote.http11.Http11AprProtocol$Http11ConnectionHandler.process(Http11AprProtocol.java:579)
org.apache.tomcat.util.net.AprEndpoint$Worker.run(AprEndpoint.java:1555)

Answer

The problem with my test code was environmental to do with the .jar files needed for Weka to programmatically run LibSVM.

If my code is:

public static void classify() {      
    try {            
        Instances train = new Instances (...);            
        train.setClassIndex(train.numAttributes() - 1);         
        Instances test = new Instances (...);            
        test.setClassIndex(test.numAttributes() - 1);                      
        ClassificationType classificationType = ClassificationTypeDAO.get(6);  // 6 is SVM.        
        LibSVM classifier = new LibSVM();
        String options = (classificationType.getParameters());
        String[] optionsArray = options.split(" ");                          
        classifier.setOptions(optionsArray);        
        classifier.buildClassifier(train);        
        Evaluation eval = new Evaluation(train);
        eval.evaluateModel(classifier, test);
        System.out.println(eval.toSummaryString("\nResults\n======\n", false));       
    } 
    catch (Exception ex) {            
        Misc_Utils.printStackTrace(ex);
    }                       
}

I found that I needed to place weka.jar (from Weka) and libsvm.jar (from http://www.csie.ntu.edu.tw/~cjlin/libsvm/ in the application's .lib folder. But because of the naming clash in Windows, I renamed the file LibSVM.jar (from Weka) to LibSVM_Weka.jar and added it to the .lib folder.

Running the program I now have results which match Weka's Explorer using keyword frequencies distributed unevenly across 5 categories of data.

Comments