How do I compile on a Windows PC?

• The following instructions apply to SVM^light, SVM^struct, SVM^perf, SVM^cfg, SVM^multiclass, and SVM^hmm.
• The easiest is to install CYGWIN and use the gcc compiler that comes with the Cygwin distribution. Just open a cygwin command window, change to the directory that contains file "Makefile" and type
              make
  at the command prompt. This creates executables that run under Cygwin.
• If you want execuables that do not need Cygwin to run, follow the same instructions as above, but make sure that you download the MinGW package as part of Cygwin. Then use the command
              make 'SFLAGS=-mno-cygwin'
  to create the executables.
• Another method for generating executables that do not need Cygwin to run, especially if you want 64-bit executables, is to use the MinGW64 package of Cygwin. You need to explicitly download the MinGW64 gcc compiler by selecting the appropriate Cygwin package in the installer. Then use the command
              make 'CC = x86_64-w64-mingw32-gcc' 'LD = x86_64-w64-mingw32-gcc'
  to create the executables.
• You can also use the Visual C compiler, but it will be much more of a hassle than using Cygwin. To use Visual C, you need to build the appropriate project file according to the provided "Makefile". Make sure you do not include the file "svm_loqo.c" in your project, unless you want to use the PL_LOQO optimizer instead of the built-in optimizer.

How do I compile on Mac OSX?

• Just get gcc, and it should compile without changes. This applies to SVM^light, but should also work for SVM^struct, SVM^perf, SVM^cfg, SVM^multiclass, and SVM^hmm.

How do I compile on a PowerMac using Code Warrior?

• This applies to SVM^light, but should also work for SVM^struct, SVM^perf, SVM^cfg, SVM^multiclass, and SVM^hmm.
• You need to modify the source code a little (as suggested by Jewgeni Starikow). Use #include "console.h" to emulate a UNIX shell. Then add argc=ccommand(&argv) as the first instruction of each main().
• CPU-timing might cause another problem. The timing routines are used to calculated the runtime of the program. If you do not need this feature, remove the body of get_runtime() in svm_common.c. Otherwise replace the body with the appropriate Mac routines from 'time.h'.

How do I integrate SVM^light into C++ code?

• Compile "svm_learn.c", "svm_common.c", and "svm_hideo.c" as C code.
• The C++ program you want to call svm_learn/8 and classify_example/2 (or classify_example_linear/2) from needs to include the following headers:
  extern "C" {
  # include "svm_common.h"
  # include "svm_learn.h"
  }
• Link "svm_learn.o", "svm_common.o", and "svm_hideo.o" to your program.

Is there an option for doing multi-class classification?

• Not in SVM^light. If you do not use kernel, you can use SVM^multiclass to do multi-class classification. Otherwise, I recommend to split the task into multiple binary classification tasks for multi-class or multi-label classification. For example, you can do one-class-against-the-rest classification, or pairwise classification. Refer to Chapter 2 in [Joachims, 2002a].

Is there an option for doing cross-validation?

• Yes, there is such an option in SVM^light, but not in SVM^struct. By setting the option "-x 1", SVM^light computes the leave-one-out estimates of the prediction error, recall, precision, and F1. By default, it also outputs the XiAlpha-Estimates of these quantities. They are strictly conservative approximations to the leave-one-out (i.e. they never underestimate the leave-one-out error). Refer to Chapter 5 in [Joachims, 2002a].

How can I output the dual variables at the solution?

• Use the option -a <filename>. The file contains the value of the dual variables (multiplied by the label y) for each training example in the order in which the examples appeared in the training file.

How can I get the weight vector of the hyperplane for a linear SVM?

• There is a small PERL script to compute the weight vector (similar script in PYTHON by Ori Cohen) based on the model file output by svm_learn and svm_perf_learn. Of course, it works only in the linear case and not for kernels. All the script does is compute the weighted sum of the support vectors (first element in line is alpha*y, what follows is the feature vector). For further info, see the comment in the model file for its format.

What is the format of the learned model file?

• The model files produced by svm_learn and svm_perf_learn are in ASCII and easy to read. The first few lines contain parameters. The comments in the model file are self-explanatory. The last of these is the threshold b of the learned hyperplane classifier sign(w*x - b). The following lines each contain a support vector, and the first element of the line is its coefficient (alpha_i * y_i) in the kernel expansion. The support vectors are listed in random order. Note that the term support vector has a different meaning in SVM^perf compared to SVM^light.

How can I implement my own Kernel?

• You can write your own kernel by extending the function "double custom_kernel(KERNEL_PARM *kernel_parm, SVECTOR *a, SVECTOR *b)" in "kernel.h" and then select it via the "-t 4" option. The "a" and "b" are pointers to the examples you are computing the kernel for.
• If the data you are working with is vectorial, then you can use the existing sparse vector operations defined in svm_common.c. For example,
      pow(sprod_ss(a->words,b->words),2.0)
  implements the homogeneous polynomial kernel of degree 2.
• If the data is non-vectorial (e.g. strings), then you can use the following functionality. In the training and test data files, whatever is written behind the "#" is copied into the "userdefined" field of the internal datastructure "SVECTOR" as a string. So, for the line
      +1 #abcdefg
  you will find "abcdefg" in "svector->userdefined".

Error messages and known problems:

• ERROR: terminating optimizer - choldc failed, matrix not positive definite
• If the program terminates after this message, get the lastest version of PR_LOQO and SVM-Light V2.01 (or later).
• It the program continues after this error message, don't worry :-)
• The CPU-time is negative or looks bogus:
• To be compatible with Windows, I used the clock/0 function to measure CPU-time. This function returns a long integer and counts microseconds, so that it wraps around after about 30 minutes. You could use timing routines more appropriate for your system. For example on Solaris, you might want to use the routines from 'sys/times.h'.

The program hangs when ...

• ... reading in the examples.
• Get version 3.02 or later.

Convergence during learning is very slow!

• In verbose mode 2 I observe that max violation bounces around and does not really converge.
• Use a smaller value for the option -n (default n=q). This makes sure that only n new variables enter the working set in each iteration. This can prevent zig-zagging behavior.
• Use a smaller or larger value for the size of the working set (option -q). Reasonable values are in the range [2:50].
• You might be using an excessively large value of C (option -c) in relation to your data. The value of C should typically be less than 1000 times 1.0/(the maximum squared Euclidian length of your feature vectors).
• You have weired data and the convergence simply IS very slow J . Sorry, not much you can do about it.
• Nearly all my training examples end up as support vectors.
• Use a "stiffer" kernel (e g. a lower value of gamma for the RBF-kernel). If you pick a kernel which is very far away from the optimum, you will not get good generalization performance anyway.
• Your data is really difficult to separate. Think about a better representation of your data. It is a bad idea - for example - if different features have values in very much different orders of magnitude. You might want to normalize all features to the range [-1,+1].

It does not converge!

• If you are using the built-in HIDEO optimizer, get the version 3.50 or later. There used to be problems for low dimensional data sets for the old HIDEO optimizer.
• It should always converge, unless there are numerical problems :-)
• Numerical problems.
• Make sure your data is properly scaled. It is a bad idea - for example - if different features have values in different orders of magnitude. You might want to normalize all features to the range [-1,+1], especially for problems with more than 100 features. Or even better, normalize all feature vectors to Euclidian length 1.

It crashes!

• It crashes during learning when using a non-linear kernel.
  • Get version 5.00 or later. There was an initialization bug in the kernel cache.

The results look bogus!

• When doing transductive learning, for some training set sizes it does not obey the -p parameter and the results look bogus.
  • Get version 5.00 or later. There was a bug introduced in version 4.00.

Last modified July 25, 2013 by Thorsten Joachims <thorsten@joachims.org>