weka.classifiers
Class LogitBoost

java.lang.Object
  |
  +--weka.classifiers.Classifier
        |
        +--weka.classifiers.DistributionClassifier
              |
              +--weka.classifiers.LogitBoost

All Implemented Interfaces:

java.lang.Cloneable, OptionHandler, java.io.Serializable, Sourcable

public class LogitBoost

extends DistributionClassifier

implements OptionHandler, Sourcable

Class for boosting any classifier that can handle weighted instances. This class performs classification using a regression scheme as the base learner, and can handle multi-class problems. For more information, see

Friedman, J., T. Hastie and R. Tibshirani (1998) Additive Logistic Regression: a Statistical View of Boosting download postscript.

Valid options are:

-D
Turn on debugging output.

-W classname
Specify the full class name of a weak learner as the basis for boosting (required).

-I num
Set the number of boost iterations (default 10).

-Q
Use resampling instead of reweighting.

-S seed
Random number seed for resampling (default 1).

-P num
Set the percentage of weight mass used to build classifiers (default 100).

Options after -- are passed to the designated learner.

Author:

Len Trigg (trigg@cs.waikato.ac.nz)

See Also:

Serialized Form

Field Summary

protected Attribute	m_ClassAttribute The actual class attribute (for getting class names)
protected Classifier	m_Classifier An instantiated base classifier used for getting and testing options
protected Classifier[][]	m_Classifiers Array for storing the generated base classifiers.
protected boolean	m_Debug Debugging mode, gives extra output if true
protected int	m_MaxIterations The maximum number of boost iterations
protected int	m_NumClasses The number of classes
protected Instances	m_NumericClassData Dummy dataset with a numeric class
protected int	m_NumIterations The number of successfully generated base classifiers.
protected int	m_Seed Seed for boosting with resampling.
protected boolean	m_UseResampling Use boosting with reweighting?
protected int	m_WeightThreshold Weight thresholding.
protected static double	VERY_SMALL A very small number, below which weights cannot fall
protected static double	Z_MAX A threshold for responses (Friedman suggests between 2 and 4)

Constructor Summary

LogitBoost()

Method Summary

void	buildClassifier(Instances data) Boosting method.
double[]	distributionForInstance(Instance instance) Calculates the class membership probabilities for the given test instance.
Classifier	getClassifier() Get the classifier used as the classifier
boolean	getDebug() Get whether debugging is turned on
int	getMaxIterations() Get the maximum number of boost iterations
java.lang.String[]	getOptions() Gets the current settings of the Classifier.
int	getSeed() Get seed for resampling.
boolean	getUseResampling() Get whether resampling is turned on
int	getWeightThreshold() Get the degree of weight thresholding
java.util.Enumeration	listOptions() Returns an enumeration describing the available options
static void	main(java.lang.String[] argv) Main method for testing this class.
protected static double	RtoP(double[] R, int j) Convert from function responses to probabilities
protected Instances	selectWeightQuantile(Instances data, double quantile) Select only instances with weights that contribute to the specified quantile of the weight distribution
void	setClassifier(Classifier newClassifier) Set the classifier for boosting.
void	setDebug(boolean debug) Set debugging mode
void	setMaxIterations(int maxIterations) Set the maximum number of boost iterations
void	setOptions(java.lang.String[] options) Parses a given list of options.
void	setSeed(int seed) Set seed for resampling.
void	setUseResampling(boolean r) Set resampling mode
void	setWeightThreshold(int threshold) Set weight thresholding
java.lang.String	toSource(java.lang.String className) Returns the boosted model as Java source code.
java.lang.String	toString() Returns description of the boosted classifier.

Methods inherited from class weka.classifiers.DistributionClassifier

classifyInstance

Methods inherited from class weka.classifiers.Classifier

forName, makeCopies

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

m_Classifiers

protected Classifier[][] m_Classifiers

Array for storing the generated base classifiers.

m_Classifier

protected Classifier m_Classifier

An instantiated base classifier used for getting and testing options

m_MaxIterations

protected int m_MaxIterations

The maximum number of boost iterations

m_NumClasses

protected int m_NumClasses

The number of classes

m_NumIterations

protected int m_NumIterations

The number of successfully generated base classifiers.

m_WeightThreshold

protected int m_WeightThreshold

Weight thresholding. The percentage of weight mass used in training

m_Debug

protected boolean m_Debug

Debugging mode, gives extra output if true

VERY_SMALL

protected static final double VERY_SMALL

A very small number, below which weights cannot fall

Z_MAX

protected static final double Z_MAX

A threshold for responses (Friedman suggests between 2 and 4)

m_NumericClassData

protected Instances m_NumericClassData

Dummy dataset with a numeric class

m_ClassAttribute

protected Attribute m_ClassAttribute

The actual class attribute (for getting class names)

m_UseResampling

protected boolean m_UseResampling

Use boosting with reweighting?

m_Seed

protected int m_Seed

Seed for boosting with resampling.

Constructor Detail

LogitBoost

public LogitBoost()

Method Detail

selectWeightQuantile

protected Instances selectWeightQuantile(Instances data,
                                         double quantile)

Select only instances with weights that contribute to the specified quantile of the weight distribution

Parameters:

data - the input instances

quantile - the specified quantile eg 0.9 to select 90% of the weight mass

Returns:

the selected instances

RtoP

protected static double RtoP(double[] R,
                             int j)

Convert from function responses to probabilities

Parameters:

R - an array containing the responses from each function

j - the class value of interest

Returns:

the probability prediction for j

listOptions

public java.util.Enumeration listOptions()

Returns an enumeration describing the available options

Specified by:

listOptions in interface OptionHandler

Returns:

an enumeration of all the available options

setOptions

public void setOptions(java.lang.String[] options)
                throws java.lang.Exception

Parses a given list of options. Valid options are:

-D
Turn on debugging output.

-W classname
Specify the full class name of a weak learner as the basis for boosting (required).

-I num
Set the number of boost iterations (default 10).

-Q
Use resampling instead of reweighting.

-S seed
Random number seed for resampling (default 1).

-P num
Set the percentage of weight mass used to build classifiers (default 100).

Options after -- are passed to the designated learner.

Specified by:

setOptions in interface OptionHandler

Parameters:

options - the list of options as an array of strings

Throws:

java.lang.Exception - if an option is not supported

getOptions

public java.lang.String[] getOptions()

Gets the current settings of the Classifier.

Specified by:

getOptions in interface OptionHandler

Returns:

an array of strings suitable for passing to setOptions

setUseResampling

public void setUseResampling(boolean r)

Set resampling mode

Parameters:

resampling - true if resampling should be done

getUseResampling

public boolean getUseResampling()

Get whether resampling is turned on

Returns:

true if resampling output is on

setSeed

public void setSeed(int seed)

Set seed for resampling.

Parameters:

seed - the seed for resampling

getSeed

public int getSeed()

Get seed for resampling.

Returns:

the seed for resampling

setClassifier

public void setClassifier(Classifier newClassifier)

Set the classifier for boosting. The learner should be able to handle numeric class attributes.

Parameters:

newClassifier - the Classifier to use.

getClassifier

public Classifier getClassifier()

Get the classifier used as the classifier

Returns:

the classifier used as the classifier

setMaxIterations

public void setMaxIterations(int maxIterations)

Set the maximum number of boost iterations

Parameters:

maxIterations - the maximum number of boost iterations

getMaxIterations

public int getMaxIterations()

Get the maximum number of boost iterations

Returns:

the maximum number of boost iterations

setWeightThreshold

public void setWeightThreshold(int threshold)

Set weight thresholding

Parameters:

thresholding - the percentage of weight mass used for training

getWeightThreshold

public int getWeightThreshold()

Get the degree of weight thresholding

Returns:

the percentage of weight mass used for training

setDebug

public void setDebug(boolean debug)

Set debugging mode

Parameters:

debug - true if debug output should be printed

getDebug

public boolean getDebug()

Get whether debugging is turned on

Returns:

true if debugging output is on

buildClassifier

public void buildClassifier(Instances data)
                     throws java.lang.Exception

Boosting method. Boosts any classifier that can handle weighted instances.

Overrides:

buildClassifier in class Classifier

Parameters:

data - the training data to be used for generating the boosted classifier.

Throws:

java.lang.Exception - if the classifier could not be built successfully

distributionForInstance

public double[] distributionForInstance(Instance instance)
                                 throws java.lang.Exception

Calculates the class membership probabilities for the given test instance.

Overrides:

distributionForInstance in class DistributionClassifier

Parameters:

instance - the instance to be classified

Returns:

predicted class probability distribution

Throws:

java.lang.Exception - if instance could not be classified successfully

toSource

public java.lang.String toSource(java.lang.String className)
                          throws java.lang.Exception

Returns the boosted model as Java source code.

Specified by:

toSource in interface Sourcable

Returns:

the tree as Java source code

Throws:

java.lang.Exception - if something goes wrong

toString

public java.lang.String toString()

Returns description of the boosted classifier.

Overrides:

toString in class java.lang.Object

Returns:

description of the boosted classifier as a string

main

public static void main(java.lang.String[] argv)

Main method for testing this class.

Parameters:

argv - the options