weka.filters
Class DiscretizeFilter

java.lang.Object
  |
  +--weka.filters.Filter
        |
        +--weka.filters.DiscretizeFilter

All Implemented Interfaces:

OptionHandler, java.io.Serializable, WeightedInstancesHandler

public class DiscretizeFilter

extends Filter

implements OptionHandler, WeightedInstancesHandler

An instance filter that discretizes a range of numeric attributes in the dataset into nominal attributes. Discretization can be either by simple binning, or by Fayyad & Irani's MDL method (the default).

Valid filter-specific options are:

-B num
Specify the (maximum) number of bins to divide numeric attributes into. (default class-based discretisation).

-O
Optimizes the number of bins using a leave-one-out estimate of the entropy.

-R col1,col2-col4,...
Specify list of columns to Discretize. First and last are valid indexes. (default none)

-V
Invert matching sense.

-D
Make binary nominal attributes.

-E
Use better encoding of split point for MDL.

-K
Use Kononeko's MDL criterion.

Author:

Len Trigg (trigg@cs.waikato.ac.nz), Eibe Frank (eibe@cs.waikato.ac.nz) (Fayyad and Irani's method)

See Also:

Serialized Form

Field Summary

protected double[][]	m_CutPoints Store the current cutpoints
protected Range	m_DiscretizeCols Stores which columns to Discretize
protected boolean	m_FindNumBins Find the number of bins using cross-validated entropy.
protected boolean	m_MakeBinary Output binary attributes for discretized attributes.
protected int	m_NumBins The number of bins to divide the attribute into
protected boolean	m_UseBetterEncoding Use better encoding of split point for MDL.
protected boolean	m_UseKononenko Use Kononenko's MDL criterion instead of Fayyad et al.'s
protected boolean	m_UseMDL True if discretisation will be done by MDL rather than binning

Fields inherited from class weka.filters.Filter

m_NewBatch

Constructor Summary

DiscretizeFilter()
Constructor - initialises the filter

Method Summary

java.lang.String	attributeIndicesTipText() Returns the tip text for this property
boolean	batchFinished() Signifies that this batch of input to the filter is finished.
java.lang.String	binsTipText() Returns the tip text for this property
protected void	calculateCutPoints() Generate the cutpoints for each attribute
protected void	calculateCutPointsByBinning(int index) Set cutpoints for a single attribute.
protected void	calculateCutPointsByMDL(int index, Instances data) Set cutpoints for a single attribute using MDL.
protected void	convertInstance(Instance instance) Convert a single instance over.
protected void	findNumBins(int index) Optimizes the number of bins using leave-one-out cross-validation.
java.lang.String	findNumBinsTipText() Returns the tip text for this property
java.lang.String	getAttributeIndices() Gets the current range selection
int	getBins() Gets the number of bins numeric attributes will be divided into
double[]	getCutPoints(int attributeIndex) Gets the cut points for an attribute
boolean	getFindNumBins() Get the value of FindNumBins.
boolean	getInvertSelection() Gets whether the supplied columns are to be removed or kept
boolean	getMakeBinary() Gets whether binary attributes should be made for discretized ones.
java.lang.String[]	getOptions() Gets the current settings of the filter.
boolean	getUseBetterEncoding() Gets whether better encoding is to be used for MDL.
boolean	getUseKononenko() Gets whether Kononenko's MDL criterion is to be used.
boolean	getUseMDL() Gets whether MDL will be used as the discretisation method.
java.lang.String	globalInfo() Returns a string describing this filter
boolean	input(Instance instance) Input an instance for filtering.
java.lang.String	invertSelectionTipText() Returns the tip text for this property
java.util.Enumeration	listOptions() Gets an enumeration describing the available options
static void	main(java.lang.String[] argv) Main method for testing this class.
java.lang.String	makeBinaryTipText() Returns the tip text for this property
void	setAttributeIndices(java.lang.String rangeList) Sets which attributes are to be Discretized (only numeric attributes among the selection will be Discretized).
void	setAttributeIndicesArray(int[] attributes) Sets which attributes are to be Discretized (only numeric attributes among the selection will be Discretized).
void	setBins(int numBins) Sets the number of bins to divide each selected numeric attribute into
void	setFindNumBins(boolean newFindNumBins) Set the value of FindNumBins.
boolean	setInputFormat(Instances instanceInfo) Sets the format of the input instances.
void	setInvertSelection(boolean invert) Sets whether selected columns should be removed or kept.
void	setMakeBinary(boolean makeBinary) Sets whether binary attributes should be made for discretized ones.
void	setOptions(java.lang.String[] options) Parses the options for this object.
protected void	setOutputFormat() Set the output format.
void	setUseBetterEncoding(boolean useBetterEncoding) Sets whether better encoding is to be used for MDL.
void	setUseKononenko(boolean useKon) Sets whether Kononenko's MDL criterion is to be used.
void	setUseMDL(boolean useMDL) Sets whether MDL will be used as the discretisation method.
java.lang.String	useBetterEncodingTipText() Returns the tip text for this property
java.lang.String	useKononenkoTipText() Returns the tip text for this property
java.lang.String	useMDLTipText() Returns the tip text for this property

Methods inherited from class weka.filters.Filter

batchFilterFile, bufferInput, copyStringValues, copyStringValues, filterFile, flushInput, getInputFormat, getInputStringIndex, getOutputFormat, getOutputStringIndex, getStringIndices, inputFormat, isOutputFormatDefined, numPendingOutput, output, outputFormat, outputFormatPeek, outputPeek, push, resetQueue, setOutputFormat, useFilter

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

m_DiscretizeCols

protected Range m_DiscretizeCols

Stores which columns to Discretize

m_NumBins

protected int m_NumBins

The number of bins to divide the attribute into

m_CutPoints

protected double[][] m_CutPoints

Store the current cutpoints

m_UseMDL

protected boolean m_UseMDL

True if discretisation will be done by MDL rather than binning

m_MakeBinary

protected boolean m_MakeBinary

Output binary attributes for discretized attributes.

m_UseBetterEncoding

protected boolean m_UseBetterEncoding

Use better encoding of split point for MDL.

m_UseKononenko

protected boolean m_UseKononenko

Use Kononenko's MDL criterion instead of Fayyad et al.'s

m_FindNumBins

protected boolean m_FindNumBins

Find the number of bins using cross-validated entropy.

Constructor Detail

DiscretizeFilter

public DiscretizeFilter()

Constructor - initialises the filter

Method Detail

listOptions

public java.util.Enumeration listOptions()

Gets an enumeration describing the available options

Specified by:

listOptions in interface OptionHandler

Returns:

an enumeration of all the available options

setOptions

public void setOptions(java.lang.String[] options)
                throws java.lang.Exception

Parses the options for this object. Valid options are:

-B num
Specify the (maximum) number of equal-width bins to divide numeric attributes into. (default class-based discretization).

-O Optimizes the number of bins using a leave-one-out estimate of the entropy. -R col1,col2-col4,...
Specify list of columns to discretize. First and last are valid indexes. (default none)

-V
Invert matching sense.

-D
Make binary nominal attributes.

-E
Use better encoding of split point for MDL.

-K
Use Kononeko's MDL criterion.

Specified by:

setOptions in interface OptionHandler

Parameters:

options - the list of options as an array of strings

Throws:

java.lang.Exception - if an option is not supported

getOptions

public java.lang.String[] getOptions()

Gets the current settings of the filter.

Specified by:

getOptions in interface OptionHandler

Returns:

an array of strings suitable for passing to setOptions

setInputFormat

public boolean setInputFormat(Instances instanceInfo)
                       throws java.lang.Exception

Sets the format of the input instances.

Overrides:

setInputFormat in class Filter

Parameters:

instanceInfo - an Instances object containing the input instance structure (any instances contained in the object are ignored - only the structure is required).

Returns:

true if the outputFormat may be collected immediately

Throws:

java.lang.Exception - if the input format can't be set successfully

input

public boolean input(Instance instance)

Input an instance for filtering. Ordinarily the instance is processed and made available for output immediately. Some filters require all instances be read before producing output.

Overrides:

input in class Filter

Parameters:

instance - the input instance

Returns:

true if the filtered instance may now be collected with output().

Throws:

java.lang.IllegalStateException - if no input format has been defined.

batchFinished

public boolean batchFinished()

Signifies that this batch of input to the filter is finished. If the filter requires all instances prior to filtering, output() may now be called to retrieve the filtered instances.

Overrides:

batchFinished in class Filter

Returns:

true if there are instances pending output

Throws:

java.lang.IllegalStateException - if no input structure has been defined

globalInfo

public java.lang.String globalInfo()

Returns a string describing this filter

Returns:

a description of the filter suitable for displaying in the explorer/experimenter gui

findNumBinsTipText

public java.lang.String findNumBinsTipText()

Returns the tip text for this property

Returns:

tip text for this property suitable for displaying in the explorer/experimenter gui

getFindNumBins

public boolean getFindNumBins()

Get the value of FindNumBins.

Returns:

Value of FindNumBins.

setFindNumBins

public void setFindNumBins(boolean newFindNumBins)

Set the value of FindNumBins.

Parameters:

newFindNumBins - Value to assign to FindNumBins.

makeBinaryTipText

public java.lang.String makeBinaryTipText()

Returns the tip text for this property

Returns:

tip text for this property suitable for displaying in the explorer/experimenter gui

getMakeBinary

public boolean getMakeBinary()

Gets whether binary attributes should be made for discretized ones.

Returns:

true if attributes will be binarized

setMakeBinary

public void setMakeBinary(boolean makeBinary)

Sets whether binary attributes should be made for discretized ones.

Parameters:

makeBinary - if binary attributes are to be made

useMDLTipText

public java.lang.String useMDLTipText()

Returns the tip text for this property

Returns:

tip text for this property suitable for displaying in the explorer/experimenter gui

getUseMDL

public boolean getUseMDL()

Gets whether MDL will be used as the discretisation method.

Returns:

true if so, false if fixed bins should be used.

setUseMDL

public void setUseMDL(boolean useMDL)

Sets whether MDL will be used as the discretisation method.

Parameters:

useMDL - true if MDL should be used, false if fixed bins should be used.

useKononenkoTipText

public java.lang.String useKononenkoTipText()

Returns the tip text for this property

Returns:

tip text for this property suitable for displaying in the explorer/experimenter gui

getUseKononenko

public boolean getUseKononenko()

Gets whether Kononenko's MDL criterion is to be used.

Returns:

true if Kononenko's criterion will be used.

setUseKononenko

public void setUseKononenko(boolean useKon)

Sets whether Kononenko's MDL criterion is to be used.

Parameters:

useKon - true if Kononenko's one is to be used

useBetterEncodingTipText

public java.lang.String useBetterEncodingTipText()

Returns the tip text for this property

Returns:

tip text for this property suitable for displaying in the explorer/experimenter gui

getUseBetterEncoding

public boolean getUseBetterEncoding()

Gets whether better encoding is to be used for MDL.

Returns:

true if the better MDL encoding will be used

setUseBetterEncoding

public void setUseBetterEncoding(boolean useBetterEncoding)

Sets whether better encoding is to be used for MDL.

Parameters:

useBetterEncoding - true if better encoding to be used.

binsTipText

public java.lang.String binsTipText()

Returns the tip text for this property

Returns:

tip text for this property suitable for displaying in the explorer/experimenter gui

getBins

public int getBins()

Gets the number of bins numeric attributes will be divided into

Returns:

the number of bins.

setBins

public void setBins(int numBins)

Sets the number of bins to divide each selected numeric attribute into

Parameters:

numBins - the number of bins

invertSelectionTipText

public java.lang.String invertSelectionTipText()

Returns the tip text for this property

Returns:

tip text for this property suitable for displaying in the explorer/experimenter gui

getInvertSelection

public boolean getInvertSelection()

Gets whether the supplied columns are to be removed or kept

Returns:

true if the supplied columns will be kept

setInvertSelection

public void setInvertSelection(boolean invert)

Sets whether selected columns should be removed or kept. If true the selected columns are kept and unselected columns are deleted. If false selected columns are deleted and unselected columns are kept.

Parameters:

invert - the new invert setting

attributeIndicesTipText

public java.lang.String attributeIndicesTipText()

Returns the tip text for this property

Returns:

tip text for this property suitable for displaying in the explorer/experimenter gui

getAttributeIndices

public java.lang.String getAttributeIndices()

Gets the current range selection

Returns:

a string containing a comma separated list of ranges

setAttributeIndices

public void setAttributeIndices(java.lang.String rangeList)

Sets which attributes are to be Discretized (only numeric attributes among the selection will be Discretized).

Parameters:

rangeList - a string representing the list of attributes. Since the string will typically come from a user, attributes are indexed from 1.
eg: first-3,5,6-last

Throws:

java.lang.IllegalArgumentException - if an invalid range list is supplied

setAttributeIndicesArray

public void setAttributeIndicesArray(int[] attributes)

Sets which attributes are to be Discretized (only numeric attributes among the selection will be Discretized).

Parameters:

attributes - an array containing indexes of attributes to Discretize. Since the array will typically come from a program, attributes are indexed from 0.

Throws:

java.lang.IllegalArgumentException - if an invalid set of ranges is supplied

getCutPoints

public double[] getCutPoints(int attributeIndex)

Gets the cut points for an attribute

Parameters:

the - index (from 0) of the attribute to get the cut points of

Returns:

an array containing the cutpoints (or null if the attribute requested isn't being Discretized

calculateCutPoints

protected void calculateCutPoints()

Generate the cutpoints for each attribute

calculateCutPointsByMDL

protected void calculateCutPointsByMDL(int index,
                                       Instances data)

Set cutpoints for a single attribute using MDL.

Parameters:

index - the index of the attribute to set cutpoints for

calculateCutPointsByBinning

protected void calculateCutPointsByBinning(int index)

Set cutpoints for a single attribute.

Parameters:

index - the index of the attribute to set cutpoints for

findNumBins

protected void findNumBins(int index)

Optimizes the number of bins using leave-one-out cross-validation.

Parameters:

index - the attribute index

setOutputFormat

protected void setOutputFormat()

Set the output format. Takes the currently defined cutpoints and m_InputFormat and calls setOutputFormat(Instances) appropriately.

convertInstance

protected void convertInstance(Instance instance)

Convert a single instance over. The converted instance is added to the end of the output queue.

Parameters:

instance - the instance to convert

main

public static void main(java.lang.String[] argv)

Main method for testing this class.

Parameters:

argv - should contain arguments to the filter: use -h for help