Writing New Passes For Multipass

Multipass is a GIMP plug in who's purpose is to allow the combination (or chaining) of different segmentation algorithms (that operate on gray level images) into one. In this paper we're going to see how to write new segmentation algorithms and integrate them into the whole.

Pass Overview

In the context of Multipass, a segmentation algorithm is called a ``pass''. Now, a pass has to fulfill some requirements before it can seamlessly integrate with the rest. The following figure shows the primary data flows that take place around a pass:

Pass data flows

As you can see, a pass' primary purpose is to take a histogram and an initial set of class centers(a.k.a centroids) and return and updated/improved set of class centers. A histogram gives the number of pixels of any particular gray level in an image. As for class centers, they are simply particular gray levels that represent the ``geometrical'' center of their own class of gray levels. As it may be guessed, the centroid set may come from another pass, and the output value may also be handed over to another algorithm.

Aside from the histogram and centroid set, a pass may actually take additional arguments that will allow to fine tune the algorithm's behavior to one's needs. In the context of Multipass, these arguments are called ``parameters''. It is the pass' responsibility to declare the number and type of parameters it may take. The application core will take that information into account and create the necessary user input interface to accept the data, and make it available to the pass algorithm itself.

The Template Pass

In order to illustrate the discussion we'll go through a ``template pass'' code, which has everything in place except for the algorithm implementation itself. You can find it under the names of template_pass.hpp and template_pass.cpp. Hopefully these two files can be used as a starting point every time you need to implement a new pass.

Please note that throughout this discussion we'll be referring to classes and methods implemented in Multipass core. So, it is strongly advised that you take a look at the corresponding header files for further information and details about their use, this document will not duplicate the documentation effort made in the source code.

The header file

The sole purpose of template_pass.hpp is to declare the template_pass class:


#include "pass.hpp"

/// A template for implementing a new pass
class template_pass: public pass
{
	private:
		/// The number of parameters we need
		enum { m_param_info_size=2 };

		/// Holds a param_info for each paramter we take
		static param_info m_param_info[m_param_info_size];

The header begins by declaring the template_pass class itself as child class of class pass. As you may have guessed class pass is actually the mother class of all pass classes (see pass.hpp).

Next step is to inform class pass about the number and type of parameters our template class wishes to take. This is done using an array of param_info. A param_info (again see param_info.hpp) holds information about a parameter of a pass. The code above simply starts by storing the number of parameters in a constant (m_param_info_size) before declaring a static array(m_param_info) of param_info to hold the parameter info themselves. Note that we've only declared the array here we'll be filling the it in a moment.


	public:
	/// Pass a label, an array of param_info and the size of the
	/// array to parent class
	template_pass():
		pass("Template", m_param_info, m_param_info_size)
	{}

After that starts the public interface. Here you can see that the pass' constructor takes three arguments:

A label (char const*): Which serve as the display name of the algorithm, and has to be unique among other algorithms;
An array of param_info: Which contain a param_info for each parameter we take. Notice how the code passes the earlier mentioned m_param_info;
The number of element in the array.

	/// The actual implementation of the algorithm
	/// historgram: Holds the number of pixel of each possible
	///    gray level in the image.
	/// centers: The input class centers. The algorithm should update
	///   them/improve them.
	/// ind: Can be used to report the progression
	virtual void operator() (histogram const & ,
			std::vector<histogram::gray_level_type*gt;& centers,
			progress_indicator* ind) const;

Now comes the most important member. The operatot() is the one that will be implementing the actual algorithm. As you see the centroid set is not more than an std::vector of histogram::gray_level_type. The latter being a typedef for char or unsigned char. Please note how the histogram is passed as a const reference(you need not to modify it), while the centroid set is passed a none-const reference. This means that you'll have to update the centroid set that was passed to you directly. The last parameter is a small device which has nothing to do with the segmentation itself but that makes you able to report about your progression while you're working (especially for a time consuming algorithm). It only has an operator() that you can call passing a percentage of progression (0.0 means not yet started, 1.0 means finished). Like always, be sure to have a look at histogram.hpp and progress_indicator.hpp for further details

	/// Must return a new copy of *this
	virtual pass* clone() const { return new template_pass(*this); }
};
#endif // TEMPLATE_PASS_HPP

The last part of the file contains but one more member: clone. You don't really need to know why it is for. Just make sure you declare it correctly, and that you define it to return a new-ly allocated copy of *this just like it is shown above.

The source file

The source file's purpose is to define the parameter information, and the algorithm itself.

#include "template_pass.hpp"
#include "histogram.hpp"

/// Definition of parameter info array (contains an element per parameter)
param_info 
template_pass::m_param_info[template_pass::m_param_info_size]=
{
	/// First parameter will be an integer from 1 to 10 named "Number of
	/// Iterations
	param_info(1, 10, "Number of Iterations", "Number of passes to apply"),

	/// Second parameter will be a double between 0 and 1
	param_info(0.0, 1.0, "Precision", "Accuracy of calculations")
};

Here you can see the definition of the m_param_info we saw earlier in the class declaration. Note how the code fills the array with instances of param_info. (see param_info.hpp)

/// Actual implementation of the algorithm
void template_pass::operator() (histogram const& hist,
			std::vector<histogram::gray_level_type>& centroids,
			progress_indicator* ind) const
{
	// First extract actual parameter values
	int const nr_passes=this->pass::parameter_value(0).int_value();
	double const precision=this->pass::parameter_value(1).int_value();

Finally comes the implementation of the algorithm itself. The first thing you may want to do is to retrieve the actual value of the parameters. Remember we asked for an integer from 1 to 10 and a double value from 0.0 to 1.0. Now in order to get the actual values you'll use pass::parameter_value() (implemented in parent class pass). Pass the index of the parameter you want (according to the order you provided them in the array of param_info). Now the return value of pass::parameter_value() can't be simply int or double (because type information can't be known before execution). So, the return value is actually a param, which is a multipurpose data container that may contain integers and doubles. What you have to do is to call param::int_value() to get an integer out of a param, and call param::double_value() to get a double.

	double total_pixels=0;
	// Count the total number of pixels
	for(histogram::const_iterator it=hist.begin();
			it!=hist.end(); ++it)
		total_pixels+=*it;

	int pass_count=nr_passes;
	while(pass_count--){
		// Go through the histogram
		for(histogram::const_iterator it=hist.begin(); it!=hist.end(); ++it){
			// do something about the centroids	
		}
	}// while pass_count--
}

The rest of the file is simply stub code that does nothing particularity worth mentioning. In a real example the objective would be to use information provided by the histogram and centroid set to update the latter one.

Integration

After you have written the actual code, you have to make it available (or known) to the rest of the application, which is actually the easiest part. Just open register_passes.cpp and add a line about your new algorithm just like shown below:

#include "register_passes.hpp"
#include "pass_repertoire.hpp"
#include "ainet_pass.hpp"
#include "kmeans_pass.hpp"

#include "template_pass.hpp"  // <<< See here ////////

void register_passes()
{
	pass_repertoire& rep=*pass_repertoire::instance();

	rep.add(new ainet_pass());
	rep.add(new kmeans_pass());
	
	rep.add(new template_pass()); // <<< See here ////////
}

Next you'll of course have to recompile the whole application without forgetting to add any new source files your algorithm requires.

Conclusion

I hope that this will have helped you getting the broad idea about writing new pass classes for multipass. But please understand that this is only an overview, the actual (and indispensable) details (such as ``how to use class histogram'') are in the corresponding headers. Another source of information may be to look at already existing pass source code, especially simple ones like that of K-Means(kmeans_pass.hpp and kmeans_pass.cpp).