Types of Modules

Although there are several types of modules, they can all be classified into one of two families: computational and user interface. Special care must be taken to distinguish between these two families of modules because they work differently and require a slightly different programmatic interface. The specific differences will be discussed later in this document.

Modules are also classified at a finer level by the kind of operation they perform. This classification scheme is really for the benefit of the people who create itineraries from existing modules. Each module type is represented with a different module icon in the D2K Toolkit, making for a more intuitive interface.

Modules are divided into the six types:

• User Input  User Input modules require interaction with the user. They generally present a dialog where the user can enter information or manipulate the data in a specific way. For example, modules used to identify how data is to be binned would be user input modules.
  
  
• Input  Input modules load data from files or databases. For example, modules that read a file, crawl the web, or read from a database would be input modules.
  
  
• Output  Output modules save data to files or databases or show it to the user on the screen. For example, modules that write files or print information to the screen would be output modules.
  
  
• Data Prep  Data Prep modules perform functions that prepare the data for analysis. Data selection, data cleaning, and data transformation algorithms would be of this type. For example, modules that perform binning or normalization of data would be data prep modules.
  
  
• Compute  Compute modules typically perform the main calculation for the application. For example, algorithms used to solve data mining problems would be implemented as compute modules.
  
  
• Visualization  Visualization modules provide visual feedback to the user. For example, modules that display a scatterplot graphical view of the data or a decision tree visualization of the data mining results would be visualization modules.
  
  

Model Modules

Data mining may be described as a process of searching for mechanisms that will enable the automated classification of certain characteristics given some related inputs. The problem is that the relationship between the data provided, and the desired prediction or classification may be obfuscated by the shear volume of data at hand. Data mining techniques often use machine learning algorithms to identify relationships between inputs and the result we wish to predict. A learning algorithm may process huge volumes of data to statistically quantify some pattern that will allow reasonably accurate predictions. The resulting function can then be used to classify other objects where the outcome is not known. This function is what we call a model. D2K Modules can be developed to actually output another module which implements this model. This mechanism is described in the section on building models.