Plugins

A plugin (also called plug-in, addin, add-in, addon, add-on, snap-in, extension or suplement) is a computer program that interacts with a host application like TerraView adding new capabilities/functionalities to it. Plugins can also be used to plug new functionalities in an extensible library/platform. For more information on plugin definitions see Wikipedia.

The TerraLib plugin module comes with a basic framework for those who want to create plugins in the TerraLib Platform. Instead of having a framework target to a specific application like TerraView or SISMADEN, TerraLib Plugin framework just handle the extensions (plugins) and is capable of loading then dynamically. It provides the basic foundation to simplify the burden of dealing with plugins: dependencies, configuration and startup/shutdown actions. Besides that in TerraLib 5 a plugin can be built using any language that has a support in the language binding modules. For instance you can create plugins in Haskell, Lua, PHP, Java, Python, C and C++.

Note that in TerraLib any plugin may have access to all other system parts. You can put no restrictions on it, so you can easily use all TerraLib API without worry about doing trickies. Also you have to define the API of your application. Each application plugin will be dependent on its application API. If you have parts of your plugin that can be shared between applications, you should isolate that part and make a plugin with a dependency for it.

• a plugin must provide a root object derived from an abstract plugin class that implements a couple of methods (startup and shutdown)
• a plugin can have initialization parameters
• the plugin framework provides a basic model for the information needed by a plugin
• Although a given plugin doesn't have a link dependency of another plugin it can expouse this dependency in its configuration
• It provides a nice way to control the existence of debug/release versions of the plugins

As pointed out by Vandevoorde (2006), for C and C++ programs plugins are typically implemented as shared libraries that are dynamically loaded (and sometimes unloaded) by the main program.

The next section will explain in details the design of this module.

The following class diagram shows the plugin framework:

The AbstractPlugin class is an abstract class that models a plugin. For each programming language there will be an specific concrete plugin class responsible for handling all the stuffs for linking the C++ part and the specific language support.

The following class diagram shows this relationship:

The class PluginInfo models the basic information about a plugin:

• an internal value used to identify the plugin in the system. It must be a unique value.
• the plugin name to be displayed in a graphical interface.
• a brief explanation about the plugin.
• the plugin version.
• the release date of the plugin. This may be used to identify new versions of a given plugin.
• the type of plugin execution engine: C++, LUA, JAVA.
• the TerraLib version this plugin depends on.
• a brief description about the plugin license.
• an URL where someone can find more information on the license.
• the plugin category. You can create categories for your plugins like: Data Access, Unknown Category, Language Bindings.
• an URL pointing to the plugin site.
• information about the plugin provider.
• the plugin folder (where the plugin is installed).
• the list of required plugins in order to launch the plugin.
• the list of required categories of plugins in order to launch the plugin.
• the list of resources used by the plugin.

In the case of C++ plugins, there is a proxy class named CppPluginProxy. This class helps to separate the real plugin object from its shared library. This is very important because of the unload of this type of plugin.

For each programming language there will be an specific plugin engine capable of loading and preparing the plugin. This gives us more flexibility on how plugins are loaded.

The PluginEngineFactory class is an abstract factory that each plugin engine must implement. This makes the plugin framework very extensible at runtime.

The PluginManager is a singleton that can be used to manage a set of plugins. The basic idea behind this class is that applications should orchestrate how it will work through the use of plugins finder. It doesn't imposes the way application store information about its installed plugins.

You can access the list of plugins managed by PluginManager through the getPlugins method. If you want to retrieve information about a loaded plugin you can use the getPlugin method to access it.

You can query the manager to see the list of plugin finders in use or even change this list.

It is also possible to detach a given plugin from the list of managed plugins (detach). This way the client takes the plugin ownership. Notice that it is not possible to detach a plugin on wich other plugins depend.

The method loadAll loads all the plugins installed under base plugins folder. PluginManager will check the dependency between plugins before trying to load them.

The other two load methods, with an array of PluginInfo and with just one PluginInfo, allows the class client to have a more fine grained control over which plugins are loaded and how they are loaded.

The unload methods accomplish the reverse task of the load methods.

This class has methods that allows applications to have three level of plugin management:

• in the first level applications doesn't provide information about how plugins are organized, they use the method loadAll without using the default plugin search strategy.

The AbstractFinder class allows applications to extend how PluginManager can search for plugins.

The method getPlugins can be re-implemented by subclasses in order to refine or even change how PluginManager searches for installed plugins.

There is a default finder implementation, named DefaultFinder. It keeps a list of base directories where PluginManager can search for plugins. By default it searches for plugins in some special directories defined a priori by a set of macros. The following order will be used:

1. in a default plugins dir defined by the macro TE_DEFAULT_PLUGINS_DIR (defaults to “plugins”) under application current dir;
2. if not found, then it searches for plugins dir (TE_DEFAULT_PLUGINS_DIR) under a folder defined by an environment variable that specifies the path to the TerraLib dir: TERRALIB_DIR_ENVIRONMENT_VARIABLE;

• if the above folder is not found, it gives-up searching for plugins and left the plugin list empty.

Even the DefaultFinder is very flexible and you can choose any location to be defined as a base location used by it to look for plugins through the method addPluginsDir. The getPluginsDir method shows the list of plugins base directories used by DefaultFinder.

For DefaultFinder, the getPlugins method performs a recursive search on the base directories extracting all plugins information available on these directories. The information about a given plugin is stored in a special XML file whose name is defined by the macro TE_DEFAULT_PLUGIN_FILE_NAME (defaults to plugin_info.xml - see the grammar below for the XML schema plugin_info.xsd).

The DefaultFinder also allows to retrieve information about just one plugin by informing its config file or base dir (getPlugin).

There is a default implementation for storing plugin information in XML files. The plugin_info.xsd defines the grammar that can be used to create information files about plugins.

<?xml version="1.0" encoding="UTF-8"?>
<xsd:schema xmlns:plugin="http://www.terralib.org/schemas/plugin"
            xmlns:common="http://www.terralib.org/schemas/common"
            xmlns:xlink="http://www.w3.org/1999/xlink"
            xmlns:xsd="http://www.w3.org/2001/XMLSchema"
            targetNamespace="http://www.terralib.org/schemas/plugin"
            elementFormDefault="qualified"
            attributeFormDefault="unqualified"
            xml:lang="en">
 
  <xsd:annotation>
    <xsd:appinfo>PluginInfo</xsd:appinfo>
    <xsd:documentation>This XML Schema describes the information about a given Terralib Plugin</xsd:documentation>
  </xsd:annotation>
 
  <xsd:import namespace="http://www.terralib.org/schemas/common" schemaLocation="../common/common.xsd"/>
  <xsd:import namespace="http://www.w3.org/1999/xlink" schemaLocation="../../ogc/xlink/1.0.0/xlinks.xsd"/>
 
  <xsd:element name="PluginInfo" type="plugin:PluginInfoType">
    <xsd:annotation>
      <xsd:documentation>This is the root element for plugin configuration files.</xsd:documentation>
    </xsd:annotation>
  </xsd:element>
 
  <xsd:complexType name="PluginInfoType">
    <xsd:annotation>
      <xsd:documentation>This type describes the plugin type.</xsd:documentation>
    </xsd:annotation>
    <xsd:sequence>
      <xsd:element name="Name" type="xsd:string" minOccurs="1" maxOccurs="1"/>
      <xsd:element name="DisplayName" type="xsd:string" minOccurs="1" maxOccurs="1"/>
      <xsd:element name="Description" type="xsd:string" minOccurs="1" maxOccurs="1"/>
      <xsd:element name="TerraLibVersion" type="xsd:string" minOccurs="1" maxOccurs="1"/>
      <xsd:element name="License" type="plugin:LicenseType" minOccurs="1" maxOccurs="1"/>
      <xsd:element name="Category" type="xsd:string" minOccurs="1" maxOccurs="1"/>
      <xsd:element name="Site" type="common:OnlineResourceType" minOccurs="1" maxOccurs="1"/>
      <xsd:element name="Provider" type="plugin:ProviderType" minOccurs="1" maxOccurs="1"/>
      <xsd:element name="RequiredPlugins" type="plugin:RequiredPluginsType" minOccurs="0" maxOccurs="1"/>
      <xsd:element name="RequiredPluginCategory" type="plugin:RequiredPluginCategoryType" minOccurs="0" maxOccurs="1"/>
      <xsd:element name="RequiredModules" type="plugin:RequiredModulesType" minOccurs="0" maxOccurs="1"/>
      <xsd:element name="Resources" type="plugin:ResourcesType" minOccurs="0" maxOccurs="1"/>
      <xsd:element name="Parameters" type="plugin:ParametersType" minOccurs="0" maxOccurs="1"/>
    </xsd:sequence>
    <xsd:attribute name="version" type="xsd:string" use="required"/>
    <xsd:attribute name="release" type="xsd:date" use="required"/>
    <xsd:attribute name="engine" type="xsd:string" use="required"/>
  </xsd:complexType>
 
  <xsd:complexType name="LicenseType">
    <xsd:annotation>
      <xsd:documentation>You can include a reference to an on-line resource from where the license can be accessed.</xsd:documentation>
    </xsd:annotation>
    <xsd:simpleContent>
      <xsd:extension base="xsd:string">
        <xsd:attributeGroup ref="xlink:simpleLink"/>
      </xsd:extension>
    </xsd:simpleContent>
  </xsd:complexType>
 
  <xsd:complexType name="ProviderType">
    <xsd:sequence>
      <xsd:element name="Name" type="xsd:string" minOccurs="1" maxOccurs="1"/>
      <xsd:element name="Site" type="common:OnlineResourceType" minOccurs="1" maxOccurs="1"/>
      <xsd:element name="Email" type="xsd:string" minOccurs="1" maxOccurs="1"/>
    </xsd:sequence>  
  </xsd:complexType>
 
  <xsd:complexType name="RequiredPluginsType">
    <xsd:sequence>
      <xsd:element name="PluginId" type="xsd:string" minOccurs="1" maxOccurs="unbounded"/>
    </xsd:sequence>  
  </xsd:complexType>
 
  <xsd:complexType name="RequiredPluginCategoryType">
    <xsd:sequence>
      <xsd:element name="CategoryId" type="xsd:string" minOccurs="1" maxOccurs="unbounded"/>
    </xsd:sequence>  
  </xsd:complexType>
 
  <xsd:complexType name="RequiredModulesType">
    <xsd:sequence>
      <xsd:element name="ModuleId" type="xsd:string" minOccurs="1" maxOccurs="unbounded"/>
    </xsd:sequence>  
  </xsd:complexType>
 
  <xsd:complexType name="ResourcesType">
    <xsd:sequence>
      <xsd:element name="Resource" type="plugin:ResourceType" minOccurs="1" maxOccurs="unbounded"/>
    </xsd:sequence>  
  </xsd:complexType>
 
  <xsd:complexType name="ResourceType">
    <xsd:annotation>
      <xsd:documentation>If your plugin need to track any resource you can use a resource to point to it.</xsd:documentation>
    </xsd:annotation>
    <xsd:attribute name="name" type="xsd:string" use="required"/>
    <xsd:attributeGroup ref="xlink:simpleLink"/>
  </xsd:complexType>
 
  <xsd:complexType name="ParametersType">
    <xsd:sequence>
      <xsd:element name="Parameter" type="plugin:ParameterType" minOccurs="1" maxOccurs="unbounded"/>
    </xsd:sequence>  
  </xsd:complexType>
 
  <xsd:complexType name="ParameterType">
    <xsd:sequence>
      <xsd:element name="Name" type="xsd:string" minOccurs="1" maxOccurs="1"/>
      <xsd:element name="Value" type="xsd:string" minOccurs="1" maxOccurs="1"/>
    </xsd:sequence>  
  </xsd:complexType>
 
</xsd:schema>

These grammar accounts for plugins dependency. In this case the dependency between all plugins must form a directed acylic graph (DAG) otherwise it couldn't be resolved.

For example, a given plugin could be defined as:

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<PluginInfo xmlns:xlink="http://www.w3.org/1999/xlink"
            xmlns:xsd="http://www.w3.org/2001/XMLSchema-instance"
            xmlns="http://www.terralib.org/schemas/plugin"
            xsd:schemaLocation="http://www.terralib.org/schemas/plugin plugin_info.xsd"
            version="5.0.0-20110110"
            release="2011-01-01"
            engine="C++">
  <Name>te.da.pgis</Name>
  <DisplayName>PostGIS Data Source Driver</DisplayName>
  <Description>This plugin enables TerraLib to access data in a PostgreSQL DBMS using the PostGIS extension</Description>
  <TerraLibVersion>5.0.0-20100101</TerraLibVersion>
  <License xlink:href="http://www.gnu.org/licenses/lgpl-3.0-standalone.html">GNU Lesser General Public License v 3.0</License>
  <Category>Data Access</Category>
  <Site xlink:href="http://www.dpi.inpe.br/terralib5/wiki/doku.php?id=wiki:designimplementation:dataaccess:postgis"/>
  <Provider>
    <Name>Terralib Team</Name>
    <Site xlink:href="http://www.terralib.org/"/>
    <Email>terralib-team@terralib.org</Email>
  </Provider>
  <RequiredPlugins>
    <PluginId>te.plugin</PluginId>
  </RequiredPlugins>
  <RequiredModules>
    <ModuleId>te.da</ModuleId>
  </RequiredModules>
  <Resources>
    <Resource name="SharedLibraryName" xlink:href="terralib_postgis"/>
    <Resource name="SQLDialectFile" xlink:href="postgis_dialect.xml"/>
    <Resource name="LangFile" xlink:href="pt.br"/>
    <Resource name="HelpFile" xlink:href="pgis.html"/>
  </Resources>
  <Parameters>
    <Parameter>
      <Name>DEFAULT_DATASET_CACHE_SIZE</Name>
      <Value>5000</Value>
    </Parameter>
    <Parameter>
      <Name>CLIENT_ENCODING</Name>
      <Value>LATIN1</Value>
    </Parameter>
    <Parameter>
      <Name>InitialPoolSize</Name>
      <Value>1</Value>
    </Parameter>
    <Parameter>
      <Name>MinPoolSize</Name>
      <Value>1</Value>
    </Parameter>
    <Parameter>
      <Name>MaxPoolSize</Name>
      <Value>1</Value>
    </Parameter>
  </Parameters>
</PluginInfo>

Although this framework is very flexible and extensible there will be cases where the programmer wants to have a finner control over how the singleton plugin manager works or how the plugin engines loads its plugins or how a plugin is represented. For all those cases you have solutions:

• PluginManager:
  • you can tell where it must search for plugins
  • you can just ask it to tell what are the available plugins
  • you can ask information about one particular plugin
  • you can just load and not start a plugin.
  • you can add a new plugin engines and it will automatically makes use of it
  • you can use plugin manager to load a plugin but not manage the plugin
  • you can detach a plugin taking its ownership
• Plugin Engine:
  • you can make new plugin engines for new languages
  • you can refine a given plugin engine like the C++ plugin engine or Lua plugin engine to be more specific, adding new implementaions. this will be transparent to the plugin manager.
• Plugins can share code
• Plugins can be linked to other plugins
• Plugins can receive parameters at runtime in order to be configured

• The dependency among plugins must be modeled by a DAG:
  • the load method makes a topolgical sort before loading the plugins and it detects any inconsistence on the dependencies.
• A plugin name must be unique:
  • suggestion: use a URI with the namespace name for the plugin. Example: te.da.pgis or te.pgis.
• A library is unloaded just when all plugins referencing it has been unloaded:
  • This will enable multiple plugins for a single library.
• Plugins can have its own versioning system:
  • actually indicated by the release date.
  • this will enable fixing a plugin without the need of a new TerraLib version.
• In the case of C++ plugins the shared libraries must append a “_d” suffix for the debug versions:
  • this way avoid troubles when mixing the debug and release modes.
• …to be done…

In order to create a C++ plugin you must implement a subclass of Plugin as shown in the code snippet:

class HelloWorldPlugin : public te::plugin::Plugin
{
  public:
 
    HelloWorldPlugin(const te::plugin::PluginInfo& pluginInfo);
 
    ~HelloWorldPlugin();
 
    /*!
      \brief This method will be called by TerraLib to startup any plugin's functionality.
 
      \exception Exception It throws and exception if the plugin can not be started.
     */
    void startup() throw(...);
 
    /*!
      \brief This method will be called by TerraLib to shutdown plugin's functionality.
 
      \exception Exception It throws and exception if the plugin can not be shutdown.
     */
    void shutdown() throw(...);
};
 
// needed by Windows in order to export DLL information
#ifdef WIN32
  #define TECPPPLUGINHELLOWORLDEXPORT  __declspec(dllexport) 
#else
  #define TECPPPLUGINHELLOWORLDEXPORT
#endif
 
// register plugin
PLUGIN_CALL_BACK_DECLARATION(TECPPPLUGINHELLOWORLDEXPORT);

HelloWorld.cpp:

#include "HelloWorld.h"
#include <iostream>
 
HelloWorldPlugin::HelloWorldPlugin(const te::plugin::PluginInfo& pluginInfo)
  : te::plugin::Plugin(pluginInfo)
{
  std::cout << std::endl << "C++ plugin HelloWorldPlugin loaded!" << std::endl;
}
 
HelloWorldPlugin::~HelloWorldPlugin()
{
  std::cout << std::endl << "C++ plugin HelloWorldPlugin saying bye bye!" << std::endl;
}
 
void HelloWorldPlugin::startup()
{
  std::cout << std::endl << "C++ plugin HelloWorldPlugin started!" << std::endl;
}
 
void HelloWorldPlugin::shutdown()
{
  std::cout << std::endl << "C++ plugin HelloWorldPlugin shutdown!" << std::endl;
}
 
PLUGIN_CALL_BACK_IMPL(HelloWorldPlugin)

Then you can provide a configuration file called plugin_info.xml as follow:

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<PluginInfo xmlns:xlink="http://www.w3.org/1999/xlink"
            xmlns:xsd="http://www.w3.org/2001/XMLSchema-instance"
            xmlns="http://www.terralib.org/schemas/plugin"
            xsd:schemaLocation="http://www.terralib.org/schemas/plugin ../../schemas/terralib/plugin/plugin_info.xsd"
            version="5.0.0-20110110"
            release="2011-01-01"
            engine="C++">
  <Name>te.hello_world_cpp</Name>
  <DisplayName>Hello World Plugin in C++</DisplayName>
  <Description>This plugin shows how it is easy to create C++ plugins with TerraLib</Description>
  <TerraLibVersion>5.0.0-20100101</TerraLibVersion>
  <License xlink:href="http://www.gnu.org/licenses/lgpl-3.0-standalone.html">GNU Lesser General Public License v 3.0</License>
  <Category>Examples</Category>
  <Site xlink:href="http://www.dpi.inpe.br/terralib5/wiki/doku.php?id=wiki:designimplementation:plugin#writing_c_plugins"></Site>
  <Provider>
    <Name>Terralib Team</Name>
    <Site xlink:href="http://www.terralib.org/"></Site>
    <Email>terralib-team@terralib.org</Email>
  </Provider>
  <Resources>
    <Resource name="SharedLibraryName" xlink:href="terralib_example_plugin_cpp_hello_world"></Resource>
  </Resources>
</PluginInfo>

HelloWorld.lua:

HelloWorld.js:

Some open questions:

• Windows: as there is not a standard library organization, is it ok to have a plugin sub-dir under bin/msvc and sub-sub-folders for each plugin?
• Linux: what is the most convenient way to organize the plugins by default?
• Apple: what is the most convenient way to organize the plugins by default?