TerraLib and TerraView Wiki Page

You are here: start » wiki » tclt

This is an old revision of the document!

Warning: Declaration of syntax_plugin_iframe::handle($match, $state, $pos, &$handler) should be compatible with DokuWiki_Syntax_Plugin::handle($match, $state, $pos, Doku_Handler $handler) in /var/www/html/terralib5/wiki/lib/plugins/iframe/syntax.php on line 18

Warning: Declaration of syntax_plugin_iframe::render($mode, &$R, $data) should be compatible with DokuWiki_Syntax_Plugin::render($format, Doku_Renderer $renderer, $data) in /var/www/html/terralib5/wiki/lib/plugins/iframe/syntax.php on line 18

Warning: Declaration of syntax_plugin_externallink::handle($match, $state, $pos, &$handler) should be compatible with DokuWiki_Syntax_Plugin::handle($match, $state, $pos, Doku_Handler $handler) in /var/www/html/terralib5/wiki/lib/plugins/externallink/syntax.php on line 107

Warning: Declaration of syntax_plugin_externallink::render($mode, &$renderer, $data) should be compatible with DokuWiki_Syntax_Plugin::render($format, Doku_Renderer $renderer, $data) in /var/www/html/terralib5/wiki/lib/plugins/externallink/syntax.php on line 107

TCLT (TerraLib Command Line Tools)

TCLT (TerraLib Command Line Tools)

About

A set of TerraLib based command line tools to perform automated raster/vector/tabular data processing.

Main features:

Itś a Free Sofware !
No programming languages knowledge required !
Support for command line based execution of one operation.
Support for script/project based execution (allows the creation of complex graph-based chains of execution using a multi-threaded environment).
Support for directory recursive processing.
Support to handle and process large remote sensing data sets.
Raster and vector data processing.
Support for multiple processing cores (multi-threading).
Support for all TerraLib/GDAL data set formats.

Current processing vertices:

Vertice ID	Description	Notes
ARITHMETIC	A vertice capable of performing arithmetic operations over raster data.
COLOR_TRANSF	A vertice designed to perform color space conversion.	Supported transformation mathods: RGB → IHS(Intensity, Hue, Saturation) , IHS → RGB, RGB → HSL(Hue, Saturation, Lightness), HSL → RGB, PCA(Principal Components Analysis), PCA_INV(Inverse Principal Components Analysis)
COMPOSITION	A vertice to create a composition (a mosaic created using user defined rules) from a set of geo-referenced rasters.
CONDITIONAL	A vertice to execute an action if the given condition is satisfied or execute an counter-action otherewise
CONTRAST	A vertice designed to perform raster data contrast enhancement.	Supported contrast types: LINEAR, HIST_EQ (Histogram Equalization), SQUARE, SQUARE_ROOT, LOG, MEAN_STD (The contrasted image will have a predefined mean and standard deviation normalization), DEC_ENH (Decorrelation Enhancement using principal components),2_PERC_CUT(Linear cumulative cutted in 2 - 98%)
DATASET_CLASSIFIER	Generic data sets classifier vertice.
FILTER	A vertice to apply a series of well-known filtering algorithms for images, linear and non-linear.	Supported filters: SOBEL, ROBERTS, MEAN, MODE, MEDIAN, DILATION, EROSION.
FUSION	A vertice to combine pixel information from different sensors to obtain better spatial or radiometric resolutions.	Supported methods: IHS - Fusion of 3 band low resolution raster with a high resolution raster, PCA - Fusion of a multispectral low resolution raster (no bands limit) with a high resolution raster
GEODMA	A vertice to execute GeoDMA operations	GeoDMA is a toolbox for integrating remote sensing imagery analysis methods with data mining techniques producing a user-centered, extensible, rich computational environment for information extraction and knowledge discovery over large geographic databases. For more info click here.
LINK	A vertice to create a link/reference to the output of a vertice from another analysis within the same project.
MIXMODEL	A raster pixels mixer model vertice.	Supported model types: PCA(Principal Components Analysis), LINEAR.
MOSAIC	A vertice to create a mosaic from a rasters set.	Virtual mosaic supported.
MSC	A vertice to execute MSC (Mono or Multi-source classifier)	Reference:Sistema de classificação de imagens mono e multifontes baseado em distâncias estocásticas e testes de hipóteses associados. Michael Douglas Lemes dos Santos, Sidnei João Siqueira Sant'Anna. For more info about MSC - Click here
OUT_DEMUX	A vertice to demux the output of a input vertice result of type COLLECTION_TYPE (A collection of many outputs).
RASTERCLIPPING	A vertice to clip raster data.
RASTERTRANSFORM	A vertice to transform raster data.
RASTER_CLASSIFIER	A raster data classifier vertice.	Classifier methods: ISOSEG, MAP, ISODATA.
RASTER_SPLIT	A vertice to split one input raster into many output rasters.
RECOMPOSE	A vertice designed to compose one output raster using input rasters bands.
REGISTER	A vertice designed to Perform raster data registering into a SRS using a set of tie points.
VEC_REGISTER	A vertice designed to Perform vector data registering into a SRS using a set of tie points.
SCRIPT	A vertice that allows the execution of an external script or command.
SEGMENTER	A vertice capable of creating segments (homogeneous areas) from an input image.	Supported segmentation algorithms: MEAN(Reference: Satellite Imagery Segmentation: a region growing approach, Bins, L.; Fonseca, L.; Erthal, G.), BAATZ(Reference: Multiresolution segmentation: an optimization approach for high quality multi-scale image, Martin BAATZ und Arno SCHÄPE)
SELECT	A vertice designed to copy only selected data from input to output.	Supported methods: statistical and local low-pass filter.
STRIPING	A vertice to perform destriping related algorithms over a set of raster data.
TERRAHIDRO	A vertice execute TerraHidro Operations.	Supported operations: CAVE_FLAT_AREAS_V_SHAPED-Finds and carves flat areas from a DEM input raster in a V-shaped format. PFS_PIT_REMOVAL-Remove all pits using the PFS algorithm to carve a path to an outlet cell. Generates a pitless DEM. SIMPLE_PIT_REMOVAL-Remove simple pits from a DEM(Simple pits are removing change the altimetria of just one cell). D8_FLOW-Generates the D8 flow directions from a pitless DEM (The output directions is valid only in column and row range from 2 cells of the DEM borders. D8_CONTRIB_AREA-Generates the drainage network grid from a contributing area grid (Each cell of the contributing area raster has the value of the contributing area of the cell.). D8_DRAINAGE-Generate the drainage raster lines from a contributing area raster. It is a threshold of contributing area. D8_HAND-Generates the drainage network grid from a contributing area grid. For more info about TerraHidro - Click here
TPLOCATOR	A vertice designed to find homologous tie-points inside raster areas.	Supported strategies: MORAVEC(Moravec Based Corner Detection. Reference: H. P. Moravec. Towards Automatic Visual Obstacle Avoidance), SURF(Speeded up robust features. Reference: Herbert Bay, Andreas Ess, Tinne Tuytelaars, and Luc Van Gool, Speeded Up Robust Features).
VEC_TPLOCATOR	A vertice designed to find tie-points using vector data
URI	A vertice to insert a context URI into the graph flow.
URI_EDIT	A vertice to edit a URI.
VECTORIZER	A vertice capable of vectorizing raster data.
WATERDETECT	A vertice capable of detecting water on raster data.	Reference: WATER BODY EXTRACTION FROM RAPIDEYE IMAGES: AN AUTOMATED METHODOLOGY BASED ON HUE COMPONENT OF COLOR TRANSFORMATION FROM RGB TO HSV MODEL, Laercio Massaru Namikawa, Thales Sehn Körting, Emiliano Ferreira Castejon. Full Text - Click here

News

12/20/2024 - Fixing and updating vertices: vectorizer, fusion, mosaic, register.
04/25/2024 - New contrast modes. New GRDB database tie-points locationg support. Now using a new Terralib version (5.7.1).
12/11/2023 - New command line options to automatically delete a project output. New Arithmetic vertice parameters to control the output raster data type. New raster mosaic parameters to control input bands. Output log improvements. Minor stability fixes.
01/11/2022 - Improvements on recursive processing time filter. Optimizations: raster transform, tie-points locator. New vertices: VEC_TIEPOINTSLOCATOR, VEC_REGISTER.
07/06/2022 - RECOMPOSE and RASTER_SPLIT vertices now can create virtual rasters. URI_EDIT updated. New Fusion histogram fitting method. SCRIPT vertice updated. New recursive filter by file modification time.
01/14/2022 - New RASTER_SPLIT and CONDITIONAL vertices, some minor fixes, new logging scheme.
09/01/2021 - New COMPOSITION and URI_EDIT vertices, new option of filter recursive execution files.
03/11/2021 - New GEODMA vertice.
02/26/2021 - New STRIPING vertice.
01/07/2021 - New RASTER TRANSFORM vertice.
08/07/2020 - New virtual mosaic option added to the mosaic vertice. All raster classifiers merged into one vertice. New DataSet classifier vertice. New LINK vertice.
06/14/2019 - New execution vertices added: MSC. HAND algorithm added to the TERRAHIDRO vertice.
02/7/2019 - New execution vertices added: FUSION, TERRAHIDRO
01/17/2019 - New execution vertices added: COLOR_TRANSF, CONTRAST, OUT_DEMUX, RECOMPOSE
12/05/2018 - Project files syntax updated.
12/05/2018 - New execution vertice added: SELECT.
7/31/2018 - New execution mode added: Directory recursive processing.
7/31/2018 - New execution vertices added: Register and TPLocator (Tie-Points locator).
5/30/2018 - New execution vertice added: WaterDetect.
5/25/2018 - New execution vertice added: Mosaic.
4/04/2018 - New execution vertices added: Arithmetic, Filter, MAP.
3/07/2018 - New execution vertices added: Mixture Model, Raster Clipping, External Script.
1/24/2018 - Added a IsoSeg Classifier.
1/24/2018 - Added Region Growing image segmenters (Mean, Baatz).
1/24/2018 - Added a raster data vectorizer.

Help

How to list all options

For a list of all available options run the executable file with the option “–help”. The executable must be called from a terminal window (Windows command prompt - CMD).

Example:

tclt_exe.exe --help

Single operation execution - This mode allows the execution of one operation (vertice) on each call.

Example: Apply a Mean segmentation (simple region growing) operation over an input image file named “image.tif” (located inside directory “c:\images”) and write the output inside the same directory.

tclt_exe.exe --processing_type="SEGMENTER" --input_uri="INPUT_RASTER;file://c:\image.tif" --processing_parameter="STRATEGY;MEAN" --output_directory="c:\images"

Project execution - This mode allows the execution multiple chained operations using a project.

Important concepts for this execution mode:

Project - A container for all analysis, graphs and contexts. Multiple analysis, graphs and contexts can be declared.
Context - A set of data sets related to the same geographic area.
Graph - A set of vertices (processing nodes) organized in a graph form and executed in a pre-determined way.

Notes: The same graph can be used with multiple distinct analysis. This is possible because all graph input data are references to URIs aliases provided by contexts. In this case, all used contexts must provide the same URI aliases.

This execution mode requires the creation of 3 files:

A project file (project_example.txt):

# Add one or more context files to this project
# Paths are relative to the project file location
INCLUDE_FILE "context_example.txt"
# Add one or more graph files to this project
# Paths are relative to the project file location
INCLUDE_FILE "graph_example.txt"
PROJECT_START
  # The global project name
  # Allowed characters: _ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789
  PROJECT_NAME "Sample_Project_Name_1"
  # Add one or more analysis to be executed
  # FORMAT: ANALYSIS "Analysys_Name" "Context_Name" "Graph_Name"
  ANALYSIS "Analysys_Name_1" "Context_Name_1" "Graph_Name_1"
PROJECT_END

A context file (graph_example.txt):

CONTEXT_START
  # This context name
  # Allowed characters: _ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789
  CONTEXT_NAME "Context_Name_1"
  # Add one or more resouces to this context
  # General Format: RESOURCE_URI "URI_alias" "URI string"
  #
  # Files Format: File paths are relative to the context file location.
  # Example: RESOURCE_URI "URI_alias" "file://../file_name.tif"
  RESOURCE_URI "Raster1ContextAlias" "file://../rasters/cbers_rgb342_crop1.tif"
CONTEXT_END

A graph file (graph_example.txt):

GRAPH_START
  # This graph name
  # Allowed characters: _ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789
  GRAPH_NAME "Graph_Name_1"
  # General vertice syntax:
  #
  #    VERTICE_START
  #
  #      # This vertice related class name
  #      # Allowed characters: _ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789
  #      VERTICE_NAME "Vertice_name"
  #
  #      # Vertice type
  #      VERTICE_TYPE "VERTICETYPE"
  #
  #      # Vertice output cache enabled (YES/NO)
  #      VERTICE_OUT_CACHE "NO"
  #      # Declares connections to other vertices outputs and gives them name aliases.
  #      # General format: VERTICE_CONNECTION "other_vertice_name" "connection alias"
  #      VERTICE_CONNECTION "other_vertice_name_1" "connection alias 1"
  #      VERTICE_CONNECTION "other_vertice_name_2" "connection alias 2"
  #
  #      # Declares a input URI resouce used by this vertice.
  #      # General format: VERTICE_RESOURCE "context resource URI alias" "vertice URI alias"
  #      VERTICE_RESOURCE "raster resource 1" "image"
  #
  #      # One or more specific vertice parameters
  #      # General format: VERTICE_PARAM "parameter name" "parameter value"
  #      VERTICE_PARAM "parameter name 1" "parameter value 1"
  #      VERTICE_PARAM "parameter name 2" "parameter value 2"
  #
  #    VERTICE_END
  VERTICE_START
    VERTICE_NAME "Original_image_vertice"
    VERTICE_TYPE "URI"
    VERTICE_OUT_CACHE "NO"
    VERTICE_RESOURCE "Raster1ContextAlias" "URI_ALIAS"
  VERTICE_END
  VERTICE_START
    VERTICE_NAME "Segmented_image"
    VERTICE_TYPE "SEGMENTER"
    VERTICE_OUT_CACHE "NO"
    VERTICE_CONNECTION "Original_image_vertice" "INPUT_RASTER"
    VERTICE_PARAM "STRATEGY" "MEAN"
    VERTICE_PARAM "MIN_SEGMENT_SIZE" "100"
    VERTICE_PARAM "SEGMENTS_SIMILARITY_THRESHOLD" "0.03"
  VERTICE_END
GRAPH_END

The following command must be executed in order to run the entire project and write the result to a directory “c:\images”:

tclt_exe.exe --project_file_name="project_example.txt" --output_directory="c:\images"

Directory Recursive Execution - This execution mode allows to process all data within a given directory.

The same project will be executed for each input data and the result will be written on distinct output directories.

On each execution a specific context/resource is replaced by a file URI from the given directory.

Example - To execute image segmentation on all files within a given directory, called “c:\my_input_images”, using the same project/context/graph files described above:

tclt_exe --project_file_name="project_example.txt" --output_directory="c:\images" --recursive_exec_dir="c:\my_input_images" --recursive_exec_context="Context_Name_1;Raster1ContextAlias" --recursive_exec_file_filters="+.tif;-.xml"

Trace: • tclt