iota2 feature maps ################## Assuming iota2 is :doc:`fully operational `, this chapter presents a usage of iota2: the production of features maps using satellite images time series. Introduction to data ******************** iota2 handles several sensors images : * Landsat 5 and 8 (old and new THEIA format) * Sentinel 1, Sentinel 2 L2A(THEIA and Sen2cor), Sentinel L3A (THEIA Format) * Various other images already processed, with the ``userFeat`` sensor In this chapter, only the use of Sentinel 2 L2A will be illustrated. To use other sensors, it is necessary to adapt the input parameters. To produce spectral indices, a user can use all :ref:`spectral indices already computed by iota2 ` and provide python code to produce custom features. Get the dataset ================ .. include:: i2_tutorial_dataset_description.rst Understand the configuration file ================================= iota2 exploits hundreds of parameters, native or coming from other libraries such as scikit-learn or OTB. These combinations of parameters will make it possible to choose the processes to be carried out, the choice of the algorithms used or which output to write. A documentation of all these parameters is available: :doc:`parameters descriptions `. To simplify the use, iota2 read a configuration file which is a simple text file containing sections and fields. The minimal configuration file contains all required fields to produce a land cover map. .. include:: examples/config_tutorial_features_maps.cfg :literal: For an end user, launching iota2 requires to fill correctly the configuration file. In the above example, replace the ``XXXX`` by the path where the archive has been extracted. Running the chain ***************** iota2 launch ============ To launch the chain, simply use the following command line: .. code-block:: console Iota2.py -config /XXXX/IOTA2_TESTS_DATA/i2_tutorial_features_map.cfg First, the chain displays the list of all steps activated by the configuration file: .. include:: examples/steps_features_maps.txt :literal: Once the processing start, a large amount of information will be printed, most of them concerning the dask-scheduler. Did it all go well? =================== iota2 displays a lot of information during execution. It becomes difficult to identify if everything went well or if an error occurred. To this end, in the ``output_path`` a log directory is available : ``output_path/logs``. The logs directory is ordered as: .. code-block:: console ├── CommonMasks │ ├── common_mask_T31TCJ.err │ └── common_mask_T31TCJ.out ├── html │ └── * ├── MergeFeaturesMaps │ ├── merge_feat_maps.err │ └── merge_feat_maps.out ├── MergeFeaturesMapsByTiles │ ├── merge_feat_maps_by_tiles_T31TCJ.err │ └── merge_feat_maps_by_tiles_T31TCJ.out ├── run_informations.txt ├── SensorsPreprocess │ ├── preprocessing_T31TCJ.err │ └── preprocessing_T31TCJ.out ├── tasks_status_I2FeaturesMap_1.svg ├── tasks_status_I2FeaturesMap_2.svg └── WriteFeaturesMap ├── write_features_T31TCJ_chunk_0.err ├── write_features_T31TCJ_chunk_0.out ├── write_features_T31TCJ_chunk_10.err ├── write_features_T31TCJ_chunk_10.out ├── write_features_T31TCJ_chunk_11.err ├── write_features_T31TCJ_chunk_11.out ├── : : : ├── : : : ├── write_features_T31TCJ_chunk_8.err ├── write_features_T31TCJ_chunk_8.out ├── write_features_T31TCJ_chunk_9.err └── write_features_T31TCJ_chunk_9.out Each step has its own log directory. In these directories, two kinds of logs can be found: standard output and errors. The errors are compiled in "\*.err" files and the standard output in "\*.out" files. With the dask scheduler, iota2 goes as far as possible while the data required for the next steps is available. To simplify the error identification, an interactive graph is produced in a html page. Nodes in graph can have three colors (red: error, blue: done, white: not yielded). By clicking on graph node, the corresponding log file is opened. If despite all this information, the errors can not be identified or solved, the iota2 team can help all users. The simplest way to ask help is to create an issue on `framagit `_ by adding the archive available in log directory. Output tree structure ===================== In this section, all output folders are described. .. code-block:: console ├── by_tiles │   └── T31TCJ.tif ├── customF │   ├── T31TCJ_chunk_0.tif │   ├── T31TCJ_chunk_10.tif │   ├── T31TCJ_chunk_11.tif │   ├── T31TCJ_chunk_12.tif │   ├── T31TCJ_chunk_13.tif │   ├── T31TCJ_chunk_14.tif │   ├── T31TCJ_chunk_15.tif │   ├── T31TCJ_chunk_16.tif │   ├── T31TCJ_chunk_17.tif │   ├── T31TCJ_chunk_18.tif │   ├── T31TCJ_chunk_19.tif │   ├── T31TCJ_chunk_1.tif │   ├── T31TCJ_chunk_2.tif │   ├── T31TCJ_chunk_3.tif │   ├── T31TCJ_chunk_4.tif │   ├── T31TCJ_chunk_5.tif │   ├── T31TCJ_chunk_6.tif │   ├── T31TCJ_chunk_7.tif │   ├── T31TCJ_chunk_8.tif │   └── T31TCJ_chunk_9.tif ├── features │   └── T31TCJ │   └── tmp │   ├── MaskCommunSL.dbf │   ├── MaskCommunSL.prj │   ├── MaskCommunSL.shp │   ├── MaskCommunSL.shx │   ├── MaskCommunSL.tif │   ├── Sentinel2_T31TCJ_input_dates.txt │   ├── Sentinel2_T31TCJ_interpolation_dates.txt │   └── Sentinel2_T31TCJ_reference.tif ├── final │   └── features_map.tif ├── IOTA2_tasks_status.txt ├── logs │   ├── * ├── logs.zip └── T31TCJ customF ------- Temporary data used to reassemble maps for each tile. Can be removed by hand by user at the end of the processing. by_tiles -------- The tile chunks are merged to produce a map for each tile. final ----- This folder contains the final products of features map builder. features -------- For each tile, contains useful information: - ``tmp/MaskCommunSL`` : the mask of all sensors for this tile - ``tmp/Sentinel2_T31TCJ_reference.tif`` : the image used for reprojecting data - ``tmp/Sentinel2_T31TCJ_input_dates.txt`` : the list of date detected in ``s2_path`` - ``tmp/Sentinel2_T31TCJ_interpolation_dates.txt`` : the list of output interpolation dates Final products ============== Features map ------------ .. figure:: ./Images/Features_map.png :scale: 50 % :align: center :alt: features map Features_map.tif Example To go further ============= .. toctree:: :maxdepth: 1 External features .. raw:: html :file: interactive-tree.html