Sencast ================== .. image:: logo.png :width: 120px :alt: Eawag logo :align: left Sencast is a toolbox to download and derive water quality parameters from satellite images. It acts as a framework for the use a variety of processors such as Idepix, Polymer and Acolite. It supports ESA satellites Sentinel 2 and Sentinel 3 and USGS satellite Landsat 8. It is developed and maintained by the `SURF Remote Sensing group at Eawag`_. .. warning:: Sencast is under active development. Please contact us if you find bugs or issues. Publications ------------- | **SenCast: Copernicus Satellite Data on Demand** | *D. Odermatt, J. Runnalls, J. Sturm, A. Damm* | `German `_ `English `_ Installation ------------- To install Sencast, run .. code-block:: bash git clone https://github.com/eawag-surface-waters-research/sencast.git conda env create -f ~/sencast/sencast.yml Many of the Sencast's processors reply on `SNAP`_ , the Sentinel Application Platform project, funded by the `European Space Agency`_ (ESA) or other 3rd party packages. In order to have access to all of Sencast's processors follow the installation instructions below in order to correctly configure your environment. This process will require registering accounts with data providers. .. toctree:: :maxdepth: 2 :caption: Installation install/linux.rst install/windows.rst For issues with installation, please contact `Daniel Odermatt`_. .. warning:: It can be difficult and time-consuming to get a local installation set up (particularly for Windows). For users not planning on developing the code it is recommended to use the Docker image provided (see below). Getting Started --------------- Sencast can be run in two ways: 1. By calling the ``main.py`` script with command line arguments .. code-block:: python python main.py -p parameters.ini -e environment.ini +-------------------+---------------------+-------------------------------------------------------------------+ | Parameters | Default | Description | +===================+=====================+===================================================================+ | -t --tests | False | run test processing to check setup | +-------------------+---------------------+-------------------------------------------------------------------+ | -x --delete_tests | False | delete previous test run | +-------------------+---------------------+-------------------------------------------------------------------+ | -p --parameters | Required | link to the parameters.ini file (required except for tests) | +-------------------+---------------------+-------------------------------------------------------------------+ | -e --environment | ${machine-name}.ini | link to the environment.ini file | +-------------------+---------------------+-------------------------------------------------------------------+ | -d --downloads | 1 | number of parallel downloads | +-------------------+---------------------+-------------------------------------------------------------------+ | -r --processors | 1 | number of parallel processors | +-------------------+---------------------+-------------------------------------------------------------------+ | -a --adapters | 1 | number of parallel adapters | +-------------------+---------------------+-------------------------------------------------------------------+ 2. By importing Sencast as a function .. code-block:: python import sys sys.path.append("/sencast") # Path to sencast repository from main import sencast sencast(params_file, env_file=None, max_parallel_downloads=1, max_parallel_processors=1, max_parallel_adapters=1) For this options you can pass objects as the params_file and env_file as well as links to the text files. Environment File ~~~~~~~~~~~~~~~~~~~~~~ Environment files use the INI format and contain the configuration of the machine on which Sencast runs. Refer to :ref:`environments` for details on how to set up your own environment file. You should create your own environment file for every machine you install Sencast on. Parameter File ~~~~~~~~~~~~~~~~~~~~~~ Parameter files use the INI format and contain the parameters for one execution of Sencast. Refer to :ref:`parameters` for details on how to set up your own parameter file. Perimeter Definition ~~~~~~~~~~~~~~~~~~~~~~ Perimeter definitions define a geographic area to be processed by Sentinel Hndacast. They are stored as polygons in `WKT`_ files, which are referenced from the parameter files. Some example perimeters are stored in the wkt folder. Testing -------- To test your installation run:: cd ~/sencast conda activate sencast python main.py -t This will report which processors are successfully installed and producing meaning-full outputs. Docker ------- Manual installation of all the processors is challenging and can be simplified through the use of a docker container. Users should first ensure they have a functioning docker installation. Pull container ~~~~~~~~~~~~~~~~ The docker image can be downloaded from docker hub using the following command: .. code-block:: bash docker pull eawag/sencast:latest Run Tests ~~~~~~~~~~~ In order to test the setup is working the following command can be run which will output a report on the functioning of the processors. **This must be run from inside the sencast repository.** The option `-v /DIAS:/DIAS` maps the input/ output folders to a location outside the container. This should be updated to the appropriate location, e.g. `-v /home/user/DIAS:/DIAS` .. code-block:: bash docker run -v /DIAS:/DIAS -v $(pwd):/sencast --rm -it eawag/sencast:latest -e docker.ini -t `-e` name of the environment file in `sencast/environments` `-t` flag to indicate a test should be run Run script ~~~~~~~~~~~~~ In order to run a parameters file it can be passed to the command as follows using the `-p` flag. .. code-block:: bash docker run -v /DIAS:/DIAS -v $(pwd):/sencast --rm -it eawag/sencast:latest -e docker.ini -p example.ini `-p` name of the parameter file in `sencast/parameters` Run Interactive Container ~~~~~~~~~~~~~~~~~~~~~~~~~~~ Sometimes it is desirable to interact directly with the container, this can be achieved with the following command .. code-block:: bash docker run -v /DIAS:/DIAS -v $(pwd):/sencast --rm -it --entrypoint /bin/bash eawag/sencast:latest Locally build container ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ .. code-block:: bash docker build -t eawag/sencast:latest . .. toctree:: :maxdepth: 2 :caption: CSCS cscs.rst .. toctree:: :maxdepth: 2 :caption: Configuration environment_config.rst parameters_config.rst .. toctree:: :maxdepth: 2 :caption: Sencast main.rst .. toctree:: :maxdepth: 2 :caption: Utilities utils/auxil.rst utils/product_fun.rst .. toctree:: :maxdepth: 2 :caption: Processors processors/acolite.rst processors/c2rcc.rst processors/collection.rst processors/forelule.rst processors/idepix.rst processors/mph.rst processors/oc3.rst processors/polymer.rst processors/primaryproduction.rst processors/secchidepth.rst processors/tmart.rst processors/whiting.rst .. toctree:: :maxdepth: 2 :caption: Adapters adapters/geotiff.rst adapters/merge.rst adapters/pixelextraction.rst adapters/qlrgb.rst adapters/qlsingleband.rst .. toctree:: :maxdepth: 2 :caption: DIAS API's apis/coah.rst apis/creodias.rst apis/earthdata.rst apis/eros.rst apis/hda.rst .. _SURF Remote Sensing group at Eawag: https://www.eawag.ch/en/department/surf/main-focus/remote-sensing/ .. _polymer: https://github.com/hygeos/polymer .. _SNAP: http://step.esa.int/main/toolboxes/snap/ .. _European Space Agency: http://www.esa.int/ .. _Daniel Odermatt: https://www.eawag.ch/de/ueberuns/portraet/organisation/mitarbeitende/profile/daniel-odermatt/show/ .. _example.ini: https://github.com/eawag-surface-waters-research/sencast/blob/master/environments/example.ini .. _WKT: https://en.wikipedia.org/wiki/Well-known_text_representation_of_geometry Indices and tables ================== * :ref:`genindex` * :ref:`modindex` * :ref:`search`