Sencast
Sencast is a toolbox to download and derive water quality parameters from satellite images. It acts as a framework for the use a variety of processors such as Idepix, Polymer and Acolite. It supports ESA satellites Sentinel 2 and Sentinel 3 and USGS satellite Landsat 8.
It is developed and maintained by the SURF Remote Sensing group at Eawag.
Warning
Sencast is under active development. Please contact us if you find bugs or issues.
Publications
Installation
To install Sencast, run
git clone https://github.com/eawag-surface-waters-research/sencast.git
conda env create -f ~/sencast/sencast.yml
Many of the Sencast’s processors reply on SNAP , the Sentinel Application Platform project, funded by the European Space Agency (ESA) or other 3rd party packages. In order to have access to all of Sencast’s processors follow the installation instructions below in order to correctly configure your environment.
This process will require registering accounts with data providers.
Installation
For issues with installation, please contact Daniel Odermatt.
Warning
It can be difficult and time-consuming to get a local installation set up (particularly for Windows). For users not planning on developing the code it is recommended to use the Docker image provided (see below).
Getting Started
Sencast can be run in two ways:
By calling the
main.pyscript with command line arguments
python main.py -p parameters.ini -e environment.ini
Parameters |
Default |
Description |
|---|---|---|
-t –tests |
False |
run test processing to check setup |
-x –delete_tests |
False |
delete previous test run |
-p –parameters |
Required |
link to the parameters.ini file (required except for tests) |
-e –environment |
${machine-name}.ini |
link to the environment.ini file |
-d –downloads |
1 |
number of parallel downloads |
-r –processors |
1 |
number of parallel processors |
-a –adapters |
1 |
number of parallel adapters |
By importing Sencast as a function
import sys
sys.path.append("/sencast") # Path to sencast repository
from main import sencast
sencast(params_file, env_file=None, max_parallel_downloads=1, max_parallel_processors=1, max_parallel_adapters=1)
For this options you can pass objects as the params_file and env_file as well as links to the text files.
Environment File
Environment files use the INI format and contain the configuration of the machine on which Sencast runs. Refer to Environment File for details on how to set up your own environment file.
You should create your own environment file for every machine you install Sencast on.
Parameter File
Parameter files use the INI format and contain the parameters for one execution of Sencast. Refer to Parameter File for details on how to set up your own parameter file.
Perimeter Definition
Perimeter definitions define a geographic area to be processed by Sentinel Hndacast. They are stored as polygons in WKT files, which are referenced from the parameter files. Some example perimeters are stored in the wkt folder.
Testing
To test your installation run:
cd ~/sencast
conda activate sencast
python main.py -t
This will report which processors are successfully installed and producing meaning-full outputs.
Docker
Manual installation of all the processors is challenging and can be simplified through the use of a docker container.
Users should first ensure they have a functioning docker installation.
Pull container
The docker image can be downloaded from docker hub using the following command:
docker pull eawag/sencast:latest
Run Tests
In order to test the setup is working the following command can be run which will output a report on the functioning of the processors. This must be run from inside the sencast repository.
The option -v /DIAS:/DIAS maps the input/ output folders to a location outside the container. This should be updated to the appropriate location, e.g. -v /home/user/DIAS:/DIAS
docker run -v /DIAS:/DIAS -v $(pwd):/sencast --rm -it eawag/sencast:latest -e docker.ini -t
-e name of the environment file in sencast/environments -t flag to indicate a test should be run
Run script
In order to run a parameters file it can be passed to the command as follows using the -p flag.
docker run -v /DIAS:/DIAS -v $(pwd):/sencast --rm -it eawag/sencast:latest -e docker.ini -p example.ini
-p name of the parameter file in sencast/parameters
Run Interactive Container
Sometimes it is desirable to interact directly with the container, this can be achieved with the following command
docker run -v /DIAS:/DIAS -v $(pwd):/sencast --rm -it --entrypoint /bin/bash eawag/sencast:latest
Locally build container
docker build -t eawag/sencast:latest .
Configuration
Utilities
- auxil
- product_fun
append_to_valid_pixel_expression()copy_band()copy_nc()create_band()create_chunks()filter_for_baseline()filter_for_tiles()generate_l8_angle_files()get_band_from_nc()get_band_names_from_nc()get_bounds_from_nc()get_commit_hash()get_l1product_path()get_lat_lon_from_x_y()get_lat_lon_from_x_y_from_nc()get_lons_lats()get_main_file_from_product_path()get_name_width_height_from_nc()get_np_data_type()get_pixel_pos()get_pixel_value_xy()get_pixels_from_nc()get_reproject_params_from_jp2()get_reproject_params_from_nc()get_reproject_params_from_wkt()get_s2_tile_name_from_product_name()get_satellite_name_from_product_name()get_sensing_date_from_product_name()get_sensing_datetime_from_product_name()get_south_east_north_west_bound()get_tile_name_from_product_name()get_valid_pe_from_nc()parse_date_from_name()parse_s3_name()read_pixels_from_band()read_pixels_from_nc()remove_superseded_products()write_all_pixels_to_nc()write_pixels_to_band()write_pixels_to_nc()
Processors
Adapters
DIAS API's