Release History#
0.21.0 (2023-MM-DD)#
BREAKING CHANGES: Rename
utils.stack_dict
toutils.stack
since we are stacking datasets and not dict anymore.BREAKING CHANGES: Band ID for Sentinel-3 OLCI are now int instead of band names (i.e.
7
instead ofOa07
. The names don’t change).ENH: Allow to use bands IDs, names and common name added to mapped names when trying to load a spectral band. (#111
CI: Update pre-commit hooks
DEPS: Remove as many mention as possible to
cloudpathlib
GITHUB: Update bug template
0.20.4 (2023-09-26)#
FIX: Don’t collocate a raster on itself
FIX: Better management of default pixel size for multi-resolution products (such as PAN band in Landsat)
FIX: Fixing the PAN GSD for Landsat-OLI products
FIX: Update some code to match
sertit>=1.29.0
0.20.3 (2023-07-31)#
Bug Fixes#
0.20.2 (2023-06-22)#
Bug Fixes#
FIX: Use already computed bands stored in
tmp
for Planet products
0.20.1 (2023-06-20)#
Bug Fixes#
FIX: Allow band aliases (such as
"CA"
instead of"COASTAL_AEROSOL"
) in string in band mapping when creating Custom Stacks
Other#
COMPAT: Add the alias
GREEN_1
for GREEN I band of PlanetScope data, in order to stay in the same pattern asVRE_x
,SWIR_x
…GREEN1
will be deprecated in another release.
0.20.0 (2023-05-31)#
Breaking Changes#
Bug Fixes#
FIX: Collocate bands before trying to create spectral indices: resolve the case where their size mismatches (i.e. in case of window or change of native pixel size)
FIX: Landsat band masking when specifying a custom resolution and a custom window
FIX: Round the default pixel_size of custom stacks
FIX: Convert some fields of STAC items from non JSON serializable dtypes to correct ones
FIX: Fix erroneous property set to
_get_raw_crs
for Maxar products
Other#
ENH: Don’t load data into memory when computing indices, ensuring tasks are delayed a bit longer (#58)
DOCS: Add documentation about default CRS (#87)
DEPS: Dropping support of Python 3.8 (#81)
DEPS: Pin sertit to 1.27.0
INTERNAL: Better management of logs for deprecation warnings
INTERNAL: Refactoring
simplify_footprint
insertit
libraryCI: Test that STAC items are serializable when added to a catalog
0.19.4 (2023-04-12)#
Bug Fixes#
FIX: Removing calibration step from SNAP pre-processing graph for multi-swath
Cosmo-SkyMed 1st GEN
products (to avoid ending up with empty images after pre-process)FIX: Fixing the paths to Sentinel-2 quicklooks: using PVI instead of TCI file if no .jpg preview file is found (#84 , #85, thanks a lot @floriandeboissieu)
Other#
STAC: Updates in STAC management
INTERNAL: Use
geopandas.estimate_utm_crs()
when possible
0.19.3 (2023-03-24)#
Bug Fixes#
OPTIM: Don’t recompute stacks if already existing on disk
FIX: Fixing
Custom Stacks
when specifyingdatetime=None
on creationFIX: Fix regression for multi-swath DGM CSK data (huge region) (#78)
FIX: Fix calibration issues with CSK HR data (using fallback GPT graph by default)
Other#
OPTIM: Always use chunks when reading rasters (#58)
OPTIM: Speed up VRT virtual warping
OPTIM: Better management of dask’s usage
CI: Fix projection STAC extension’s new version number (1.1.0)
0.19.2 (2023-02-23)#
Bug Fixes#
FIX: Fixing stack when saved as integer for some special cases
FIX: Clipping negative reflectances to 0 (#79)
FIX: Fixing nodata mangement for Theia product
FIX: Fixing handling of SCS multi-swath
Cosmo-SkyMed
products (#78)FIX: Writing spectral indices on disk to align with other bands (#80)
FIX: By default, calibration is not applied to slant range
CSG
data, avoiding producing an empty raster (#48)
Other#
OPTIM: Using warped VRT instead of reprojecting DEM/VHR stacks to UTM (#58)
TYP: Fixing typos in typing
INTERNAL: Moving
EOREADER_NAME
andDATETIME_FMT
into__init__.py
INTERNAL: Moving stacking function into
utils
INTERNAL: Removing unused
cache_property
decoratorINTERNAL: Factorizing
_load
functionCI: Using
assert_raster_almost_equal_magnitude
in CI to better check according to bands’ content (sertit 1.24.0)DEPS: Officially handling Python 3.11 (adding weekly tests on Python 3.11) (#71)
DOC: Updating
Custom
notebookDOC: Updating jupyter cache to match new way of handling outputs in readthedocs
0.19.1 (2023-01-12)#
Bug Fixes#
FIX: Fixing a bug for DIMAP V2 products whith GML masks opening without CRS: assigning first the raw CRS before converting to the product’s CRS
FIX: Fixing index creation when exotic bands not handled by ASI have been loaded in the same time (i.e. stacking
NDWI
withOa21
band)
Other#
CLEAN: Removing useless GCP functions regarding Sentinel-3 data
DOC: Adding a
Remove Clouds
notebookLIB: Pinning
sertit
to 1.22.0CI: Don’t run tests when only
__init__
or__meta__
is updatedCI: Some factorizing in
gitlab-ci
0.19.0 (2023-01-03)#
Enhancements#
Bug Fixes#
FIX: Fix extent computation for
CSG
products with Shapely 2.0FIX: Shapely 2.0 deprecation warnings
Other#
DEPR: Add deprecation warning for EOReader spectral indices (used for legacy in 0.18.0) that are aliases of ASI names (#72):
AFRI_1_6
:AFRI1600
,AFRI_2_1
:AFRI2100
,BSI
:BI
,NDGRI
:NGRDI
,NDRE1
:NDREI
,RGI
:RGRI
,WV_BI
:NHFD
,WI
:WI2015
,RDI
:DSI
,DSWI
:DSWI5
,GRI
:DSWI4
,WV_SI
:NDSIWV
,PANI
:BITM
DOC: Changing copyright from 2022 to 2023
0.18.1 (2022-12-08)#
Bug Fixes#
FIX: Fix regression for missing EOReader aliases for
spyndex
spectral indicesFIX: Fix bug in footprint computation of DIMAP V1 data
Other#
DOC: Add latest DOI link
LIB: Pass to
sertit==1.21.0
to handle windowed data in read natively
0.18.0 (2022-12-06)#
Breaking Changes#
BREAKING CHANGES: Refactoring spectral indices management (#47)
Using spyndex library, allowing to use all spectral indices listed here
SAR products may now compute indices if possible (see this list)
Old EOReader indices are still available for legacy purposes, with some changes:
For Sentinel-2 data, the band
NIR
andNARROW_NIR
may be interchanged for some index ( see this discussion)OSAVI formula has changed to stick with the original paper definition (see issue)
NDRE2/3
formula are fixed, now usingVRE_2/3
andNDRE1
corresponds toNDREI
and usesVRE_1
CI1
is renamedCI32
andCI2
is renamedCI21
for readability purposesNDWI21
can be writtenNDWI2100
for homogeneity purposesRDI
(orDSI
) uses nowSWIR_1
instead ofSWIR_2
(see this issue)PANI
equivalent is nowBITM
and is normalised ! (divided by 3)SBI
is normalized (divided by 2) to fit withBIXS
definition⚠ You may need to install the last
spyndex
directly from GitHub latest version to have all available indices
BREAKING CHANGES: Using
pyresample
to geocode Sentinel-3 data (#55)Cleaner: better conversion from swath to grid
Faster: Up to 4 times faster
Allows code refactoring between OLCI and SLSTR
BREAKING CHANGES: For SAR product types that are not available in the Data Access Portfolio, default resolution is now the pixel spacing instead of the rg x az resolution
Changes mainly Sentinel-1 default resolutions (except from IW mode)
Enhancements#
Bug Fixes#
FIX: Fixing
CustomProduct
initialization when fields are set to None (instead of not declaring them)FIX: SNAP cannot handle float predictors other than 1! Set it to 1 when saving ortho SAR images to disk, in order for SNAP to be able to despeckle them. See SNAP issue. (#62)
FIX: Fixing mix in
Sentinel-2
mapping forB8
(NIR
, 10m resolution, large spectral bandwidth) andB8A
(NARROW_NIR
, 20m resolution, narrow spectral bandwidth)
Other#
DOC: Add FAQ entry concerning SAR constellations extent KML files failing to be read (TLDR: needs
ogr2ogr
in your PATH)DOC: Add Technical Note published in Remote Sensing MDPI in Readme
DOC: Update optical band mapping graphs (fix regression to 0.15.0 supported constellation)
DOC: Add information about DEM management in SAR notebook (#61)
DOC: Updating indices paragraphs
CI: Using actions/checkout@v3
CI: Updating versions of pre-commit hooks
LIBS: Updating
requirements.txt
andsetup.py
to addpyresample
andzarr
0.17.0 (2022-10-12)#
Enhancements#
ENH: Adding the support of RapidEye constellation
ENH: Handling Planet data with multiple subdatasets (#45)
ENH: Adding the support of Landsat Level-2 products (#49)
ENH: Adding the support of Pleides Neo SEN and PRJ products (needs GDAL 3.5+ or rasterio 1.3.0+)
ENH: Adding the function
bands.is_thermal_band
ENH: Adding the ability for optical custom stacks to load indices
ENH: Adding BAIM (MODIS Burned Area Index) spectral index
ENH: Better management of raw units of the bands of optical products
ENH: Copying files from
tmp_process
when changing product’s output
Bug Fixes#
FIX: Stacks saved as integers on disk keep their original dtype (float32) in Python
FIX: Stacks with bands loaded “as is” are correctly saved as integers on disk (#52)
FIX: Using stack CRS (if projected) for
DIMAP
products instead of recomputing from lat/lon, solving potential discrepancies between stack and product CRSFIX: Workaround for JP2 bug when updating an existing raster (maybe related to this bug)
FIX: Better management of SkySat datetime conversion from JSON to XML (deterministic way)
FIX: Fixing computation of invalid pixels for
Sentinel-2
andDIMAP
products (do not remove straylight mask)FIX: Fixing reprojection resolution of VHR data
FIX: Computing Brightness Temperature of
Landsat
TIR bands instead of leaving them as isFIX: Better management of Landsat Instrument values
FIX: Better radiometry attribute (adding
brightness temperature
andreflectance and brightness temperature
values)FIX: Changing
Brilliance Temperature
to the correctBrightness Temperature
FIX: Fixing pandas FutureWarning
The frame.append method is deprecated and will be removed from pandas in a future version.
FIX: Fixing DeprecationWarning
invalid escape sequence \.
FIX: Manage correctly Planet dubious pixels (especially for 8 bands products)
Optimizations#
OPTIM: Reduce memory usage when updating all the bands attributes
OPTIM: Reduce memory usage when stacking as integers
Other#
DOC: Add the need of using SNAP 8.0 up-to-date or SNAP 9.0 (#42)
DOC: Add the STAC session in API documentation
DOC: Add warnings for shifts when orthorectifying DIMAP SEN products (using RPCs) (#53)
DOC: Add limitations to custom stacks
DEPS: Dropping support of Python 3.7 (#18)
DEPS: Update minimum version of libs (geopandas 0.11.0+, rasterio 1.3.0+…)
0.16.1 (2022-08-03)#
Bug Fixes#
FIX: Add the missing conversion to reflectance for
Sentinel-3 OLCI
FIX: Better condition for the conversion to reflectance for
Sentinel-2 THEIA
FIX: Add logs for
SkySat
data that cannot been converted to reflectance and fix theradiometry
field of its band xarraysFIX: Add the correct nodata (when overridden by the user) to stacks saved as uint16
Optimizations#
OPTIM: Reduce memory usage during stacking
Other#
CI: Test reflectance values
0.16.0 (2022-08-01)#
Enhancements#
ENH: Adding the support of SuperView-1 constellation (#21)
ENH: Adding the support of SPOT-4/5 constellations (#39)
ENH: Allow the possibility to pass a constellation (or a constellation list) to
Reader().open()
to speed up the opening of a productENH: Add a quicklook search for
Sentinel-3
products
Bug Fixes#
FIX: Fix quicklook media type with
JP2
filesFIX: Fix
Sentinel-3 SLSTR
F1
bands based on F gridFIX: Correct the UTM projection for
Sentinel-3
dataFIX: Fix handling of zipped
Sentinel-2 L2Ap
FIX: Fix zipped
Sentinel-2
with other XML files in GRANULE subdirectories
Other#
Renaming
master
branch tomain
0.15.1 (2022-06-02)#
Optimizations#
OPTIM: Try to create
Vision-1
footprint from the preview file instead of from the stack.OPTIM: Create footprints for stacked products (i.e.
Maxar
,SkySat
,Custom
…) without mask by opening only the first band of the stackOPTIM: Create footprints for
Maxar
Products with a resolution 10 times lower.OPTIM: Footprints have now maximum 50 vertices in order to avoid pixelized footprints
Bug Fixes#
FIX: Fixing condensed name to avoid duplicates:
adding the
job_id
forVHR
productsadding the polarization channels for
SAR
products
FIX: Remove import of pystac in
stac_utils
FIX: Fix bug for
Vision-1
data looking for non-existing RPC files in case ofORTP
product typeFIX: Fix quicklook regex for
Vision-1
dataFIX: Fix regex for raw bands for extracted
Sentinel-3 OLCI
productsFIX: Fix
PlanetScope
identifying regex to handle products with a satellite_id containing a letterFIX: Force metadata regex for
Maxar
products to look for a file with pattern{name}.XML
to avoid other misplaced XML to be found in place of the true XML.FIX: Fix regression for
Landsat-7
footprintFIX: Manage the case with
cloud_cover = -999.0
forMaxar
products (returnsNone
)
Other#
CI: Add new optical products to be tested for end-to-end tests
0.15.0 (2022-05-30)#
Breaking Changes#
BREAKING CHANGES:
Optical
becomesSpectral
when more appropriateBREAKING CHANGES:
Platform
andSensor
becomeConstellation
when more appropriate, to fit STAC vocabulary (#29):Platform
enum becomesConstellation
prod.platform
becomesprod.constellation
prod.sat_id
becomesprod.constellation_id
BREAKING CHANGES: File
alias
is removed, replaced by*_bands
files and proper imports inbands.__init__
BREAKING CHANGES: Product attribute
band_names
becomesbands
in order to be STAC compliant (#29)BREAKING CHANGES: Better use of
NIR
andNARROW_NIR
in theindices
file (according to the gsd ofSentinel-2
bands composing the indices)BREAKING CHANGES: Correcting Landsat product types to better manage processing levels and instrument. Landsat-8/9 condensed name may change!
Enhancements#
ENH: Adding the support of
SkySat
(Collect) products (#20)ENH: Bands in mapping are now objects, instead of just IDs (#29). This allows us to:
Add band metadata (such as center wavelength, bandwidth…)
Map spectral bands between STAC spec and EOReader format (#29)
Add a better
__repr__
functions
ENH: Handling 8 bands
PlanetScope
data (#20)ENH: Adding the
GREEN1
mapped band, corresponding to PlanetScopeGREEN I
andSentinel-3 OLCI
Oa05
bandENH: Handle some slightly broken
Sentinel-2
products:when the metadata files are corrupted or when the detfoo vectors are empty (#34)
with missing MSK prefix for QI_DATA files (i.e.
DETFOO
instead ofMSK_DETFOO
)
ENH: Handle exception for corrupted bands (in
Sentinel-2
andutils.read
) (#34)ENH: Add a STAC object that can be used to retrieve STAC Items from every Product (
prod.stac.create_item()
) (#29)ENH: Add a
get_mean_viewing_angles
for Optical Products to fill STAC View Extension (#29)ENH: Extending
get_raw_band_paths
to every product (#31)ENH: Adding a
is_ortho
attribute corresponding to when the product is already orthorectified/geocoded, in order to avoid computing heavy processes without wanting it (i.e. footprint…)ENH: Adding the instrument name of every constellation, under
prod.instrument
ENH: Handling
COSMO
product with only theh5
file in it (if missing XML metadata file) (#36)
Optimizations#
OPTIM: Retrieve name from filename if possible
OPTIM: Retrieve extent from metadata when possible (for VHR data)
Bug Fixes#
FIX: Fixing the band mapping of
WorldView-2/3 Multi
(8 bands)FIX: Retrieval (if possible) of Sentinel-1 unique ID (was missing from the product name, as it is not in the product preview)
FIX: Fixing PAZ/TDX MTD regex
FIX: Optical products: Only set cloud cover and radiometry attributes if spectral bands are asked
Other#
INTERNAL: File
spot_6
andspot_7
are removed, replaced by a uniquespot
file. This shouldn’t affect the user.INTERNAL: Refactoring Landsat-XX products into
LandsatProduct
, this should be invisible for user.INTERNAL: Some refactoring in
VHR
filesWARNINGS: Filter warnings from
__init__
CI: Do not process two times the zipped Sentinel-1 in end-to-end tests and manage when the runner kills SNAP
DOC: Adding a STAC notebook
DOC: Various updates
0.14.0 (2022-04-14)#
Breaking Changes#
BREAKING CHANGES:
footprint
,extent
,wgs84_extent
andcrs
properties are converted back to methods in order to prevent side effects of expensive computation when displaying the object when debugging (rollback before version 0.8.0)BREAKING CHANGES:
get_all_index
becomesget_all_indices
BREAKING CHANGES:
acquisition_datetime
becomesdatetime
anddefault_resolution
becomesresolution
forCustomProduct
in accepted keywords, and the metadata fields have been renamed according to theCustomFields
enum
Enhancements#
ENH: Adding spectral indices:
Shadow Index (
SI
)Global Vegetation Moisture Index (
GVMI
)Soil Brightness Index (
SBI
), Soil Cuirass Index (SCI
)Panchromatic mocking Index (
PANI
)Green-to-Red ratio Index (
GRI
)Soil Adjusted Vegetation Index (
SAVI
)Optimized Soil Adjusted Vegetation Index (
OSAVI
)Visible Atmospherically Resistant Index (Green) (
VARI
)Enhanced Vegetation Index (
EVI
)Chlorophyll Index RedEdge VRE_3/VRE_2 (
CI1
)Chlorophyll Index RedEdge VRE_2/VRE_1 (
CI2
)Normalized Difference Moisture Index (with SWIR_21) (
NDMI21
)
ENH: Making SAR attribute
snap_filename
publicENH: Handling
ICEYE
pure SLC productsENH: Allowing the user to choose if they want the GRD or SLC image for
ICEYE
productsENH: Add the possibility to directly load the cloud cover for optical data (and add it in the band attributes) (#28)
ENH: Add the possibility to retrieve the quicklook path (if existing) and add the
plot
function allowing the user to plot the quicklook (if existing) (#28)ENH: Add the possibility to retrieve the orbit direction (and add it in the band attributes) (#28)
Bug Fixes#
FIX: Fixing the inversion between
8
and8A
bands forSentinel-2
andSentinel-2 Theia
productsFIX: Loading every optical band in reflectance (fixed for
Sentinel-2 THEIA
,Maxar
,Planet
andVision-1
data) (#30)FIX: Fixing
ReferenceError: weakly-referenced object no longer exists
when deleting an objectFIX: Do not set sea values to nodata when orthorectifying SAR data with SNAP
FIX: Handle
Sentinel-2
data with PB < 02.07 asL2Ap
productsFIX: Fixing nodata and offset for
Sentinel-2
data with PB > 04.00FIX: Handle new
ICEYE
metadata name’s nomenclatureFIX: Fixing harmless regex error when searching for B1 path for
Landsat
productsFIX: Fixing platform for
Sentinel-2 Theia
Other#
DOC: Creating a real
base
notebook and renaming the old one tooptical
DOC: Better type hints (replacing
XDS_TYPE
byxr.DataArray
)CI: Using
sertit.ci.reduce_verbosity
instead of recreating the function
0.13.1 (2022-03-08)#
Bug Fixes#
FIX: Handling
Sentinel-2 L2Ap
dataFIX: Do not use
--no-binary fiona,rasterio
directly inrequirements.txt
(breaks on Windows)FIX: Fixing stacking with string bands
FIX: Better
__repr__
functionFIX: Read README as UTF-8 in setup.py
Other#
CI: Adding a tag for choosing the runners
DOC: Fixing cartopy/GEOS conflicts making the documentation build to fail
0.13.0 (2022-03-02)#
Enhancements#
ENH: Adding the support of
Landsat-9
sensorENH: Support Sentinel-2 with missing datatake metadata file(sometimes happens with data downloaded from AWS buckets and converted to .SAFE)
Bug Fixes#
FIX: Using default SAR resolution from official Copernicus Data Access Portfolio (2014-2022) ( Sentinel-2 default resolution goes to 10.0 m !)
FIX: Use
--no-binary fiona,rasterio
directly inrequirements.txt
FIX: Removing useless
outputComplex
line in GPT graphs that is breaking SNAP on LinuxFIX: Removing the workarounds caused by some bugs of
cloudpathlib
and enabling retrieval of nested SAR products (TSX, TDX, PAZ, RCM) from S3 compatible storage.FIX: Do not process nodata for a band already existing
FIX: Fixing an error when reading
TIR
bands with Landsat-7FIX: Fixing an error when additive/multiplicative coefficients are set to
NULL
for Landsat dataFIX: Returning sun angles always as float (some
Sentinel-3
angles were returned asnp.array
)
Other#
CI: Do not try to process SAR end to end if GPT cannot be found
CI: Publishing wheel from GitHub instead of Gitlab
REPO: Setting GitHub as the main repository and using new Gitlab runners
0.12.0 (2022-02-09)#
Enhancements#
ENH: Adding the support of
Pleiades-Neo
,Vision-1
andSAOCOM
sensorsENH: Adding a keyword to allow passing a specific DEM path in
load
/stack
(for VHR orthorectification andDEM
bands)ENH: Adding the name of the DEM in DEM band (i.e. allow to compute the
HILLSHADE
with a DEM and theSLOPE
with a DTM)
Bug Fixes#
FIX:
Sentinel-2
Processing Baseline 04.00:NARROW_NIR
bands are now loaded correctlyFIX:
Maxar
products (withMulti
band ID) are now correctly handledFIX: Using
COPDEM-30
(GLO-30
) by default for SNAP as it appears that the retrieval has been fixed.FIX: Fixing the default name for cleaned bands for
Sentinel-3 SLSTR
data (was set onCLEAN
instead ofNODATA
)FIX: Fixing default band for Custom stacks
FIX: Fixing
get_existing_band_paths
behavior for Custom stacksFIX: Remove other never covered lines of code (archived
RCM
products, complexICEYE
products, others…)FIX: Re-enabling loading str bands (regression)
FIX: Proper check for empty fields when parsing metadata
FIX: VHR
_get_dem_path
raisesValueError
instead ofTypeError
FIX: Pre-process SAR bands before despeckling if not existing (was OK in most of the cases, but broke in some cases, especially with CI folder activated and S3 compatible storage)
FIX: Remove warning
invalid escape sequence \.
,\w
,\D
and\s
FIX: Do not set
long_name
forRAW_CLOUDS
arraysFIX: Providing a URL DEM on Windows throws a
OSError
instead of a bareException
Optimizations#
OPTIM: Do not pre-process existing Sentinel-3 geocoded bands
OPTIM: Do not look for valid metadata further than a given nested level in product’s directory (for extracted products)
Other#
CI: Using another (faster) runner
CI: Add on disk and end-to-end tests
CI: Do not write tmp files when running on disk tests
CI: Coverage:
Get coverage as HTML
Remove useless lines from coverage
Combine coverage of S3 and on disk tests
DOC: Adding a DEM notebook
0.11.2 (2022-01-19)#
Bug Fixes#
FIX: Fixing archived SAR processing
FIX: Needs extraction for
RS2-SLC
data as SNAP does not handle the productFIX: Fixing the default name for cleaned bands for optical data (was set on
CLEAN
instead ofNODATA
)
0.11.1 (2022-01-17)#
Bug Fixes#
FIX: Fixing complex and orthorectified products for
SAR
dataFIX: Fixing
RADARSAT-2
SLC
product type
Optimizations#
OPTIM: Only preprocessing wanted SAR bands (instead of all existing)
OPTIM: Do not interpolate nan values by default when writing SAR bands to disk (using a keyword instead)
Other#
DOC: Updating the SAR notebook and documentation
0.11.0 (2022-01-13)#
Breaking Changes#
BREAKING CHANGES: Renamed
is_band
tois_sat_band
to better reflect that this function only checks optical and SAR bandsBREAKING CHANGES: Invalid pixels are not processed by default anymore! Only the nodata is set (to go a bit faster)
Enhancements#
ENH: Allowing the user to choose the pixel processing for optical bands: raw band, only nodata or total cleaning of defective pixels (#16)
ENH: Adding a CustomProduct, allowing the user to load any stack as an EOReader Product !
ENH: Check if a band exists before trying to load it
Bug Fixes#
FIX: Better handling of
__all__
in__init__.py
filesFIX: Ensure that extents and footprints are in UTM
FIX: Removing docs from wheel
FIX: Fixing
TIR
bands reading for Landsat data
Optimizations#
OPTIM: Optimizing
manage_invalid_pixels
forSentinel-2
data (processing baseline >= 04.00)
Other#
DOC: Update README, documentation and notebooks
DOC: Water Extraction notebook has been refined to show how to manage multiple products
DOC: Update the installation paragraph in README
DOC: Adding a
For Contributors
section in the documentation (contributing, release history and GitHub repository)DOC: Remove doc testing in GitHub (as the docs are built with readthedocs)
INTERNAL: Better management of project metadata (version…) in a dedicated file
0.10.1 (2022-01-04)#
Bug Fixes#
FIX: Resolve a bug when
methodtools
is not present (for conda package)
0.10.0 (2022-01-04)#
Enhancements#
ENH: Adding
has_bands
to products, ingesting lists as a shortcut for testing the availability of multiple bandsENH: Simplifying imports. Now you can replace:
from eoreader.bands.alias import RED, NDVI
byfrom eoreader.bands import RED, NDVI
,from eoreader.products.optical.optical_product import OpticalProduct
byfrom eoreader.products import OpticalProduct
,from eoreader.products.optical.s3_slstr_product import SlstrRadAdjustTuple
byfrom eoreader.products import SlstrRadAdjustTuple
, …
Optimizations#
OPTIM: Writing cloud bands on disk to speed up multiple calls to
load
orstack
functions (#17)
Bug Fixes#
FIX: Correctly naming cloud xarrays
FIX: Add missing
SLEA
(Spot Extended Area) product type toICEYE
dataFIX: Sentinel-2 clouds (with processing baseline >= 4.0) are now given with a rasterio shape (
count
,height
,width
)
Other#
CI: Remove
pages
stage and run only the tests when a Python file has changedDOC: Updating notebooks
DOC: Updating copyright to 2022
0.9.5 (2021-12-14)#
Bug Fixes#
FIX: Do not force import
methodtools
(not existing lib in conda)FIX: Using
GRD
resolution given by the constructors as default values forSLC
products. Do not look it up in metadata as SLC resolution is NOT the GRD resolution !
0.9.4 (2021-12-13)#
Bug Fixes#
FIX: Caching properties and functions only for object instances
FIX: Fixing metadata reading for
COSMO-SkyMed 1st Generation
withWide Region
and complex product type (handling of multiple swaths)FIX: Updates of SNAP GPT graphs for complex SAR data
FIX: Interpolate nodata inside SAR images (badly handled by SNAP -> fill the gaps that shouldn’t exist)
Other#
INTERNAL: Creation of a class
CosmoProduct
handling generic methods for bothCOSMO-SkyMed
generations
0.9.3 (2021-12-09)#
Bug Fixes#
FIX: Fixing the search for
.TIL
files forMaxar
products (with on disk files)FIX: Fixing the search for metadata files for
Landsat
products (with on disk files)FIX: Fixing the search for metadata files for
TerraSAR-X
,TanDEM-X
andPAZ SAR
products (with on disk files)FIX: Fixing SNAP files for
TerraSAR-X
,TanDEM-X
andPAZ SAR
productsFIX: Fixing when reading CRS code for
DIMAP
products
0.9.2 (2021-12-07)#
Bug Fixes#
FIX: Fixing flag type for
Sentinel-3
dataFIX: Do not multiply the flags values by the radiance adjustment factor for
Sentinel-3 SLSTR
!FIX: Fixing flag exception threshold for
Sentinel-3 SLSTR
FIX: Fixing preprocessed band filenames for
Sentinel-3 SLSTR
0.9.1 (2021-12-07)#
Bug Fixes#
FIX:
Reader().valid_mtd
now correctly accepts strings instead of onlyPlatform
objectsFIX: Better handling of
Sentinel-2
product typeFIX: Save bands’ new attributes in
str
(to pickle them)FIX: Add a
clear()
function to clear products cache
0.9.0 (2021-12-06)#
Enhancements#
ENH: Adding the support of the ICEYE sensor
ENH: Adding the support of the COSMO-SkyMed 2nd Generation sensor
ENH: Adding some attributes to bands and stack:
sensor
,sensor_id
,product_type
,acquisition_date
,condensed_name
#7ENH: Replace name by filename and read directly the true name of the product in the metadata (#15)
Bug Fixes#
FIX:
Sentinel-1
metadata file with archived products (discarding RFI folder in its search).FIX: Add
Quickbird
,GeoEye
andWorldView
sensors inreader
regexes.FIX: Add scipy in
requirements.txt
andsetup.py
Other#
DOC: Fix references to
pcigeomatics
that doesn’t exist anymore (RADARSAT-2 and Constellation)REQ: Update
dask
to fix a security issue (only in requirements asdask
is not mandatory)
0.8.1 (2021-10-26)#
Bug Fixes#
FIX: Do not force
chunk
inutils.read
if dask is not installed
0.8.0 (2021-10-25)#
Breaking Changes#
BREAKING CHANGE:
crs
,footprint
,extent
,wgs84_extent
are now properties !BREAKING CHANGE: Removing raw
gdaldem
CLI from EOReader (theHILLSHADE
andSLOPE
bands are now slightly different !) (#10)BREAKING CHANGE:
HILLSHADE
is given infloat32
instead ofuint8
BREAKING CHANGE:
SLOPE
is given in degrees instead of percents
Enhancements#
ENH: Adding the support of the PAZ SAR sensor
ENH: Adding the support of the Sentinel-2 processed with the processing baseline 4.0 (#11)
ENH: Removing SNAP from Sentinel-3 pre-process -> Freeing optical data from SNAP dependency ! (#12)
ENH: Enabling the use of other S3-SLSTR suffixes than
an
(stripe A at nadir position)ENH: Thermal bands of Sentinel-3 SLSTR can now be used
ENH: All bands of Sentinel-3 SLSTR/OLCI can now be used (
S7
,F1
,F2
for SLSTR,Oaxx
for OLCI) (#14)ENH:
YELLOW
band is mapped toOa07
band of Sentinel-3 OLCIENH: Zipped Sentinel-3 products can now be processed
ENH: Allow the use of
kwargs
inload
, mainly forrasters.read
(and allowing i.e. radiance adjustment in S3-SLSTR)
Optimizations#
OPTIM:
crs
,footprint
,extent
,default_transform
,wgs84_extent
are cached ( using@cached_property
) (#13)OPTIM:
get_mean_sun_angles
anddefault_transform
are now cached ( using@cache
) (#13)OPTIM:
get_datetime
: Look for the date only ifdatetime
attribute is None (#13)OPTIM: Better management of
fspath
for cloud-stored products (download the files only once)OPTIM: Stop downloading/extracting files if not necessary
Bug Fixes#
FIX: Bands are correctly ordered in stacks
FIX: Only load a band once, even if asked several time in the bands
FIX: Use band size for cleaning optical pixel (instead of user resolution/size)
FIX: Always take the absolute value of the resolution when converting it to strings (for filenames)
FIX: Take the default resolution if nothing is given when converting it to strings (for filenames)
FIX: Always use
utils.read/write
instead ofrasters.read/write
(for Dask management)FIX: Fixing a bug in
utils.write
FIX: Add .xml files from
eoreader/data
in the MANIFEST.inFIX: Add forgotten
@abstractmethod
where neededFIX: Better management of
_tmp_process
FIX: Fixing minor bug when trying to read metadata with a POSIX path
FIX: Fixing the
**kwargs
omission inutils.read
FIX: Better management of
_temp_process
directoryFIX: Landsats and TSX: Can use other filenames now
Other#
DEPR:
FAR_NIR
band is removedREQ: Using
h5netcdf
instead ofnetCDF4
DOC: Add a Context paragraph in the README
DOC: Add a Conda x SNAP question in the FAQ
DOC: Creation of a Sentinel-3 notebook
DOC: Updates of notebooks
DOC: Numerous updates
0.7.1 (2021-09-29)#
Bug Fixes#
FIX: Fixing a bug when opening archived Sentinel-1 (wrong metadata file found)
DOC: Updating CSS and readme
0.7.0 (2021-09-23)#
Enhancements#
ENH: Implementing RADARSAT-Constellation products (as
RCM
)ENH: Implementing Maxar products (such as
GE01, WV02, WV03, WV04
, but others should be supported too)ENH: Implementing TanDEM-X products (as
TDX
)ENH: Adding
RH
,RV
,RH_DSPK
andRV_DSPK
SAR bandsENH: Adding the
YELLOW
optical band (forWorldView-2
,WorldView-3
andSentinel-3 OLCI
)ENH: Adding WorldView index (without the ones using SWIR)
ENH: Loading by size -> round resolution to the closest meter (or decimeter for resolution < 1.0m)
ENH: Super class for VHR data
Bug Fixes#
FIX: Fixing reading PlanetScope archived products (error in read band)
FIX: Fix band name with complex resolutions
FIX: Fixing minor bug in RADARSAT-2 data when looking for product type
FIX: Fixing SAR band search in BEAM-DIMAP files
FIX: Fixing python version in environment.yml
FIX: Discard unused MIR and FNIR bands
FIX: Check for existence of given path when reading any product
FIX: Workaround for a bug involving some downloaded but badly formatted archives for Sentinel-2
FIX: Allow NARROW_NIR for and DIMAP data (== NIR)
FIX: Better management of writeable band folder
Other#
DOC: Fix documentation of the NDWI index
DOC: Update graph for optical band mapping
CI: Adding a test loading invalid band name
CI: Setting CI log level to DEBUG
CI: Accelerating the CI processes
0.6.4 (2021-09-15)#
Bug Fixes#
FIX: Sentinel-3 band mapping (
Coastal Aerosol
<->03
,BLUE
<->04
)
Other#
DOC: Adding an interactive graph for optical band mapping
0.6.3 (2021-09-10)#
Bug Fixes#
FIX: Load works with string bands (
prod.load('BLUE')
)FIX: Fixing missing
_remove_tmp_process
for products needing extractionFIX: Remove multi converting for Sentinel-3
0.6.2 (2021-09-10)#
Bug Fixes#
FIX: Better handling of archives for products that needs extraction
FIX: TerraSAR-X products need to be extracted to be processed by SNAP !
0.6.1 (2021-09-10)#
Bug Fixes#
FIX: Fixing critical bug for Sentinel-3 (mapping between clean bands and SNAP bands)
0.6.0 (2021-09-02)#
Enhancements#
ENH: Ensuring EOReader supports Dask
Bug Fixes#
FIX: Fixing and adding BAIS2 index in alias
FIX: Fixing GLI index
FIX: Fixing a bug when writing reprojected DIMAP band
FIX: Fixing a bug with SCS Cosmo-SkyMed data
Other#
DOC: Adding a DASK notebook
DOC: Updating notebooks
0.5.0 (2021-08-24)#
Breaking Changes#
BREAKING CHANGE: Read metadata/namespaces only once and store it as a private member. Keep accessing it through the
read_mtd
function (#9) WARNING: Breaking change for Landsat:read_mtd()
loses the argumentforce_pd=True
as it always returns an Etree
Enhancements#
ENH: Adding the BAIS2 index
ENH: Reads Sentinel-3 global attributes as metadata:
absolute_orbit_number
comment
contact
creation_time
history
institution
netCDF_version
product_name
references
resolution
source
start_offset
start_time
stop_time
title
ac_subsampling_factor
(OLCI
only)al_subsampling_factor
(OLCI
only)track_offset
(SLSTR
only)
ENH: Refining Despeckle Graph (#6) to use a more usual filter (
Refined Lee
)ENH: Allowing the user to open the datatake metadata for Sentinel-2 products
Bug Fixes#
FIX: Decoupling classic metadata reading from the name as EOReader accepts now modified product names (#9)
FIX: Better handling of cloud-stored DEM (raising an exception for non-ortho DIMAP data as GDAL and rasterio does not handle that case)
FIX:
environment.yml
to respect the stricter use offile:
syntax. See here for more information.FIX: Fixing bug when opening an archive product with
name
mode and nested dictionary (when looking for a filename instead of the directory name)
Other#
CI: Fixing
test_dems_https
and resetting DEM afterwardCI: Fixing DEM and ds2 database management
DOC: Adding a FAQ page and enhancing the Main Features page (#3)
DOC: Read Metadata has its own paragraph in Main Features
0.4.8 (2021-07-23)#
Enhancements#
ENH: Allowing
stack
to take single band in input instead of a list
Bug Fixes#
FIX: Fixing a regression loading optical bands which have been previously cleaned (Landsat, Theia, possibly PlanetScope)
FIX:
load
andstack
always returnsfloat32
arrays
Other#
CI: Testing if loading 2 times a band gives the same result
0.4.7.post0 (2021-07-23)#
Bug Fixes#
FIX: Fixing a regression loading Landsat bands which have been previously cleaned
0.4.7 (2021-07-23)#
Enhancements#
ENH: Adding a
default_transform
function returning data from default band (without warping it) -> mappingcalculate_default_transform
fromrasterio
ENH: Adding a
clean_tmp
function allowing the user to clean the product’s temporary output by handENH: Simplifying DEM warping code
Bug Fixes#
FIX:
DIMAP
products return always projected (in UTM) default bands (get_default_band_path
uses_get_default_utm_band
)FIX: Theia Footprint returns a
GeoDataFrame
instead of aGeoSeries
FIX: Better management of the
size
keyword withload
andstack
functionsFIX: Landsat retrieval of multipart cleaned bands (like
SWIR_1
)FIX: Some typehints fixes
FIX: Sentinel-3 open sun angles with NetCDF files stored in the cloud
FIX: Silently pass the writing of clean bands if we cannot (like if permission error)
Other#
CI: Do not reinstall everything if not needed (only PyYAML)
CI: renaming
build
stage intolint
CI: simplifying geometry comparison
CI: Testing the
size
keyword for every sensor
0.4.6 (2021-07-19)#
Bug Fixes#
FIX: Fixing no data for Sentinel-3 cloud bands
FIX: In alias:
DeprecationWarning: using non-Enums in containment checks will raise TypeError in Python 3.8
Other#
CI: Set default S3 client to point to unistra’s bucket
CI: Tox SNAP relay S3_DB_URL_ROOT
0.4.5 (2021-07-13)#
Bug Fixes#
FIX: Adding condensed name in the search when loading S3-SLSTR clouds
FIX: Fixing no data for Sentinel-3 bands processed by SNAP
FIX: Fix bug when stack path’s directory doesn’t exist
0.4.4 (2021-07-13)#
Bug Fixes#
FIX: Do not verbose empty lists when loading optical bands
FIX: Sentinel-3: Bands processed by SNAP are written with the condensed name as suffix
0.4.3.post1 (2021-07-08)#
Bug Fixes#
FIX: Fixing another bug for DEM_PATH using S3 Paths
Other#
DOC: Add a DOI
0.4.3.post0 (2021-07-08)#
Bug Fixes#
FIX: Fixing DEM_PATH using S3 Paths
0.4.3 (2021-07-05)#
Enhancements#
ENH: Optimizing loading cloud bands for DIMAP Products
stack
accepts**kwargs
in order to pass options torioxarray.to_raster()
Bug Fixes#
FIX: Fixing not found masks with S3+zip Sentinel-2 products
FIX: Fixing some type hints
Other#
CI: Fixing network directories with pathlib
0.4.2 (2021-07-01)#
Enhancements#
ENH: Enabling the use of products stored in the cloud (S3, S3 compatible storage, Google, Azure…) through
cloudpathlib
ENH: Using correct band names in long_name
Other#
CI: Use pre-computed cleaned band if existing
DOC: Adding examples for using S3 data, especially for S3 compatible storage
0.4.1.post0 (2021-06-21)#
Bug Fixes#
FIX: cloud mask values were inverted in Sentinel-2 cloud masks
FIX: Landsat collection 2 cloud masks are now OK
0.4.1 (2021-06-21)#
Bug Fixes#
FIX: Improving stacks saved as uint16:
Only satellite bands and index are scaled (*10.000)
DEM bands are just rounded
Cloud bands (booleans) are saved as is
FIX: Fixing a rasterization bug affecting S2 and DIMAP masks, happening when the vectors have another size than the image
FIX: Adding a warning on bad georeferencing when using GS and GT Landsat products
Other#
CI: Minor updates in documentation and code
0.4.0 (2021-06-10)#
Enhancements#
ENH: Adding THR data support:
PlanetScope
Pleiades
SPOT-6/7
ENH: Better handling of SNAP DEMs (using External DEM and other available SNAP DEMs)
Bug Fixes#
FIX: More robust way of looking for
data
directoryFIX: Bug fix in
stack
that causes some bands to be inexplicably empty sometimesFIX: Bug fix in
alias.isindex
FIX: Forcing extent to UTM
Optimizations#
OPTIM: Write clean bands on disk to avoid redoing invalid pixel computation and allow the user to remove them on deletion
OPTIM:
prod.has_bands
/prod.get_existing_bands
do not orthorectify/despeckle SAR bands anymore
Other#
CI: Adding a bimonthly test for SNAP processes
DOC: Adding two new notebooks (SAR and VHR data)
DOC: Completing the documentation
0.3.4 (2021-05-28)#
BREAKING CHANGES:
read_mtd()
returns a dict for the namespace map in order to manage multi namespace XMLsENH: Introduced support for DEM files from web urls (starting with http(s)://)
FIX: Landsat Zenith angle computation (mixing elevation and zenith angle)
CI: Adding weekly tests (
tox
on Python 3.7, 3.8, 3.9 on Linux and Windows)DOC: Updating documentation, setting DEM path as an environment variable
0.3.3 (2021-05-21)#
Migrating documentation to sphinx and readthedocs
Refactoring documentation and ReadMe
Bug correction when loading invalid masks for S2 sensor (fiona.errors.UnsupportedGeometryTypeError)
0.3.2.post2 (2021-05-04)#
Bug when reading cloud mask on Linux
Log update in
product._check_dem_path()
README & documentation updates
0.3.2.post1 (2021-05-04)#
0.3.2 (2021-05-04)#
In case of DEM bands, checks if the DEM is set and exists and raise an exception if not
Setting minimum versions in setup.py and in requirements.txt
ReadMe improvement
Updating docs
0.3.1-post2 (2021-05-03)#
Bug in NDVI formula, typo when passing to xarray
0.3.1-post1 (2021-04-30)#
Correcting a broken link in setup.py
Setting a minimum version of sertit
Footprint:
Fixing a bug in the default function
Optimization for S2 and S2-Theia sensors
Fixing L7 footprint which was too complex due to nodata stripes
CI:
Testing footprints and extent
Resolving non update of gitlab documentation
Changing the resolution of CI processes (to a multiple of the real resolution to speed up the computation)
0.3.1 (2021-04-29)#
Multiple optimizations when reading and processing the bands
Bug resolutions:
S2-Theia masks
SAR DSPK bands always recomputed
SAR nodata set to 0 as SNAP expects it
Bad mask nodata setting when computing invalid pixels
Inverted default resolution between L4/5 MSS and TM sensors
Removing useless logs when missing DEM but no computed DEM bands
Adding copyright headers to every python files
Fixing and adding examples
0.3.0 (2021-04-28)#
Going Open Source