AWS

Let’s read satellite data stored on AWS.

This tutorial is a bit tricky as EOReader uses rasterio and cloudpathlib libraries and they are currently handling S3 buckets differently. That’s why we’ll use the helper context manager temp_s3 from sertit.s3, that will do the right connections for us.

Note: These Sentinel-2 products are not stored in the .SAFE format. Landsat format is howver the same.

Cloud data processed by Element84: Sentinel-2 L2A as COGs

See this registry (arn:aws:s3:::sentinel-cogs). This registry is open access, so you don’t have to sign-in.

# Imports
import os
import tempenv
import logging

from sertit import logs, s3
from eoreader.reader import Reader
from eoreader.bands import BLUE

with tempenv.TemporaryEnvironment({
    "AWS_S3_ENDPOINT": "s3.us-west-2.amazonaws.com",
}):
    with s3.temp_s3(no_sign_request=True):
        logs.init_logger(logging.getLogger("eoreader"), logging.DEBUG)
        path = r"s3://sentinel-cogs/sentinel-s2-l2a-cogs/39/K/ZU/2023/10/S2A_39KZU_20231031_0_L2A"
        prod = Reader().open(path)
        prod.plot()
        blue = prod.load(BLUE)[BLUE]

2023-11-20 13:46:49,201 - [DEBUG] - Loading bands ['BLUE']
2023-11-20 13:46:49,403 - [DEBUG] - Read BLUE
2023-11-20 13:46:50,948 - [DEBUG] - Manage nodata for band BLUE
2023-11-20 13:46:50,949 - [DEBUG] - Converting BLUE to reflectance

blue[:, ::10, ::10].plot(cmap="Blues_r")

<matplotlib.collections.QuadMesh at 0x7fdd7c580f10>

Cloud data processed by Sinergise: Sentinel-2 L1C

See this registry (arn:aws:s3:::sentinel-s2-l1c). This registry needs authentication.

NB: L2A would have been the same (arn:aws:s3:::sentinel-s2-l2a)

Note: Sinergise data are stored as requester pays in AWS. Don’t forget to state this when requesting data!

with tempenv.TemporaryEnvironment({
    "AWS_S3_ENDPOINT": "s3.eu-central-1.amazonaws.com",
    "AWS_SECRET_ACCESS_KEY": os.getenv("AMAZON_AWS_SECRET_ACCESS_KEY"),
    "AWS_ACCESS_KEY_ID": os.getenv("AMAZON_AWS_ACCESS_KEY_ID"),
}):
    with s3.temp_s3(requester_pays=True):
        path = r"s3://sentinel-s2-l1c/tiles/10/S/DG/2022/7/8/0"
        prod = Reader().open(path)
        prod.plot()
        blue = prod.load(BLUE)[BLUE]

2023-11-20 13:49:28,993 - [DEBUG] - Loading bands ['BLUE']
2023-11-20 13:49:29,040 - [DEBUG] - Read BLUE
2023-11-20 13:49:29,887 - [DEBUG] - Manage nodata for band BLUE
2023-11-20 13:49:30,180 - [DEBUG] - Converting BLUE to reflectance

Cloud data processed by USGS: Landsat-8

See this registry (arn:aws:s3:::usgs-landsat). This registry needs authentication.

Note: USGS data are stored as requester pays in AWS. Don’t forget to state this when requesting data!

with tempenv.TemporaryEnvironment({
    "AWS_S3_ENDPOINT": "s3.us-west-2.amazonaws.com",
    "AWS_SECRET_ACCESS_KEY": os.getenv("AMAZON_AWS_SECRET_ACCESS_KEY"),
    "AWS_ACCESS_KEY_ID": os.getenv("AMAZON_AWS_ACCESS_KEY_ID"),
}):
    with s3.temp_s3(requester_pays=True):
        path = r"s3://usgs-landsat/collection02/level-1/standard/oli-tirs/2021/016/042/LC08_L1TP_016042_20211027_20211104_02_T1"
        prod = Reader().open(path)
        prod.plot()