My first experiences with Sentinel-2A data

First, some pretty pictures

Dublin, Ireland on 8 December 2015

Operational ESA Sentinel-2A data recently became available for public download from the Sentinels Scientific Data Hub.  I managed after several attempts to download a 3.2 GB sized zip file containing data from Tuesday, 8 December 2015. After extracting the zip file, I opened the data using ESA SNAP 2.0 with the Sentinel-2 Toolbox. After selecting true colour bands at 10 meter resolution and some minor processing, I produced a lovely image of Dublin, Ireland and its surroundings- urban areas, farms, upland bogs, clouds at different altitudes as judged by the distance to their shadows, and suspended sediments and humic and tannic acids in the Irish Sea.

Drogheda, Ireland on 2015

In the image above of Drogheda, we can clearly see plumes of humic and tannic acids (dark brown in colour) emptying from the River Boyne into the Irish Sea, accompanied by suspended silts and clays (light brown), which can also be seen from a lesser extent to the south from the River Nanny. The humins and tannins in the water give it a distinctive tea colour, and are leached from upland bogs following heavy rains:

View this post on Instagram

A post shared by Guy Serbin (@dr_guy)

View this post on Instagram

A post shared by Guy Serbin (@dr_guy)

So how did I make those pretty pictures in SNAP 2.0?

Extracting the data

First and foremost, ESA loves excessively long filenames. In fact, they’re so long, that Windows has a problem with them, as it doesn’t allow names that are longer than 260 characters. Now if you’re a Windows user like myself and like a decent directory tree structure, you’re going to get this error when trying to open Sentinel zip files:

Problems unzipping Sentinel-2A data on Windows

The solution? Use only one directory level above root, e.g., C:\S2A. Once you’ve done that, you can open the scene in SNAP using the XML file, and choosing the resolution (native, or with all bands resampled to 10, 20, or 60 m). For this exercise I chose native, but note that you won’t be able to display bands that have different native resolutions together this way.

SNAP Desktop 2.0

You’ll see the product available in the Product Explorer Tool Window:

This is how you can display an RGB image.

And then you can choose between Sentinel 2 MSI natural or false colour images. When you open it it up, the default stretch may be biased by cloud cover:

What you’ll initially see in a very cloudy scene

Thus, you may have to adjust the stretching by using the band histograms:

Using band histograms to remove the cloud effects on display

And after adjusting the red, green, and blue bands you end up with a much nicer looking image:

Now we can see the land and the sea

Now you can zoom in, out, and pan around the image. If you want to save a copy of image in JPEG, PNG, or another format choose “/File/Export/Other/View as Image” from the menu. Alternatively, you can export to other remote sensing formats using /File/Export.

Problems with the data

Downloading:

Getting scenes from the Sentinels Scientific Data Hub is somewhat problematic as Sentinel-2A data can only be downloaded as swaths, not individual 100 × 100 km granules. The smallest Sentinel-2A file for Ireland that I encountered was at least 3 GB compressed, and some were over 7 GB in size. As such, this greatly increases demand on their bandwidth as users are downloading a lot more data than they actually want, and downloads have broken for me more often than they have succeeded. As I am interested in clear land pixels, I would likely ignore granules that were mostly or completely cloud covered, or were all water. I contacted the ESA Sentinel Help Desk about this, and they said that granule download should be available sometime in 2016.

File names:

Zip files

Unlike with Landsat scene identifier strings, which are relatively short and to the point, Sentinel-2A filenames and granule identifiers are excessively long and not very useful for the end user. For example, the data displayed above came from a zip file named S2A_OPER_PRD_MSIL1C_PDMC_20151208T182059_R123_V20151208T114642_20151208T114642.zip. Within this name there are some useful data- the satellite (S2A), sensor (MSI), processing level (L1C), processing time (20151208T182059), relative orbit number (R123), and acquisition start time (20151208T114642). There’s also an end time in there, but it is consistently the same as the start time for all of the files. I’ve noticed that multiple files from the same orbit can share the same start time, but the processing times all differ, and as such, it’s a unique identifier. I guess that OPER_PRD means “operational product” along with the site centre (PDMC) . As such, I think the filename could be shortened to S2A_MSIL1C_20151208T182059_R123_V20151208T114642.zip.

Granules

The granules contained within this file are also quite long: S2A_OPER_MSI_L1C_TL_SGS__20151208T170410_A002408_T29UPV_N02.00.  Within these are some data that I consider of interest:

1. Satellite – S2A
2. Sensor – MSI
3. Processing level – L1C
4. Processing centre – SGS_
5. Acquisition date – 20151208
6. MGRS (US Military Grid Reference System) tile name – 29UPV

I really don’t see much use in the rest of the information there- again, it’s best placed in an XML metadata file unless there is an operational need for the end user to have this there (e.g., a given tile may be imaged more than once on a given day), in which the absolute orbit number (A002408) would become of interest. However, it could be shortened in a similar fashion to Landsat scene identifiers, as a string of SSSXMMMMMYYYYDDDGSIVV_PPP, where:

1. SSS is the satellite (S2A)
2. X denotes the sensor (M for MSI)
3. MMMMM is the MGRS string for the granule (29UPV)
4. YYYY is the acquisition year (2015)
5. DDD is  the acquisition day of year (342)
6. GSI denotes the ground station where the data were collected, following USGS conventions (unsure about this, am using XXX as filler)
7. VV = Archive version number of the granule (00 for a first version)
8. PPP the processing level of the granule data (L1C).

Thus, the granule identifiers are reduced to S2AM29UPV2015342XXX00_L1C, which is much more useful for us end users. This would also take care of any issues with unzipping files on Windows machines.

My complaint to the ESA Sentinel Help Desk, and their response

Being that their filenames are problematic to Windows users, I decided to complain, and gave them similar suggestions to the ones above. They quickly replied with some workarounds:

Please note that this is the solution to the too long filename issue:

Windows has a MAX_PATH of 260 characters size fixed.

There is at least 3 ways of working around this limitation:

1) Extract the file near the root of the drive, like in: C:\S1A\. It should handle most of the products

2) If, for many reasons, you need to extract your file in an already long path, for example:

C:\Users\operator\Desktop\products1A\and_a_longer_subdirectory\

Windows should let you extract the file anyway, but you will not be able to operate with the files.

Via the explorer, go to C:\Users\operator\Desktop\products1A\and_a_longer_subdirectory\,

and Shift right click on the directory, you should then be able to select Open Command windows here.

Then the tricky part: type susbt T: C:\Users\operator\Desktop\products1A\and_a_longer_subdirectory\

where T is an unused drive letter (A is generally free too).

To quickly get your path, you can left clic on an empty space in the address bar and Ctrl C, then right click on the cmd windows and paste.

press enter.

You have created a new virtual drive, named T:, with direct access to the products.

The method is long but very handful if you have many products stored in the same location. It will last until the windows session is closed.

3) The quick way to fool windows around:

In your explorer, where you extracted the product, left clic on an empty space in the adress bar, go full left, before the drive lettre, and add : \\?\ , then press enter.

ex : C:\Users\operator\Desktop\products1A\and_a_longer_subdirectory\

[1]\\?\C:\Users\operator\Desktop\products1A\and_a_longer_subdirectory\

Windows will work with full unicode path for this processus, letting you create, rename, or edit files with pathes up to 32,767 characters.

This is a temporary fix, should be done each time you reopen an explorer window.

I hope this helps.

I also hope they passed my email along to whomever can effect my suggested changes- they would make the products much more usable for the geospatial community.

Update

21 December 2015: I just posted on how to download individual Sentinel-2A granules using QGIS.