Empty zip-file retrieved from scihub.copernicus.eu/dhus

[kristianpagh]

Trying to run this on my local PC. Firstly I found that I need to uncomment the option for downloading S3 data in S3_wrapper.sh, rather than the default use of -l $SEN3_local . I guess that this could also be a challenge for other users. Here a switch for local vs download of S3 data could be an option!

The actual error I get is after this command:

wget --show-progress -nc --continue --user=kristianpagh --password=*
'https://scihub.copernicus.eu/dhus/odata/v1/Products('\''77c272b9
-e79e-4265-8fa6-645901b04e93''')/$value' -O /home/kpn/data/SICE/S3/2021
/2021-08-01/S3A_SL_1_RBT____20210801T184022_20210801T184322_20210803T044
059_0179_074_341_1620_LN2_O_NT_004.zip

..., which results in an empty zip file:

--2022-02-14 20:44:47-- https://scihub.copernicus.eu/dhus/odata/v1/Prod
ucts('77c272b9-e79e-4265-8fa6-645901b04e93')/$value
Resolving scihub.copernicus.eu (scihub.copernicus.eu)... 146.59.215.226
Connecting to scihub.copernicus.eu (scihub.copernicus.eu)|146.59.215.226
|:443... connected.
HTTP request sent, awaiting response... 401 Unauthorized
Authentication selected: Basic realm="OData service"
Reusing existing connection to scihub.copernicus.eu:443.
HTTP request sent, awaiting response... 202 Accepted
Length: 0 [text/plain]
Saving to: ‘/home/kpn/data/SICE/S3/2021/2021-08-01/S3A_SL_1_RBT____20210
801T184022_20210801T184322_20210803T044059_0179_074_341_1620_LN2_O_NT_00
4.zip’

0K                                                        0.00 =0s

2022-02-14 20:44:48 (0.00 B/s) - ‘/home/kpn/data/SICE/S3/2021/2021-08-01
/S3A_SL_1_RBT____20210801T184022_20210801T184322_20210803T044059_0179_07
4_341_1620_LN2_O_NT_004.zip’ saved [0/0]

Here is the full log:

S3_wrapper.log

Can you see what I am doing wrong?

My full S3_wrapper.sh script can be seen here:

S3_wrapper.sh.txt

(renamed to .sh.txt since github doesn't accept upload of .sh files)

[AdrienWehrle]

Hi Kristian,

thank you a lot for creating a Github issue in parallel to our initial discussion.

I just reproduced your issue on Ubuntu 16.04.6 LTS on PolarTEP. Working on it and will keep you updated!

[AdrienWehrle]

e.g.

wget --no-check-certificate --user=USERNAME --password=PASSWORD "https://scihub.copernicus.eu/dhus/odata/v1/Products('85a3389a-6848-466f-828c-ab1ccdced893')/$value" -O ./test.zip

downloads a test.zip which is not empty but can't be properly unziped (results in a End-of-central-directory signature not found similar to your log)

[AdrienWehrle]

I faced this issue again today and managed to fix it using wget with the following arguments:

wget --show-progress -nc --continue --no-check-certificate \
                 --user=${username} --password=${password} \
                 "https://scihub.copernicus.eu/dhus/odata/v1/Products('${uuid}')/\$value" \
                 -O ${outfolder}/${filename}.zip

@kristianpagh could you try and let me know if that also works for you?