Cannot download backup before it's removed from server

[tpluscode]

I'm trying to download a backup of a project which is about 1.2 GB. My connection is not too fast but the download should take about 3-4 minutes

I tried multiple times but the download get interrupted and if I try to download again, the export link returns 404 which seems to indicate that it's getting removed from the server. And indeed, if I refresh the export page, it says "No exports". Each time, the export is stopped shortly after 1 GB mark, if that's not a coincidence.

Is there an issue here? How long should and export be retained?

Running v32.0

[reckart]

tl;dr solution: check that your nginx configuration includes the line:

proxy_max_temp_file_size 0;

Thanks to @tpluscode for looking up the sources for this. Thanks to @jcklie for having originally included this in the nginx instructions of our Admin Guide long time ago 😉.

---

Looks like currently the file is retained until when the connection used to download it is terminated or until temporary files are deleted (typically at the time the JVM is shut down). However, the export job that links may have a different lifecycle (e.g. I'm not sure at the moment if it is shared across users).

[tpluscode]

It was suspicious that the download would break each time around 1 gb. I found that it may be caused by nginx proxy cache which in turn fools the server into thinking that the download has already been consumed.

My immediate solution was to set proxy_max_temp_file_size 0; in the nginx config to disable the cache at the expense of the upstream server.

That said, I think that the download expiration is prohibitively short. Could it be extended? Configurable maybe?

[reckart]

What do you mean by expiration?

[tpluscode]

You said

Looks like currently the file is retained until when the connection used to download it is terminated or until temporary files are deleted (typically at the time the JVM is shut down).

The nginx proxy appears to interfere with the former, possibly explained here. The latter is a bit unpredictable too.

By expiration I mean how long the exports are kept on the server. I would have expected some kind of fixed time after they would be deleted. One hour? 24 hours?

[reckart]

Normally, when you press "Download", the job should disappear from the list the moment that the download completes (or that the connection is closed). I didn't think there would be a point in keeping the file around after it has been downloaded. #

If you do not download the file, the export will be cleaned up by a scheduled job after 30 minutes or when the JVM is shut down. The cleanup interval is currently not configurable, but could be easily made configurable.

[tpluscode]

Clearly another nginx quirk. A workaround is available so I wouldn't really worry about this now :)