This should help memory some more. See https://github.com/turt2live/matrix-media-repo/issues/237

Theory is that by dumping the streams we'll relieve any bytes held on by the S3 libraries. 

Might reduce memory usage over time

Fixes https://github.com/turt2live/matrix-media-repo/issues/84

Part of https://github.com/turt2live/matrix-media-repo/issues/209

Part of https://github.com/turt2live/matrix-media-repo/issues/209

Fixes https://github.com/turt2live/matrix-media-repo/issues/170

See https://github.com/matrix-org/synapse/issues/4242 for some information - we were previously vulnerable to the short TTL problem. If someone were to take sufficient control of DNS, they could trick us into doing requests to a blacklisted host.

This also fixes a vulnerability where OpenGraph images were not passed through the same validation rules.

**Caution**: This is a major migration that changes how files are tracked within the database. Although tested to ensure things won't go wrong, there is the possibility of something going haywire. It is recommended to run the following database commands before upgrading:

CREATE TABLE media_backup AS TABLE media;
CREATE TABLE thumbnails_backup AS TABLE thumbnails;

In the event you need to roll back, run the following:
DROP TABLE media;
DROP TABLE thumbnails;
CREATE TABLE media AS TABLE media_backup;
CREATE TABLE thumbnails AS TABLE thumbnails_backup;
DELETE FROM gomigrate WHERE migration_id = 7;

This will cause any files uploaded between upgrading and rolling back to be lost.

NOTE: The migration will take a while and runs on the first access of a file. If you have a lot of files, this could take a while.

-----

Fixes https://github.com/turt2live/matrix-media-repo/issues/106
Step towards https://github.com/turt2live/matrix-media-repo/issues/47

Part of #58

Downloads go through quite a few layers, and some of those layers are copy/pasted. Needs some refactoring, but it works.

Part of #58

The middle layer is being completely replaced. We're keeping it for reference for now though.

Part of #58

They are a shared resource, so treat them as such

Fixes #61

with optional config option. Fixes #24

Adds #13