Ask HighScalability: Web asset server concept - 3rd party software available?
We are serving dynamic (PHP) websites and their assets (JS/CSS/images/videos/binary downloads) via the same apache hosts. The static files are only being used as origins for CDN services used to distribute those files. Yet, in the current development-deploy pipeline, these files are checked into the same version control repositories as the code is. This is what we would like to change, for several reasons (decouple asset deployment from development & developers, lessen size of code repositories, etc.)
My idea is to do the following: Set up a media server (cluster) which serves as an API (REST e.g.). You can PUT files to it, and get back the URL the file is available through from the public. In between input and output, the media service deals with everything that's necessary to serve the files: Upload them to the CDN, create the public URL, write the meta data to a (relational?) database, assign a version number... This API can be used by a) the application/website directly to provide CMS system functionality (upload and reference images directly from the website), and b) the build pipeline to upload and correctly reference the files that are being used by the templating engine directly/statically.
For the media service, a GUI could be implemented which enables management of the uploaded files (CRUD).
What do you think of the idea? Is there any software out there that already does things like that? How do other large website solve the problem of asset hosting?
Thanks in advance.
P.S.: As the question might arise: We are not planning to upload DIRECTLY to a CDN hoster in order to have a complete backup on our servers as well. As soon as the CDN hoster goes down, we could (theoretically) let the media server return its own domain name and deliver the files itself. Or an easy switch to another CDN is also possible, as the information about the CDN URLs would be stored centrally on that media server.