Secure, fast, low-resource file delivery

Here’s the problem. I have a handful of files between 500MB-800MB that I need to host up on my website. I need to get these files to people that belong to a certain vBulletin community, and only people of that vBulletin community. Furthermore, the people of that community aren’t the sharpest knives in the drawer, so it has to be easy. I have limited resources, which include a shared LAMP webserver, and a 150/150 VPS.

The requirements? Just a few:

  • Must be able to track users to sessions to monitor and log abuse
  • Links generated should be anti-leech and secured with one-time use tokens
  • Links generated can only be accessible from this vBulletin community
  • Links must be easy to use

The limitations? It’s like being handcuffed to a telephone pole:

Things I’ve tried, and failed to work:

  • Doing the classic PHP readfile() and writing to the buffer
    • Far too slow
    • Persistant PHP session isn’t nice to shared hosting
    • readfile() in lighttpd and Apache cause the process to consume massive amounts of memory for large files, doesn’t work in VPS
    • More than 5 users or so and the load spikes to high heaven. Server /suicides
  • Using lighttpd and mod_secdownload
    • Far faster than PHP readfile()
    • Will only work on VPS as shared hosting is Apache only
    • If you don’t have enough RAM (which I don’t on the VPS) – the process with explode and hang. I can’t kill it, and the server refuses connections until the watchdog resets the process or until I reboot the VPS. Server /suicides again.
  • Using mod_rewrite in conjunction with rewritemap and prg in Apache
    • Sounds perfect – you can create tokens, and then revoke them with the rewritemap program, then use mod_rewrite to hide path source!
    • …Until you realize that you need to modify your apache.conf to specify the rewritemap program (requires root or sudo, not happening on either VPS or shared)
    • Your rewrite program is executed on server start, and is persistent – which is not allowed on shared hosting anyways. I /suicide
  • Apache with mod_xsendfile
    • Can’t install custom modules on Apache in shared or VPS hosting
    • Apache is still a fat hog of a server, and using Apache to send binary files is about as efficient as using Humvees for mass transit.
  • Dynamic mod_rewrite definitions in .htaccess files
    • Use a manual lockfile with a SQL database to dynamically generate .htaccess mod_rewrite definitions
    • Each rewrite entry for a unique URL a time limit
    • Each successful request for token updates both the database and the .htaccess file
    • This seems like the best solution – it will work with the most hosting environments too
    • This behavior is basically emulating mod_secdownload for lighttpd in Apache with mod_rewrite

The last entry (emulating mod_secdownload in Apache with mod_rewrite) looks good, and is probably going to be what I’m going to use – I’ll be posting the results here as soon as I finish coding it. Weeeee!

    You may also like

    1 Comment

    Leave a Reply

    Your email address will not be published. Required fields are marked *