More updates working on implementing a mod_rewrite\Apache implementation of mod_secdownload – just finished coding in a public key encryption verification for the script, basically the same method as implemented in mod_secdownload. Whew, I’m pooped, goin’ to sleep.

Here’s the problem. I have a handful of files between 500MB-800MB that I need to host up on my website. I need to get these files to people that belong to a certain vBulletin community, and only people of that vBulletin community. Furthermore, the people of that community aren’t the sharpest knives in the drawer, so it has to be easy. I have limited resources, which include a shared LAMP webserver, and a 150/150 VPS.

The requirements? Just a few:

• Must be able to track users to sessions to monitor and log abuse
• Links generated should be anti-leech and secured with one-time use tokens
• Links generated can only be accessible from this vBulletin community
• Links must be easy to use

The limitations? It’s like being handcuffed to a telephone pole:

• Hosting is done on shared hosting (no persistant readfile() sessions in PHP or other ugly CPU intensive scipting)
• Hosting can also be done on 150/150 VPS. I’ve used lighttpd, but it chokes on sending files that are bigger than my RAM capacity

Things I’ve tried, and failed to work:

• Doing the classic PHP readfile() and writing to the buffer
  • Far too slow
  • Persistant PHP session isn’t nice to shared hosting
  • readfile() in lighttpd and Apache cause the process to consume massive amounts of memory for large files, doesn’t work in VPS
  • More than 5 users or so and the load spikes to high heaven. Server /suicides
• Using lighttpd and mod_secdownload
  • Far faster than PHP readfile()
  • Will only work on VPS as shared hosting is Apache only
  • If you don’t have enough RAM (which I don’t on the VPS) – the process with explode and hang. I can’t kill it, and the server refuses connections until the watchdog resets the process or until I reboot the VPS. Server /suicides again.
• Using mod_rewrite in conjunction with rewritemap and prg in Apache
  • Sounds perfect – you can create tokens, and then revoke them with the rewritemap program, then use mod_rewrite to hide path source!
  • …Until you realize that you need to modify your apache.conf to specify the rewritemap program (requires root or sudo, not happening on either VPS or shared)
  • Your rewrite program is executed on server start, and is persistent – which is not allowed on shared hosting anyways. I /suicide
• Apache with mod_xsendfile
  • Can’t install custom modules on Apache in shared or VPS hosting
  • Apache is still a fat hog of a server, and using Apache to send binary files is about as efficient as using Humvees for mass transit.
• Dynamic mod_rewrite definitions in .htaccess files
  • Use a manual lockfile with a SQL database to dynamically generate .htaccess mod_rewrite definitions
  • Each rewrite entry for a unique URL a time limit
  • Each successful request for token updates both the database and the .htaccess file
  • This seems like the best solution – it will work with the most hosting environments too
  • This behavior is basically emulating mod_secdownload for lighttpd in Apache with mod_rewrite

The last entry (emulating mod_secdownload in Apache with mod_rewrite) looks good, and is probably going to be what I’m going to use – I’ll be posting the results here as soon as I finish coding it. Weeeee!

http://lifehacker.com/5237503/five-best-free-data-recovery-tools

For once lifehacker has something useful posted. I’d like to add that I’ve also used PC Inspector File Recovery with success too.

If you’re running a CMS with dynamic redirection using mod_rewrite on DreamHost, you may run into a problem trying to access your statistics – since the .htaccess file specifies files and links, and not server configuration redirects like the /stats/ directory, you can be in a bit of trouble. When you try to access the /stats/ virtual directory, the url will get rewritten, your CMS will get confused, and you will get an error instead of seeing your precious website statistics. I had been using Google Analytics, but that requires that your website’s code be loaded in order for the stats to be generated. This does not catch hotlinking or deeplinking into images or other media. 

I had been searching for weeks on how to write a mod_rewrite exception, but anyone that knows mod_rewrite code knows it will make you want to jump off a cliff. It’s honestly almost as bad as writing a config file for sendmail or bind before they introduced the structued layout.

Today I had a thought, since this problem is nearly unique to DreamHost, why not search there for potential solutions to the problem? 10 minutes of digging in the DreamHost wiki and I had my solution – Making stats accesible with htaccess – this simple rule inserted before your permalink rules allows a catch and exception for the /stats/ virtual directory:

RewriteEngine On
RewriteBase /
RewriteCond %{REQUEST_URI} ^/(stats|failed_auth\.html).*$ [NC]
RewriteRule . – [L]