Development Seed Blog

How We Got Back Ups and Restores Working in an On a Stick Environment

As we talked about here and here, we recently built a dynamic disaster relief kit in the form of a Drupal website on a usb thumb drive for the American Institute of Architects’ Communities by Design initiative. Users are able to add and edit content in the kit, so we wanted to make it possible for users to create a backup of the Drupal on a Stick database and, when necessary, restore the database from one of the available backups.

Here's how it looks:

This sounds simple enough, and in fact there are some useful options out there for running backups when operating a Drupal installation in normal environments (e.g. LAMP on a hosted server). However, the constraints of running in an "on a stick" environment meant we needed a custom solution. Why? To begin with large numbers of writes and reads from the flash drive take quite a long time. Also, we wanted to avoid the classic situation of creating a  snake that eats its own tail - we didn't want to attempt to fully restore a Drupal database from within Drupal itself.

So, we opted to create a simple stand-alone php script. Whenever a backup or restore is called for, it checks for a MySQL server using good old

mysql_connect()

and then uses system() to run either

mysqldump --opt -u $user -p $password < $backup_file
...in the case of a backup, or both...

mysqldump --opt -u $user -p $password > $backup_file

...and...

mysql -D $db -u $user -p $password < $backup_file
(Note: some parameters were omitted for clarity)

...in the case of a restore. For restores, we ran both actions to preserve the latest version of the database in case there is a problem during the restore.

In our solution backup files are stored in a pre-defined directory. Each file is named according to the UNIX timestamp of when it was created so that as php scans the backup directory, it can list backups by formatting the filename with php's date() function. As a convenience to the user, the script also queries the Drupal database to see if there have been any changes since the most recent backup. Since we know the backup's date, all we need to do is this:

SELECT COUNT(nid) FROM node WHERE changed > $filename

The result is that users can select which backup to restore from using a select element:

Finally, as a nice touch for users, we added a progress bar for both the restore and the backup functions, complete with a warning that these processes may take a while to complete.

A little bit of javascript can go a long way to keeping users informed. I also want to add that the benefits of all of these interface features are amplified by Young’s fantastic design work, which you can see here.

Since we control the "on a stick" environment, we can specify just about everything: the ports, the locations of the binaries, and the directory that holds the backups files. We decided to go with separate locations for Mac and PC binaries, since as Jeff explained, we're running two different server environments (Macs run off MAMP and PCs use XAMMP).

Testing this locally went well. The DB file for this project is small (2.3MB), and backups and restores were quick at less than 0.5 seconds. However, testing on the stick was significantly slower, to the point where merely backing up that same 2.3MB database was triggering php's execution time limits.

As I mentioned above, a universal constraint of developing applications to work in an "on a stick" environment is that reads and writes are much slower than with a conventional disk. We got around the php execution timeout by simply editing php.ini like so:

max_execution_time = 600.

This, combined with the warning to users that this takes awhile, solved the problem for this particular application.

Getting backups working in the "on a stick" environment was definitely a challenge, but as with every production environment, it has benefits along with the limitations. Luckily, we were able to harness these benefits (a high degree of control) to get around the roadblocks (slow writes and reads).