Drop It!

After some serious musing over various CentOS server backup alternatives, I finally settled on the cloud for my modus primo. Obviously the server company could have provided either a secondary HDD or a network share to which I could have SFTP'd out, and I could have also gone third party and used SFTP with them... or even stored just critical data locally... but these options were all limited or costly.

Dropbox is a service I've been using for personal backup for some time, it's a free cloud based system, running from Amazon's S3 backend. It provides 2GB initial data offering, expandable on subscription to 100GB. OK, so it does cost in the upper storage bands, but to me $99 for a year of 100GB cloud storage isn't bad at all. I'm also thinking for people wanting to go above 100GB that Dropbox would happy oblige on a prorated basis.

I used both the headless Tips and Tricks page on the Dropbox Wiki and multiple forum entries to finally arrive at the solution.

My first step was to get the Dropbox client. Dropbox release a stable client on their website and a semi-stable client on their forums. I would recommend the latter; cloud technology is constantly evolving and Dropbox seem to fairly stable semi-stable releases... if that makes sense!

After downloading with wget and extracting to home, I started the process with:

~/.dropbox-dist/dropboxd &

As soon as I had done this, the CLI issued me with a verification link, which I could then paste into a local browser either to setup a DB account or link an existing one. When the program detected that this link had been made, it completed execution and returned me to a prompt. Dropbox was running!

You'll see forum posts and web references to the official Dropbox CLI, which allows for starting/stopping etc. This is a useful script, however requires a very much updated version of Python. This was messy, as I had 2.43 already installed, and built parallel version of 7 in order to execute the script. Once both Python's are installed, you can prefix scripts with python (instead of just running them natively) to allow the higher version to be used. As certain system processes are built around Python, removing 2.43 would require other modules such as Yum to be recompiled as well. It would get very messy. I would actually suggest that you forgo this "official CLI" because in reality you will rarely use it, and the cost is not worth the gain.

So the next step was to manually edit some of the Dropbox settings. Without a GUI, the sqlite3 database has to be manually edited. First off stop Dropbox. If you haven't got your script working as above, then just kill the process.

Switch into your /home/user/.dropbox folder and run sqlite3 dropbox.db or... for newer releases such as mine, sqlite config.db.

The database interface will then load and you are ready to tweak some settings! These first two will allow you to remove the upload/download cap. If you are running on a server then you should definitely do this and take full advantage of your backbone. The third option stops Dropbox broadcasting locally looking for other Dropbox instances to P2P with. This is really important in a server environment, because your server company may see any local broadcasting, especially P2P broadcasting, as dangerous or consumptive. There are both new and old SQL statements below as Dropbox have usefully changed their DB structures between releases. If you have a config.db, then use the new statements, else use the old ones!

Old:

insert into config (key, value) values ('max_upload_kBs', 'RjAKLg==');
insert into config (key, value) values ('max_download_kBs', 'RjAKLg==');
insert into config (key, value) values ('p2p_enabled',0);

New:

delete from config where key='throttle_download_style';
delete from config where key='throttle_upload_style';
INSERT INTO "config" VALUES('throttle_download_style',0);
INSERT INTO "config" VALUES('throttle_upload_style',0);
insert into config (key, value) values ('p2p_enabled',0);

Then just quit and save with .quit. You can use the same process to move the Dropbox storage folder. There are scripts for this, however I found them to be outdated and resulted in corruption. By issuing the following commands as above you can change the location:

delete from config where key='dropbox_path';
INSERT INTO "config" VALUES('dropbox_path','/home/ufblue/Dropbox');

Your final step is to get Dropbox to start at startup and run as a sytem process. If you do this, the CLI script will become redundant because you'll be able to use the service dropbox command to either stop, start or restart the service.

For the service script, I used the script available on the Tips and Tricks page. You should be able to find your distrib there. I used the Fedora/Red Hat script and it worked just fine. It's really important to set the users file correctly if you are using symbolic or soft links, as permissions issues can occur. Dropbox looks at parent directory permissions rather than specific file permissions, so you can work around it that way, but for recursive permission liberated structures, you'll have to run the DB client as root. I'm really not sure of the ramifications of this, and can only entertain that the Dropbox company could potentially exploit this at worst.

If security is a concern, there are always the options of using TrueCrypt or encFS and then Dropboxing the encrypted volume in it's entirety. I can see why this is an attractive option, because you are still risking unencrypted data to both Dropbox and ultimately Amazon's S3 service.

The possibilities are then fairly limitless. Remember the you can use softlinks to link files into your Dropbox directory system wide, and these will then be included in the backup. Remember that Dropbox contains a 30 day revision history, so every time a file is changed, it's revision is stored. This got me to consider the possibility of Dropbox watching an active mySQL database. I'm not sure how this would work however and even if the files would be locked? There is no data about this on the net right now so it might be worth experimenting.

Consider the additional server based possibilities though, from torrents, to email stores, to databases, to VM instances... even more if you are running a local client and then you have a smooth replacement for SFTP or WebDAV... the options are limitless, and would provide an outstanding interface for any client. It's only a matter of time before we see more server-side application of this, but beware that although this will bring ease of use, it will also bring with it cost as people start to recognize it's value. DB is already more than Googleworthy, and I am sure they've been approached, but I guess for now they are holding out for their own reasons... for now.

Dropbox on their part are doing a stellar job at promoting the various uses, and don't seem to mind any of it's applications. There is even a DB client for smartphones now, enabling you to literally sync on the go. To me, effective cloud storage application is an important a milestone as VM. Your store of information is now truly dynamic, truly transversal. Information has ascended.