Few months ago I saw the tweet from @gandalfar
My understanding was that Ma.gnolia is a bookmarking service (like del.icio.us), they lost all the data and so Gandalfar lost all the bookmarks he was saving into it for years(?). The whole thing is extremely absurd because the only functionality of bookmarking services is to remember what you save to them, and they lost it all…
When I saw that, I got an simple idea. I liked it and was determined to try it at the first chance I get:
The whole fiasco with Magnolia would not happen, if the service would send emails with attachment to all their users weekly. Attachment would hold a gzipped file with all user’s data in some general format. So even if service goes to hell the users still have their data.
In such case user could re-import data into the original service or simple tools could be made that would import data into other similar services or transform it to general file with format of their liking.
The important thing being: User would have his data and a choice to do with it whatever he wants!
With this you effectively create another backup system for your service. The one that is not in any way centralized and works on the level of peers / users.
I will use this technique with cebelca.biz , a very lightweight business web-application that we are finishing. The data dumps will also be encrypted with user’s passwords in this case.
Related articles by Zemanta
- Care For Your Personal Data In the Cloud (cloudave.com)
- HOW TO: Back Up Your Tweets (mashable.com)
- Pinboard brings back the glory days of bookmarking, for a fee (downloadsquad.com)
![Reblog this post [with Zemanta]](http://img.zemanta.com/reblog_e.png?x-id=0d0519fb-c8f7-4797-9429-f3edf9818a8a)
12 Comments
Dear god, please no.
This might be reasonable when attached data is few KB big, but quickly becomes a burden when not and can get quickly worse if service becomes popular.
Well, it’s not one-shoe-fits-all solution but I think it can be applied and extended/optimized in certain cases. If quantity of data is bigger and service popular it is additional burden, but this is a question of weighting between positives and negatives.
For my example. I absolutely can’t *ever* afford to loose user’s data at cebelca.biz and the cost of sending even few MB-s of data each week per user is not cruicial so I will offer this option to them.
some random thinking outloud:
- at Site Assistant data file that is of size 1.7MB ( a user with around 100 domains and all the generated data + history) compressed to 100kB
- A company I made business app for had around 2000 invoices per year + similar other docs + few 1000 articles and their data files if zipped at the end of year were always only few MB (that was MS Access many years ago, I still need to support it)
- You could send emails only when some certain number of changes occurred (for example in a bookmarking app), most users don’t use it so intensely I suppose.
- If app is free it can be a “deluxe” payed option where even $0.5 can more than compensate sending few megs 4 times a month
Your critics are very welcome otherwise, they make me/(us all) think and I admit I wasn’t thinking about whole scale/cost thing much before.
I agree with your reason, not with your method. This is a common misconception, e-mail is not designed to keep your files and sending backups automatically with email builds further upon this misconception.
I however am a big fan of API’s and I loathe services like Facebook that do not recognize that it’s MY DATA they are profiting from and I might want to have a copy of MY FRIEND LIST, MY PHOTOS, MY PHONE NUMBERS and other information that should be easily exportable.
Google so far has gone out of its way to provide a way to connect to their backends. Even if they store all my emails, I can still fetch them with pop3. Even if my calendar is stored there, I can still fetch it through iCal if I want.
For social bookmarking, you should be able to access your bookmarks in an XML file (with a api key) and do with it what ever you like. Like creating a scheduled event that fetches it every day.
Myself, I don’t use social bookmarking for this very reason. I use x-marks with my very own webdav server to synchronize my different computers.
Anyways, good writeup. Just my 5 cents.
I like the idea of letting users have easy access to their own data, but agree this could be a nightmare with a large site.
Maybe a reasonable compromise would be to have an easy way for users to download the data when they want it, or to set up automated downloads on a schedule of their choosing. Then users who really cared about not losing their data, or who wanted to be able to use it elsewhere, would have access to it, while users who may no longer be actively using the site wouldn’t be bothered with a weekly email that they don’t want and don’t need.
Any half decent website owner plans a backup policy and routine.
If you are taking proper backup of user data on your site, you don’t need to burden users with “backing up” their own data, which in essence – sending their data email – would be.
Also, sending their data zipped via email, could also lead into obscure privacy issues.
Store the encrypted backup in Amazon S3 storage, it’s perfect for
medium-to-large files you don’t expect to change in any way once
written. Then send the user a notice the backup is available with the
S3 URL.
The data dumps will also be encrypted with user’s passwords in this case.
Does it mean you will be storing users’ passwords in plain-text? BAD bad idea…
Also, what’s up with all the links to Wikipedia? Seriously, how many internet users need to look up the term ‘Password’?
I like the idea, but it should be made opt-in, and off by default, for reasons explained in the comments above. The thing is, even if you provide your users with an export option, or opt-in e-mail backup, the vast majority of the users will not use these features. If your site crashes, it doesn’t make any difference if 5% of the users have backed up their data.
Also, opt-in and regular exports are extra work, which people just won’t do. Even if it’s a single click away, it’s still one click too many. So you have to find a way to either force your users to make backups (yuck), notify your users to make backups (nicer, but won’t do, and it’s still annoying), or provide a good backup yourself, possibly off-site. The third solution is the only good solution for cases like these, and of course the hardest to provide.
I quite like Ted’s idea; you back it up somewhere online, and only provide a link to that backup. This approach has a few problems (privacy of the backup, safety of that data, spamming with backup notifications), but it’s a very good start.
Putting users in control of their data is excellent practice, but making them responsible for keeping track of the data they gave you probably won’t go over too well. I’d give it a 3-step approach:
1) Have an API that exposes all the data they’d want to back up. Write a client app for importing that data to a user’s desktop, and expose the API- That way if your users have specific needs for that data, they can back it up themselves in a way that’s consistent with their systems.
2) Allow the email thing *as an option*, but make it opt-in, and not the only method for pulling their data out of your webservice. As mentioned above, email for file storage is weaksauce.
2) Automatic offsite backup. Amazon S3, some webserver farm in a neighboring state/country, something like that. If your server crashes, it might be their fault they didn’t use the backup options, but it’s still your fault that their mistake cost them all that data.
Thanks all for very interesting replies! I will try to answer below.
About “why email” vs the other solutions..
I agree email is not the ideal medium for this. But it’s sadly sort of the only one with such properties.
A) On one end you have option where you offer users links to download their data. Nice, totally unobtrusive.. BUT.. I don’t even back-up my important files on local disc until it’s too late. Do you think people would “back-down” data from their bookamrking services on regular basis?
B) On the other end you have a smart client that silently makes backups for them in the background, and also does conversion between formats, etc… So user just says at any point. I want my data in CSV and get’s it. BUT do you think anyone would install a native application to use a online browser based service. That “no download or install” is the key accessibility point why web-services are so popular.
Android:
I agree with you a lot. And I also agree email used like this has concrete flaws, as I tried to explain above I proposed to use it in this because I see no other real option. Google “exporting” to natural formats/protocols like you mention pop3 / iCal is really good approach. I wasn’t even aware of it.
I agree that for tech savy people good api-s that allow you to make your own little solutions are very good. But this only covers smaller amount of internet users. Otherwise I also fully agree with your reasons
BruceA:
Yes your idea is the middle ground. It gives the data and options to users that want them. This is a good and not problematic solution I think.
My proposal was a bit more extreme in trying to make sure users won’t loose their data (even the ones that won’t care or know how to set up ther backuping. ), but by the same reasons it’s also more problematic (which can be seen by your comments).
CaptainOBackup:
Yes, I know decent sites should have good old centralized backup solution. But as you can see not all sites are as decent when it comes to real problems, or their backuped systems fail and user is simply notified that their data is “no more”.
This idea with emails doesn’t mean that you don’t need to have backups. It means that the each user ALSO HAS the HIS data on his disc.
I have some client that I made portal for, the hosting provides daily backup, but every now and then she asks me to manually download backup data of her mysql database and send it to her. When I send it to her she is relieved because she is holding the data now, she can bake it on a CD or whatever, it’s there..
Ted Z:
This is good third option, a little more progressive than “you can download your data”. It would be especially handy or the only option if there was in size that would make problems to regular emails also.
Chetan:
That was my mistake,… we have no plain passwords stored on server yes. I also haven’t concretely looked at how exactly we would encrypt data. Does anyone see any problems in hashed (& salted) passwords being the key for encryption?
Links were proposed to me by Zemanta plugin, I agree I wouldn’t have to click the password keyword to make a link to wikipedia ,.. My fault
gasper_k:
Hm.. your story that is hard to disagree with slowly transports me from “I like this” to “5% solved is of no use” to “The only option is stellar backup by the service itself and forget the emails to users”
In the same way I dismissed the “here is your data / download it” and “install smart client that will backup your data nicely” you dismissed emails with opt-in. I see now things might not be so smooth as I thought.
But, I still have one option, I will try this and see what users of real app say… will they care about it at all, will it cause more confusion than it will be worth, etc.. experimenting is always fun.
Alex:
The first part of your scentence is what I was aiming for. The point of this is NOT that the service doesn’t need to take care of backups any more. It’s more that the user also has the data.
Your 3 points are good, I will try some combination of them and see how it goes. For all of you that mention API , I think API access to your data is very good and more basic feature than the “emails for backup” .. it’s not only usable for backup but also in integration of your app with the outside world.