I migrate to the cloud – and march into a swamp

2011-07-11

Motivation
Microsoft finally has killed the Word 2000 installed on my (3 years old) laptop: the mouese cursor is constantly flashing between hourglass and arrow, annoying. I checked the newest Word, and – as expected – all controls are completely changed, I cannot even find the menu bar. Since for me text editing was perfect even in the very first Write on the first Macintosh, and everything which followed later was rather a complication than an improvement (which I duely ignored). I have to start learning text editing from scratch, and as Microsoft has cut all strings, I’m free now to look out for something completely new.

I have a version of Open Office installed (some students send their homeworks in all weird formats); I try this and the cursor in the writer is only visible if I type, but disappears immediately when I stop. However, I think I will continue to use it for backward compatibility, maybe a new version.

Generally, I have the feeling, it is time for a larger leap: directly into the CLOUD.
My requirements:

  • I can keep all my existing 47000 Files with 15 GB
  • Full text searchable
  • and editable
  • Local copies of the files for offline use
  • Useful as a backup solution also
  • resonable price (though I’m happy to pay for good service – I think programmers should not work without reward, and I hate adds)
  • Accessible from my Android Phone (local sync not necessary)
  • Searchable from my Android Phone

options I check:

  • DropBox has, with regards to the WEB, the best syncing engine. It keeps a local “Dropbox” folder in sync with a cloud storage at a dropbox server.  The hash code system it uses should avoid new uploads of files when they are moved or renamed (I did’nt check this however). For my 15 GB, I would need the cheapest payed account, some EUR 25 if I remember correctly; there is an Android app to access the files, and DocumentsToGo has integrated access.
    Unfortuately, it has no full text search feature. Thus, it is worthless for me.
  • Google Docs (and online storage): I need the cheapest payed account, ~EUR 5 for 20 GB. Unfortunately, there is no built-in local sync, and even no bulk upload of my 47000 files is possible. Still it is my choice, since it has a search engine, and I think it will be easier for Google to add the DropBox functions than for DropBox to add the GoogleDocs functionality. I expect the Android functionality of GoogleDocs to be perfect; DocumentsToGo has also Google Docs integration.  Third-party tools for syncing exist.
  • Memeo Connect probably the most advanced version, but I apparently need a Google Apps acount for that – and not a plain Google account – this is a difference! An Apps account seems to require an administrator,  a registered internet domain, and still there seem to be some limitations concerning Android integration etc. So not for me.
  • Syncplicity is a clud storage provider similar to dropbox, but with the option to sync with Google Docs also. However, to use it, I would need two payed accounts: one at Syncplicity, one at Google. No, thanks.
  • Insync is a startup one year old; it has a beta out, which is a tool to sync with GoogleDocs, however, one needs an invitation from another already registered beta tester. Additionally, after bold claims, the project seems to be frozen since a few month now – no more activity on the WEB?
  • SyncDocs: like Insync, but even newer, and active. My choice – I try it out.

I buy additional storage at google, and the acknowledge page says that it may take about one day before I can access it. Actually, Google Docs shows the new space after some 5 min. I install SyncDocs, and decide to put all my data on my laptop into a GooDocs folder inside my Documents folder. From my upload speed I calulate some four full days to upload the data. I’m on holidays now, so it should be possible to have the computer running day and nigth. This will also be required because Windows has to rebuild it’s Search index also because of the data move, I think.

2011-07-17

Syncing is apparently ready. It continued working for seven days, with the computer shut down in the mean time several times, and through different internet connections. I tseems however to continue forever with repeated “Querying Google Documents” and ~15% CPU load and some 25 kB/s network traffic. If this stays like that, it is not acceptable; let’s see. I change the settings so that only my “Work” and “Private” folders are synchronized. This pauses the syncing, and I restart it, it again does something. However, after about one hour it now shows “all files are up to date”🙂

I check the search feature. When I search “medikamentenliste” both on in Google Docs (in Firefox) or on my Andorid Google Docs app, it finds the same four pdf’s, which were some time ago created by photographing papers of my mother with the phone and OCRing with ABBYY FineReader. When I search the name of a student who participated in some students lab of me, it finds all eleven or so .pdf homeworks, and the .xls where I did the administration of that lab. The Windows PC, however, finds locally also some 20 e-mails archived as .eml; apprently, Google Docs does not index .eml. More severe, further tests reveal that it does not index .txt and .wri; I use these formats often to take quick notes. This is really a disappointment.

I try out the DocumentsToGo integration. After clicking on the Google Docs link, it shows a “Please wait” for quite some time. When I check after a few minutes, it shows an error message, and says “Please try again”. I do so; again “Please wait…”

-*-*-*-*-

Desaster

When trying to find some documentation on the weird “Collection” philosophy of Google Docs, which are neither folders nor tags, I find the forum contributions by the following author:

http://www.google.com/support/forum/p/Google+Docs/user?userid=11377705067774228009&hl=en

All is left is that I cite my own forum contribution:

Oh dear! I have just spent a whole week syncing all my 47,000 files to Google Docs as a backup solution. When I was confused by the strange behaviour of the “Collections” I found this forum. I’m now convinced that Google online storage is completely useless for this purpose, since it seems to have a weird internal machinery. It is not under control by the programmers, and there seems to be no documentation; it’s behaviour can only be explored experimentally!

If I think that the same API as used by the UI is also used by the API used in my sync program Syncdocs, and that I had two-way synchronization running for a few days, then I can be happy that the data did not disappear from my PC also! Actually, one folder which I produced during the sync disappeared; fortunately, Windows Desktop Search found the contents in the root directory.

This was my last experience with Google Docs, I will use Drop Box.

I will now check my data against my backup on SD card (done before all the migration) and see what is lost. Actually, the “backup” deleted some 40 Files from my PC; they were in two folders which I had synchronized initially to test the procedures; at that time, it looked nice. I think also the Syncdocs developers are not aware of the fact that the last-modification-date as only measure for file changes does not allow  reliable file synchronization (see e.g. the discussion of “Alway Sync” or “vuBrief”).

I go marching’ on…

after reading this page: http://www.mydrive.com/2011/03/01/introduction-to-cloud-storage-providers/

… but can find no way through the swamp

The only solution fulfilling all my needs would be Box.Net, however, for $59.99 per month. Without full text search, DropBox would still cost $10 per month. I think the time is not yet there for my web migration.

2011-07-18
Post-mortem observations:
a message to the dataviz support (Documents To Go):

Since I uploaded some 15 GB in 47,000 files to may Google Docs account, DTG just shows “Please wait…” for several minutes, and finally “Connect failed” when I try to access through google docs.

In the testing phase with much less files, everything worked well; while the upload was taking place, the “please wait…” duration increased, but finally succeeded.

Is there a solution?

Documents To Go is the only reasonable tool to view/edit MS Word and Excel files on Android phones (actually, it is fairly good).

Google Drive and DropBox

2012-05-11

It would be useful if bloggers tell how they use their online storage. To dump music and videos (large, but few files of little importance), to backup their daily work (mainly one way, usually several 10,000 files of which only a few change, data loss unacceptable), to synchronize between several computers (two-way synchronization is much harder two implement reliably), to share files with others (few files, maybe large, of little value), or for distributed collaboration on the same documents with others (requires reliable revisioning and revision merging, specialized file formats, full text search, and copes usually with very important files).

Whoever divides GB/$ is heading in the completely wrong direction. I’m not planning to be loyal to any provider at all (I’m testing a free DropBox account since half a year, have a paid 15 GB GoogleDocs account since about 8 month, and a 50 GB payed Dropbox account since two days).

You should divide value/$. And for me, the 12 GB of data I have accumulated since the last 20 years of work as a scientist, represent a significant value. Similarly, the 7 GB of private data do (50,000 files in total).

I’ve uploaded about 13 GB of data 8 month ago to Google Docs, and it just was too unstable for reliable work. When Google Drive arrived, I again gave it a try, also because my Docs account was migrated to Drive with all the (outdated) data. I tried to delete this old data by deleting all folders in the web interface, and it ended up in the trash bin. I purged the trash bin, and the files show up still in the “All items” collection, however, as a flat list without any directory structure. The web interface allows to delete 30 or so of them at once. To delete all of them I would have to employ a holiday student.

At this occasion, I decided to buy 50 GB from DropBox, where the free account had worked flawless for several month for me, containing my most active subfolders.

DropBox has a significant technological advantage (for specialists: it is based on hash sums) over all competitors. If you have important data, store it there. I think with this bussiness model DropBox can live for some time, they may be even happy if the free “customers” (remember: only who pays is a customer; otherwise one most likely is trade goods) switch to the competitors.

I will also try out the large, free SkyDrive; I have some 20 GB of Photos, I have never sorted out nor looked at. I will dump them there. The furter future is unknown. Still, I’m optimistic DropBox will decrease the GB/$ by increasing my GB, as my data grows.

To my experience (and the technical specifications, where available), only dropbox fullfills the requirements for daily backup, synchronization so far. The technical reason is probably that it is based on hash sums of the complete files and not on metadata like modification data, etc; additionally, their software, on any platform, is in a much more mature state than that of the others: apparently much less bugs compromizing reliability, and many details, e.g. syncing back a file after editing on a mobile phone, work much smoother.

For the most demanding task, but for expensive bussines solutions, distributed collaboration, only Google Docs (=Google Drive) has really some potential. However, because of the still alpha-like stability I will wait before I really will use it.
I do not claim to have tested all providers.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s


%d bloggers like this: