They claim a limit of around 300,000 files - this user apparently has 700,000 files.
The Google sync client doesn't have a hard limit per se, but yes, the performance does similarly decline after a certain point - it depends on a number of factors, but it's possibly at 700,000 that you would see issues. It should be able to comfortably handle 300,000 though.
In fact, most Desktop sync clients experience similar things.
I do know this is something we're aware of and are actively improving. In fact, we have a new Backup and Sync client that we announced recently and should be going GA shortly.
There's no details I have to contact this specific customer to see if I can intercede, but if anybody knows, please comment here.
(Disclaimer: I work for Google, but any comments above are my own).
That seems like a ridiculously small number of files at which to start encountering problems. I would expect Google to make things more scalable. Yes I realize you have to work with what the OS gives you, but still, you can build your own stuff on top of that, no?
As a result it has at most 3.5 gigs of memory for the entire system, and e.g. on windows this will limit a single process to 2 gigs. If the sync client keeps some kind of in-memory representation of your files (a very reasonable thing to do for the vast majority of clients), and if you have 700,000 files, and assuming that folder references magically take up no space and that the sync client uses no other memory whatsoever, that's less than 3k per file representation before you crash.
If that is indeed contributing to the problem, it's most likely just a case of doing what's optimal for normal workloads, and simply not having engineered a backup solution for cases which fall so far outside of your expected usage that it didn't seem worth considering at the onset. This kind of change would also be very non-trivial to engineer, likely requiring a rather large amount of refactoring.
Well, if only we had high-performance data-retrieval systems which were accustomed to limited memory on its 32-bit host. A system like that might be able to stream indexes from a HDD to a couple processes in memory which read it as it comes through. Or even just a successful full-table scan.
We could call it "Postgresql". Or even "Sqlite".
"Well, because it is submersed in a marine environment, I've always called it the Going-Under-The-Water-Safely Device."
-- Leonard da Quirm
RDMBS are the "accesses-the-data-safely" device. This is a failure of architecture.
> https://www.dropbox.com/help/space/file-storage-limit
They claim a limit of around 300,000 files - this user apparently has 700,000 files.
The Google sync client doesn't have a hard limit per se, but yes, the performance does similarly decline after a certain point - it depends on a number of factors, but it's possibly at 700,000 that you would see issues. It should be able to comfortably handle 300,000 though.
In fact, most Desktop sync clients experience similar things.
I do know this is something we're aware of and are actively improving. In fact, we have a new Backup and Sync client that we announced recently and should be going GA shortly.
There's no details I have to contact this specific customer to see if I can intercede, but if anybody knows, please comment here.
(Disclaimer: I work for Google, but any comments above are my own).