There are four times in my life I have suffered regrettable data loss incidents.
The first was when the telephone pole outside our house was struck directly by lightning. Not only was it the loudest thing I have ever heard, the current surged through the telephone line, into the internal fax modem, and fries everything within its vicinity. I was 10. I did have backuos, but only only floppy and they didn't cover everything.
The second was storing data in OneDrive - a change to their terms surrounding "lifetime" unlikely noted storage, combined with a client that was unusably slow to download and a deadline for data retrieval meant that I lost most of my files.
The third was SD card failure in digital camera on holiday, the controller chip died catastrophically, leaving the card completely unrecognised. It was a brand new Sony 128GB card, manufactured by Toshiba, and it seemed to be a common issue. I now shoot to two cards simultaneously.
And the fourth time was ... Performing a backup. An errant script deleted the source content, but I'd also deleted the existing backup to free up space for the new backup. I've been weary of using rewritable media for some time now as a consequence, but I think backups themselves are high risk activities.
>I did have backuos, but only only floppy and they didn't cover everything.
This made me remember the time during the 90s when I lost all my BASIC programs due to a hard disk crash, and seeing me sitting sad, my mother handed me a bunch of floppies and asked me to see if they contain something useful.
And turned out that it had all the programs! I took the backups and forgot about that. Apparently that is one of the hard things with backups. You need to track them...
I would say you didn't get quite the right lesson from the fourth one. Rewritable devices are fine, but have (at least) one backup completely unplugged while you update another backup.
> a change to their terms surrounding "lifetime" unlikely noted storage
I had a similar experience with a Chinese cloud storage provider. They didn't even give a convenient way to export the data. And the client throttled download to like 100KB/s. I luckily was able to parallel download by running the client on several VMs...
> An errant script deleted the source content, but I'd also deleted the existing backup to free up space for the new backup.
Sounds like the backup workflow is fundamentally flawed... But I also have the fear that the backup program with root privileges can go off the rails... So I keep my backup job as simple as just running restic with some systemd restrictions.
Man i absolutely hate how windows insists that my various personal folders need to be OneDrive and then complains that there's not enough space. I've tried to just disable/remove onedrive but if it's possible it's certainly not easy. I'm pretty sure there's a bunch of stuff I've lost due to the same thing the other guy described, limit was reduced and files just deleted.
Either give me unlimited space or get out of my file system. They could have just made it a separate thing and let me put stuff in there if I wanted to, instead of hijacking my file system and just messing it up.
I have the paid OD, and think I know what I'm doing, and still got burned on occasions, luckily with little consequence due to other mechanisms in place. But when I really didn't want to deal with OD's crap, I went all nuclear on it. Uninstall. When MS insisted to reinstall it, I replaced the executable and went for file system permissions. I twisted them so hard that even MS couldn't (or at least didn't try to) overwrite my fake OD executable.
I stopped running Windows wherever I can not because the tools or OS are that bad but because a machine should come with sane defaults but should respect my wishes if I want to go against those defaults. When it starts to override me, it's not my machine anymore and whoever "owns" it (MS in this case) can shove it.
That’s not true in the slightest. None of Canon’s 50D and up line, 7D line, 5D line, and rebel line did anywhere near that long ago. I did not shoot on Nikons back then but I imagine most of their fleet didn’t have dual slot recording either. You’d have to get high end pro gear until 10-12 years ago for it.
The 5D3 was first, so that was like 12 or 13 years ago now. It rolled out after that over a few years across their fleet.
I also have the same fuji. Digital cameras are still in the stone ages. I think Tony Northrup covered it well. Nothing has changed since digital cameras came to market, the workflow is still the same, copy files from a card to PC. They aren't connected to the internet, they don't have automatic syncing to cloud for backups or sharing, fuji doesn't even have a gps chip built in to store the location (only works if connected to a smartphone), they can't be locked via fingerprint scanner. It's all tech from 25 years ago.
In some senses it's good. That feature list just says "use a smartphone then". No thank you. It's everything that a camera should not be.
GPS is useful, but not wifi. I don't need my camera becoming another IOT always online telemetry device. They're one of the few electronic devices built to last. Imagine needing fingerprint unlock on a camera, and for what? Prevent others from viewing your photos? Changing your settings? Also copy from card to pc is the best possible workflow. Imagine cameras like smartphone with no expandable storage.
Stop giving these people ideas! I am now very afraid they will listen to this guy since he actually has a follower base.
Unfortunately, your comment reminds me that car manufactures aren't Google or Apple either, and still it seems that modern, high-end cars have a tendency of collecting more than what's needed for the car to remain in good shape or be diagnosed. I guess we became so defeatist when more and more companies started realizing they could make extra profit by collecting data from their customers.
> Also copy from card to pc is the best possible workflow
So turn off the camera, open the memory card slot cover which is flimsy 1mm prong, eject the card, insert it in the reader (if you have SD reader) or fiddle the microSD first from the convertor and then insert, copy, (insert microSD to SD convertor), open up slot cover, insert the card.
Surely beats just connecting the camera to the WiFi (which it does have anyway) and downloading through SMB. Suuure.
S9900 from 15 years ago had a built-in GPS receiver (and absolutely unneeded and never updated POI system). P1100 needs a smartphone for GPS for whatever reason - and app doesn't work 5 times from 5. But the latter is on me, I knew what I would get when I bought Nikon.
Cameras had gps for a moment but they all dropped it because it sucks. GPS takes a long time to get a fix so unless you want to turn the camera on and sit for 5 minutes before taking the photo, it doesn’t work. Meanwhile an app on your phone can ping the current location to the camera for free. Your phone always has a location lock and it’s super power efficient at getting it.
And the rest of it is essentially features for a market that doesn’t exist. People buying cameras today don’t want to send straight to Facebook. They want to sort through raws, edit in Lightroom and then post from a laptop. The market for the features you want all moves to phones long ago.
Sony also has a bunch of the features you are asking for. It can connect to the internet and live sync files to an ftp server. It can even upload proxy videos so your editors can start work before you get back to copy the master copy over. Cameras are sold to prosumers and actual professionals now so the featureset reflects it.
> Your phone always has a location lock and it’s super power efficient at getting it.
So why can't cameras do it? They are the same form factor, could be a feature to enable. My smart garmin instinct can track gps for a day, and it's a tiny sportswatch.
Because they don’t have internet access to download the gps almanac or ping the apple/google servers to ask where the wifi ap they saw is located. And they aren’t always on so they have to get the gps fix from scratch rather than maintaining it which is faster.
Sure you could put a mobile modem and internet connectivity on a camera, but it adds price, reduces battery life, and doesn’t really have any advantages over a phone companion app
> People buying cameras today don’t want to send straight to Facebook.
Pretty sure that's the main point of Fuji camera, the output and built it filter is decent enough that you can upload to Social network without any editing.
I'd think even the JPEG shooters still prefer to sort through the results on a computer and don't need to fire off instagram posts direct from the camera. At any rate, most cameras these days have a companion app that let you grab the photos from your phone without pulling out an SD card.
Bluetooth is slow. The transport is 3Mbps at max, if you subtract all the headers from different layers, you may have like 1.5-2Mbps practical transfer speed.
And harm a lot too, you phone can't deep sleep since it has to wake up periodically to process data from bluetooth stack. Combined with slow transfer speed, that gonna eat your battery like nothing.
Playing audio over bluetooth also wakes things periodically, but it uses almost no power. With a similarly competent design photo transfer should be very battery efficient.
No, most(if not all) bluetooth controller has audio offload, basically the cpu does not involve(or very little) when transfering data from storage to radio.
Don't you mean yes bluetooth controllers have audio offload? "similarly competent design" means you do the exact same thing in the other direction. Instead of rarely waking up the CPU to coordinate sending, you rarely wake it up to coordinate receiving.
You would often not have wifi available, and you wouldn't want to have to type in wifi passwords to your camera all the time. But your phone will almost always be nearby and persistent bluetooth is easy.
Problem is they have to disconnect from the current wifi ap to connect to the cameras ap which is inconvenient and means there can’t be a constant connection.
Wifi NAN could be used to solve this problem but I’ve not seen anything using it. Apple only just added support last year though.
>I've been weary of using rewritable media for some time now as a consequence
I briefly considered using bluray disks as a backup for my photos and other critical docs. But getting a decent bluray burner seems not so easy these days with most production winding down. Next best thing looks like the "object lock" feature on object store services that prevents deleting objects for a certain time.
I pretty much instantly wrote off physical media because I know I won't reliably move it to a separate location. If I'm going to have backups, I want ones that are not within fire/flood range of my computer.
In terms of preventing "oops" moments, I'm mainly relying on software (restic) for that, where I trust that (A) backups always append data rather than replacing and (B) it's logic works for marking which data to purge based on rules is accurate. [0]
You want to make sure the append only is enforced by the hosting provider rather than duplicity. Your backups should be resistant to having an attacker on the system who can run arbitrary commands with duplicity.
Right, defense against internal "oops" accidents looks rather different than defense against purposeful sabotage.
While the local backup service uses an API-key with limited privileges, I need to go back to the bucket/storage-zone settings and see what I can enable, how it would affect total storage, and whether I (with additional privileges) can reliably grab the correct mix of old-blobs to do a restore.
I kinda overdid it with a tape library (40 tapes capacity, LTO-9), but I'm using it for my company's AI model backups. _That_ setup was about $12k.
I had a smaller setup before, with a simple external LTO-5 drive. Used drives are now are about $300, and you can probably find them cheaper. And LTO-5 is the minimum realistic version, it's the first one that supports LTFS and it has reasonable tape capacity.
This is for backups. The threat here is that random ware uses your backup script to delete/encrypt the backups too. With object lock the api key the script has can only add data.
The probability that the object store goes out of business at the exact time your own copy dies is insignificant.
That only matters if it happens literally at the same time as a failure of your primary storage. If they increase the price you can just close your account and move the backup to another service.
The probability of a hard drive failure or ransomware at the same time as backblaze or aws going out of business is pretty much not worth thinking about.
The probability of an attacker using the api key in your backup script to destroy the backups is far more possible.
Amazing question with nobody really answering your question 100%.
The answer is : it's hard. If you only have a iphone and more photos on your icloud than can fit on your phone, you're kinda screwed if you don't know what you're doing.
AFAIK Apple makes it very difficult to get all your icloud photos bulk downloaded, original quality off icloud.
Google is no better.
It's much easier to start backing up from the beginning but once you're stuck on icloud it's hard.
If you have a Mac with sufficient storage, you can just enable iCloud to store photos locally, then export them. That could miss some data (I think descriptions you set for a photo were not exported the last time I checked), but it is most of the data.
I did not try it, and I am nervous the day I need to - but I think there are some solutions to export photos from the Apple technology stack using open source software [1,2,3].
I remember seeing a longer blogpost re the topic of retroengineering the Apple phot sync story, but I could not find it.
Yup. Only reason I'm on OneDrive is that they don't mess with photos and their metadata. For now, at least.
I guess it's because the core offering is still some version of "a folder that syncs", and the photo gallery is just a view bolted on top.
I steer clear of any solutions scoped to photos only, as it always turns out they either screw you up by reducing quality and throwing away the originals, or trying to lock you in by only offering metadata they keep in a database they will not let you export.
It's not iCloud Photos, in my case it's Google Photos. But I keep a copy of all my photos in a self-hosted instance of Immich [1], which I then back up to a Hetzner Storage Box using Backrest [2].
So I end up with 3 copies of all my photos, on 3 different providers (I mean... 2 different providers plus my homelab disk).
It's mainly just rsync. When the phone was plugged in, the photos were accessible at a path like "/run/user/1000/gvfs/gphoto2:host=Apple_Inc._iPhone_abc123/".
Now I use Time Machine, but I still need to check whether it includes iCloud photos.
While I haven't tested for that myself, the script should keep pretty much everything including image metadata. I just made a small change so that it also keeps extended attributes in case any image-related metadata is there.
The script currently requires changing one or two folder paths at the top (more details in the script).
I have a weird setup lol.
I use a shared photos library with my partner (so that we have all family photos together) and have a MacBook Air that has Photos being download in raw quality and backups then via Time Machine on a Synology NAS.
Plus, Synology photo sync on my iPhone (backing up on my NAS separately from Time Machine) AND Google Photos on my partner’s (syncing on Google Photos).
Yes it’s horrible, just never had time to clean I up properly but I prefer to have them in multiple cloud locations and locally for now than optimise for having a clean setup.
That's tedious. Can't believe nobody mentioned Parachute Backup yet. It had a moment in the sun earlier this year or last year when it first started getting traction.
I have a Mac, so not just an iPhone, so I use Arq there, it materializes the iCloud stuff as necessary for backup. But if all you have is an iPhone or iPad along with an external disk, NAS or whatever, Parachute Backup will do backups to a variety of destinations.
There's plenty of backup services and cloud solutions. The article mentions Borg and Restic, others like Backblaze are more consumer friendly. Which needs do you have that aren't met by any of these tools?
online backup should not be your primary method of backup, you're one billing, identity theft, financial issue, or health issue away from getting your backup nuked
PBS's situation is entirely PBS fault. If you've got 50TB of data to backup it's either worth $300 to $400 a month to you to do it properly with a reputable vendor along with on-site storage, or ... just accept you'll lose it.
It was such a tremendously silly decision I almost wonder if it wasn't a politically motivated decision. Maybe the provider that went under had some relative DEI points versus Backblaze or AWS. Just bizarre.
But I wouldn't extrapolate anything out from their situation.
The first was when the telephone pole outside our house was struck directly by lightning. Not only was it the loudest thing I have ever heard, the current surged through the telephone line, into the internal fax modem, and fries everything within its vicinity. I was 10. I did have backuos, but only only floppy and they didn't cover everything.
The second was storing data in OneDrive - a change to their terms surrounding "lifetime" unlikely noted storage, combined with a client that was unusably slow to download and a deadline for data retrieval meant that I lost most of my files.
The third was SD card failure in digital camera on holiday, the controller chip died catastrophically, leaving the card completely unrecognised. It was a brand new Sony 128GB card, manufactured by Toshiba, and it seemed to be a common issue. I now shoot to two cards simultaneously.
And the fourth time was ... Performing a backup. An errant script deleted the source content, but I'd also deleted the existing backup to free up space for the new backup. I've been weary of using rewritable media for some time now as a consequence, but I think backups themselves are high risk activities.