Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

> So if I do fopen/fwrite/fsync/fclose, that is not enough?

That is my understanding.

> Also, how many levels of parents do you need to fsync?

Only one, at least if you didn't create the parent directory (if you did then you might have to fsync its parent, recursively). The fsync on the parent directory ensures the dir entry for your new file is flushed to disk.



I've never heard about this in my years of programming. I just tried to read through the Win32 documentation, as I've done several times over the years, and it mentions a lot of edge cases but not this that I could see.

Is this some Linux/Unix specific thing? Am I blind?


As far as I know, it's specific to the combination of certain POSIX-ish OS and file systems, like linux/Ext3. I have no clue what BSD does here, or whether ReiserFS is different.

Windows/NTFS is a different world, there are still edge cases that can go wrong but I don't think this particular one is a problem because FAT/NTFS is not inode-based.

I imagine if you looked at the SQLite source code you'd see different edge-case-handling code for different OSes.


NTFS has inodes.

The thing about Windows is that because the file open operation (`CreateFile*()`) by default prevents renames of the file, Windows apps have come to not depend so much on file renaming, which makes one of the biggest filesystem power failure hazards less of an issue on Windows. But not being able to rename files over others as easily as in POSIX really sucks. And this doesn't completely absolve Windows app devs of having to think about power failure recovery! "POSIX semantics" is often short-hand for shortcomings of POSIX that happen to also be there in the WIN32 APIs, such as the lack of a filesystem write barrier, file stat-like system calls that mix different kinds of metadata (which sucks for distributed filesystems protocols), and so on. And yes, you can open files in Windows such that rename-over is allowed, so you still have this problem.


Ya NTFS file naming and long path is a wreck. Unzipping files from mac/linux is a easy way to end up with missing data. Applications quite often break on long file paths especially Microsofts own stuff like powershell.


I am talking about posix semantics yes, I have no idea how things work on windows.


There's the old saying that on Windows*, files have names; in POSIX, names have files. I think that's what makes the difference here.

* technically it's the filesystem as much as the OS that is relevant here.


* a very limited set of names in relation to what's possible on other operating systems.

If your processing files from other systems on NTFS you'll very likely have rename said files in an application and store an index of the names.


Phew. I've primarily used Windows. Not that any of the posix programs I've been exposed to have done the dir sync though.

For cross-platform stuff I've mainly used Boost, which I assumed handled such details.


I'm sure it does not.

Also these things are needed very very rarely (which is why few even know about the issue) and are not good for performance and battery life.


More specifically, these things are for trying to improve the behavior on unclean system shutdown (e.g. power loss) which is inherently chaotic and unless all parts (most critically the disk and its controller) are well behaved you don't have any real guarantees anyway.

Windows also doesn't guarantee that data is written to disk by the time WriteFile/CloseHandle returns, the Windows version of fsync is FlushFileBuffers.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: