So, the xfs filesystem really prefers to use 64-bit inode numbers on large filesystems; btrfs would like 64-bit inodes and ext4 is considering it as well. Is the world ready for this? not yet. Running a tool written by Greg Banks over the binary directories in all x86 Fedora Rawhide packages, it comes up with:
4070 29.1% are scripts (shell, perl, whatever) 6598 47.2% don't use any stat() family calls at all 1829 13.1% use 32-bit stat() family interfaces only 1312 9.4% use 64-bit stat64() family interfaces only 180 1.3% use both 32-bit and 64-bit stat() family interfaces
I need to correlate this with packages that actually use the st_ino stat field, but a spot-check so far includes postfix and sendmail. :( But, we have the technology, we can fix it!
On edit: ugh, I forgot to scan the libraries. It’s worse than I thought.
Nearly 4 years later and nothing has changed, if anything it’s worse. This is on RHEL 4, 5, and 6. Legacy applications have been humming along and then all of a sudden stop working when we switch out our backend storage. We have traced it down to the broken 32 bit ABI. I have a proof of concept “fix” that uses LD_PRELOAD to replace the stat and readdir commands so that they use the 64 bit versions and print warnings on overflow rather than failing.
While it’s easy to blame Linux, the same behavior can be seen on other unix systems including Solaris. There appears to be no kernel level compatibility shim for 32bit applications to deal with 64bit inodes ( or devices, links, … ).
Yep, there has been no real work on this upstream.
XFS is about to enable 64-bit inodes by default on large (> 1T) filesystems, so that may force the issue if 32-bit apps get used in conjunction…
If by “force” you mean the kernel developers blaming the sysadmins for not recompiling their “broken” applications then I full agree with you. While it’s true that recompilation will solve this issue there are still RHEL5 and 6 applications and libraries that still not compiled for LFS/64bit inodes and that doesn’t even count all the legacy applications where recompilation may not work so well.
I still think it’s insane that such an opaque variable should cause basic kernel syscalls to error out. I can’t find any real reason most applications even need the true inode or device numbers rather than just stable stand-ins. Heck, most calls to stat do not even act on the inode value.
When I have more than an evening hack, I plan on posting the shim up to github or other repository.
Well, I’m not about blame. ;) But somebody has to move first, or there is no motivation to fix things.
I agree that stat32() failing due to a field most likely nobody cares about is a crummy situation.
NFS, IIRC, has some hacks where they try to hash 64-bit space into 32. That might be something to consider kernel-side if lots of apps freak out at 64-bit inodes, and can’t be recompiled.
If you do put your shim up somewhere, please drop a link here as well!
Not 4-years-later like post to first comment but, any chance you posted that shim or POC somewhere? I see a couple other people have LD_PRELOAD libs but wanted to see your version too if possible.
Pingback: Does the world need 32 bit inodes? « Fmehphisto
Ten years latter, still need 32bit
In places, yes. Meanwhile ext4 is learning how to do 64-bit inodes now. :)