Subject: coreutils: chmod, chown,
and chgrp change ctime even when no change was necessary
Date: Tue, 02 Sep 2008 11:30:46 +0200
Package: coreutils
Version: 5.97-5.3
Severity: wishlist
It would be nice if chmod (and chown and chgrp) only made changes to
inodes when necessary so as not to change the ctime of files.
I like to use the integrit file integrity-checking program which scans
for changes to mtime and ctime. I also occasionally set up "fix-perms"
cronjobs that scan directories recursively and fix any mis-configured of
permissions/ownerships. The problem is that "chmod -R" and company
change the ctimes of all files, not just those that need to be fixed.
This causes excessively long (and useless) file integrity reports.
I could use cfengine to do the same job, but chmod is sometimes much
more convenient.
I have tested Busybox chmod and it suffers from the same behavior.
If unconditionally resetting the ctime of inodes is normal POSIX
behavior for chmod, then this function should be implemented as a option
flag. If not, can the authors make "change-only-if-necessary" the
default behavior of chmod and company?
Thanks.
-- System Information:
Debian Release: 4.0
APT prefers oldstable
APT policy: (991, 'oldstable'), (991, 'stable'), (66, 'oldstable'), (33, 'unstable')
Architecture: i386 (i686)
Shell: /bin/sh linked to /bin/dash
Kernel: Linux 2.6.18-6-vserver-686
Locale: LANG=en_US, LC_CTYPE=en_US (charmap=ISO-8859-1)
Versions of packages coreutils depends on:
ii libacl1 2.2.41-1 Access control list shared library
ii libc6 2.3.6.ds1-13etch7 GNU C Library: Shared libraries
ii libselinux1 1.32-3 SELinux shared libraries
coreutils recommends no packages.
-- no debconf information
Subject: Re: Bug#497514: coreutils: chmod, chown,
and chgrp change ctime even when no change was necessary
Date: Tue, 02 Sep 2008 08:19:38 -0400
On Tue, Sep 02, 2008 at 11:30:46AM +0200, you wrote:
>It would be nice if chmod (and chown and chgrp) only made changes to
>inodes when necessary so as not to change the ctime of files.
I'm fairly certain there have also been complaints about these utilities
not resetting the inode even when not necessary. I'd suggest using them
in conjunction with find(1) if you only want to modify files that don't
match a particular setting.
Mike Stone
Subject: Re: Bug#497514: coreutils: chmod, chown, and chgrp change ctime even when no change was necessary
Date: Tue, 2 Sep 2008 15:50:16 +0200
On Tue, Sep 02, 2008 at 08:19:38AM -0400, Michael Stone wrote:
> On Tue, Sep 02, 2008 at 11:30:46AM +0200, you wrote:
> >It would be nice if chmod (and chown and chgrp) only made changes to
> >inodes when necessary so as not to change the ctime of files.
>
> I'm fairly certain there have also been complaints about these utilities
> not resetting the inode even when not necessary.
That is certainly possible, if that is normal POSIX behavior.
> I'd suggest using them in conjunction with find(1) if you only want to
> modify files that don't match a particular setting.
Yup.
But it would be so much more elegant if chmod just Did The Right Thing.
Can I ask you to pass along this wishlist item to upstream and maybe ask
them why ctime must be modified even when the permissions are not? I
would really like to know why.
--
Erik Rossen OpenPGP key: 2935D0B9
[email protected] If you do not know what
http://people.linux-gull.ch/rossen to do, start with RTFM.
Subject: Re: Bug#497514: coreutils: chmod, chown,
and chgrp change ctime even when no change was necessary
Date: Tue, 02 Sep 2008 10:48:30 -0400
On Tue, Sep 02, 2008 at 03:50:16PM +0200, Erik Rossen wrote:
>But it would be so much more elegant if chmod just Did The Right Thing.
It's certainly arguable whether adding another non-standard flag is the
Right Thing when it's fairly trivial to get the result in a more
portable fashion using find(1).
>Can I ask you to pass along this wishlist item to upstream and maybe ask
>them why ctime must be modified even when the permissions are not? I
>would really like to know why.
Upstream monitors the debian bug logs and will presumably chime in if
interested. I vaguely recall that people want these utilities to act
even if not strictly necessary because of possible side effects for
extended attributes.
Mike Stone
Subject: Re: Bug#497514: coreutils: chmod, chown, and chgrp change ctime even when no change was necessary
Date: Tue, 02 Sep 2008 20:06:51 +0200
Erik Rossen <[email protected]> wrote:
> On Tue, Sep 02, 2008 at 08:19:38AM -0400, Michael Stone wrote:
>> On Tue, Sep 02, 2008 at 11:30:46AM +0200, you wrote:
>> >It would be nice if chmod (and chown and chgrp) only made changes to
>> >inodes when necessary so as not to change the ctime of files.
>>
>> I'm fairly certain there have also been complaints about these utilities
>> not resetting the inode even when not necessary.
>
> That is certainly possible, if that is normal POSIX behavior.
>
>> I'd suggest using them in conjunction with find(1) if you only want to
>> modify files that don't match a particular setting.
>
> Yup.
>
> But it would be so much more elegant if chmod just Did The Right Thing.
>
> Can I ask you to pass along this wishlist item to upstream and maybe ask
> them why ctime must be modified even when the permissions are not? I
> would really like to know why.
Good idea.
At least for chmod, it is not only possible, but the optimization
would be essentially free, since chmod already has the required stat data.
AFAICS POSIX
(http://www.opengroup.org/onlinepubs/007908775/xsh/chmod.html)
allows the behavior you propose. That same feature was even
requested a few years ago, but no one implemented it.
A patch would be welcome.
If you're not interested, let me know in any case and
I'll add it to the TODO list.
If you are interested in writing the patch,
please follow the guidelines outlined here:
http://git.sv.gnu.org/gitweb/?p=coreutils.git;a=blob;f=HACKING;hb=HEAD
For chgrp (probably chown, too, at least in some cases), it's not
as obvious, since the current implementation does not stat files
before changing permissions. So, to do what you want would involve
adding a stat call per file to get owner/group in order to avoid(maybe)
the offending ownership-changing syscall. That would not be an
improvement.
Subject: Re: Bug#497514: coreutils: chmod, chown, and chgrp change ctime even when no change was necessary
Date: Tue, 2 Sep 2008 20:56:43 +0200
On Tue, Sep 02, 2008 at 08:06:51PM +0200, Jim Meyering wrote:
> Good idea.
> At least for chmod, it is not only possible, but the optimization
> would be essentially free, since chmod already has the required stat data.
Yeah, I thought it was a good idea too.
> AFAICS POSIX
> (http://www.opengroup.org/onlinepubs/007908775/xsh/chmod.html)
> allows the behavior you propose. That same feature was even
> requested a few years ago, but no one implemented it.
Thanks for the lead - I had no idea where to go looking for POSIX specs.
But you got the wrong URL. The URL above refers to chmod() the
function. What we needed was chmod the utility. Here it is:
http://www.opengroup.org/onlinepubs/007908799/xcu/chmod.html
And, if one wants to be REALLY pedantic, it looks like the file node is
supposed to be changed each time. For example, here is an extract:
+
If perm is not specified, the + operation will not change
the file mode bits. If who is not specified, the file mode
bits represented by perm for the owner, group and other
permissions, except for those with corresponding bits in the
file mode creation mask of the invoking process, will be
set. Otherwise, the file mode bits represented by the
specified who and perm values will be set.
Here it says that the bits will be set. Unfortunately, they do not say
something like "set unless already set". If the feature is to added, it
will be a non-standard option.
> A patch would be welcome.
I totally agree.
> If you're not interested, let me know in any case and
> I'll add it to the TODO list.
I think I will let you add it to the TODO list. :-)
> For chgrp (probably chown, too, at least in some cases), it's not
> as obvious, since the current implementation does not stat files
> before changing permissions. So, to do what you want would involve
> adding a stat call per file to get owner/group in order to avoid(maybe)
> the offending ownership-changing syscall. That would not be an
> improvement.
As far as speed is concerned, you are right that an extra stat() would
usually not improve matters. (Would it be an enormous penalty? I doubt
it.) But users of file integrity checking systems like myself will be
grateful for the change of behavior.
--
Erik Rossen OpenPGP key: 2935D0B9
[email protected] If you do not know what
http://people.linux-gull.ch/rossen to do, start with RTFM.
Subject: Re: Bug#497514: coreutils: chmod, chown, and chgrp change ctime even when no change was necessary
Date: Tue, 2 Sep 2008 14:30:20 -0600
Erik Rossen wrote:
> Jim Meyering wrote:
> > For chgrp (probably chown, too, at least in some cases), it's not
> > as obvious, since the current implementation does not stat files
> > before changing permissions. So, to do what you want would involve
> > adding a stat call per file to get owner/group in order to avoid(maybe)
> > the offending ownership-changing syscall. That would not be an
> > improvement.
>
> As far as speed is concerned, you are right that an extra stat() would
> usually not improve matters. (Would it be an enormous penalty? I doubt
> it.) But users of file integrity checking systems like myself will be
> grateful for the change of behavior.
You would be surprised how much performance can change when adding
extra stat calls. Many filesystems such as networked filesystems are
extremely latency bound and those kinds of changes have a strong
negative impact upon them.
Bob
Subject: Re: Bug#497514: coreutils: chmod, chown, and chgrp change ctime even when no change was necessary
Date: Tue, 2 Sep 2008 22:57:28 +0200
On Tue, Sep 02, 2008 at 02:30:20PM -0600, Bob Proulx wrote:
> Erik Rossen wrote:
> > As far as speed is concerned, you are right that an extra stat() would
> > usually not improve matters. (Would it be an enormous penalty? I doubt
> > it.) But users of file integrity checking systems like myself will be
> > grateful for the change of behavior.
>
> You would be surprised how much performance can change when adding
> extra stat calls. Many filesystems such as networked filesystems are
> extremely latency bound and those kinds of changes have a strong
> negative impact upon them.
OK. But I still wonder how expensive are the extra stat() calls that
would be spent compared to the extra chown() calls that could be saved
by checking first to see if a chown() was even necessary?
--
Erik Rossen OpenPGP key: 2935D0B9
[email protected] If you do not know what
http://people.linux-gull.ch/rossen to do, start with RTFM.
Subject: Re: Bug#497514: coreutils: chmod, chown, and chgrp change ctime even when no change was necessary
Date: Wed, 3 Sep 2008 16:05:46 +0200
I asked the people on the mailing list of our local LUG to test if chmod
changes ctime on non-GNU systems even when there is no difference beteen
the mode before and after the chmod. From these few data, the trend
seems to be that ctime gets changed.
Here are the results thus far (identified by "uname -a"):
systems that change ctime:
--------------------------
AIX XXXXXXXX 3 4 000650834C00
SunOS XXXXXXXX 5.8 Generic_108528-29 sun4u sparc SUNW,UltraSPARC-IIi-cEngine
OpenBSD XXXXXXXX 4.1 GENERIC#187 i386
systems that do not change ctime:
---------------------------------
FreeBSD XXXXXXXX 7.0-RELEASE FreeBSD 7.0-RELEASE #0: Sun Feb 24 19:59:52 UTC 2008 [email protected]:/usr/obj/usr/src/sys/GENERIC i386
--
Erik Rossen OpenPGP key: 2935D0B9
[email protected] If you do not know what
http://people.linux-gull.ch/rossen to do, start with RTFM.
Hi,
*IF* POSIX or general use mandates that chmod(2) should not change
ctime if no change really occurs, this should be fixed in the
Linux kernel itself.
Fixing it in user space in the chmod(1) utility is wrong, because it
could introduce user-space race conditions.
For example:
timestamp 1: perm is 0500
timestamp 2: perm is 0700
timestamp 3: perm is 0500
we would expect c_time to be timestamp 3 at the end of this flow.
Now imagine that you have two processes. Process one wants to set
perm to 0500, process 2 to 0700. Imagine it works as follows:
process 1 process 2
stats file, gets perm 500
stats files, get perm 0500
set permissions to 0700, ctime changes
set perm, no real change,
so don't do anything
WRONG! permission is 0700!
(one could argue that the overall result is undefined if the two
processes are scheduled in parallel anyway; however there seem to be
something wrong in the user-space semantic here anyway)
(in general, any "I check manually, then I set" method is wrong,
unless it is done atomically, which is not the case for user space
and can only be guaranteed with kernel locks with currently used
Linux kernels)
Marc SCHAEFER <[email protected]> wrote:
> *IF* POSIX or general use mandates that chmod(2) should not change
> ctime if no change really occurs,
Neither mandates that.
Subject: Re: Bug#497514: coreutils: chmod, chown, and chgrp change ctime even when no change was necessary
Date: Thu, 11 Sep 2008 10:56:34 -0700
Erik Rossen <[email protected]> writes:
> And, if one wants to be REALLY pedantic, it looks like the file node is
> supposed to be changed each time. For example, here is an extract:
As I read the spec, chown and chgrp are explicitly required to make
the equivalent of a chown() call, which in turn is required to change
the ctime. However, chmod is not required to make the equivalent of a
chmod() call, and there is no requirement in the 'chmod' spec that it
change the ctime. So POSIX allows the optimization for the 'chmod'
command, but not for the 'chown' and 'chgrp' commands.
Subject: Re: Bug#497514: coreutils: chmod, chown, and chgrp change ctime even
when no change was necessary
Date: Thu, 11 Sep 2008 21:20:21 -0600
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
According to Paul Eggert on 9/11/2008 11:56 AM:
>
> As I read the spec, chown and chgrp are explicitly required to make
> the equivalent of a chown() call, which in turn is required to change
> the ctime. However, chmod is not required to make the equivalent of a
> chmod() call, and there is no requirement in the 'chmod' spec that it
> change the ctime. So POSIX allows the optimization for the 'chmod'
> command, but not for the 'chown' and 'chgrp' commands.
On the other hand, the POSIX spec for chmod(1) mentions that it is
implementation defined on how it affects alternate access control. I have
come to expect on many systems that support ACLs, that 'chmod 755' on a
file with rwxr-xr-x+ permissions will remove the ACLs. In other words,
optimizing away the chmod(2) call because it compares equal to the stat
information would break this side-effect of clearing ACLs.
- --
Don't work too hard, make some time for fun as well!
Eric Blake [email protected]
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Cygwin)
Comment: Public key at home.comcast.net/~ericblake/eblake.gpg
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
iEYEARECAAYFAkjJ3/UACgkQ84KuGfSFAYAjhACdFl0013RJgN8wk9ay0tO0Icts
tiEAoKH2Sp5qaStFBGRo61omIOPCUMGO
=zlNv
-----END PGP SIGNATURE-----
Subject: Re: Bug#497514: coreutils: chmod, chown, and chgrp change ctime even when no change was necessary
Date: Fri, 12 Sep 2008 21:51:04 +0200
Eric Blake <[email protected]> wrote:
> According to Paul Eggert on 9/11/2008 11:56 AM:
>>
>> As I read the spec, chown and chgrp are explicitly required to make
>> the equivalent of a chown() call, which in turn is required to change
>> the ctime. However, chmod is not required to make the equivalent of a
>> chmod() call, and there is no requirement in the 'chmod' spec that it
>> change the ctime. So POSIX allows the optimization for the 'chmod'
>> command, but not for the 'chown' and 'chgrp' commands.
>
> On the other hand, the POSIX spec for chmod(1) mentions that it is
> implementation defined on how it affects alternate access control. I have
> come to expect on many systems that support ACLs, that 'chmod 755' on a
> file with rwxr-xr-x+ permissions will remove the ACLs. In other words,
> optimizing away the chmod(2) call because it compares equal to the stat
> information would break this side-effect of clearing ACLs.
That sounds like a good reason to retain the behavior you've come to
value, even if it's not guaranteed or portable, but only via a new
option. Then we can still change the default to be more efficient.
Subject: Re: Bug#497514: coreutils: chmod, chown,
and chgrp change ctime even when no change was necessary
Date: Fri, 12 Sep 2008 16:05:09 -0400
On Fri, Sep 12, 2008 at 09:51:04PM +0200, Jim Meyering wrote:
>That sounds like a good reason to retain the behavior you've come to
>value, even if it's not guaranteed or portable, but only via a new
>option. Then we can still change the default to be more efficient.
Why on earth would we want to? Some people obviously like the current
behavior, and depend on the side effects, the desired behavior is easy
to get with existing tools, and adding a new option seems like something
that shouldn't be done without a very good reason. This seems like
optimization for the sake of optimization. (And it would make chmod
inconsistent with chown and chgrp.)
Mike Stone
Subject: Re: Bug#497514: coreutils: chmod, chown, and chgrp change ctime even when no change was necessary
Date: Sat, 13 Sep 2008 09:50:35 +0200
Michael Stone <[email protected]> wrote:
> On Fri, Sep 12, 2008 at 09:51:04PM +0200, Jim Meyering wrote:
>>That sounds like a good reason to retain the behavior you've come to
>>value, even if it's not guaranteed or portable, but only via a new
>>option. Then we can still change the default to be more efficient.
>
> Why on earth would we want to? Some people obviously like the current
> behavior, and depend on the side effects, the desired behavior is easy
> to get with existing tools, and adding a new option seems like
> something that shouldn't be done without a very good reason. This
> seems like optimization for the sake of optimization. (And it would
> make chmod inconsistent with chown and chgrp.)
I don't feel very strongly one way or the other,
since I don't plan to do the work.
Realize that afaik no one is working on this.
I merely added the item to the TODO list, and we
know from experience that that's no guarantee it'll be done.
Subject: Re: Bug#497514: coreutils: chmod, chown, and chgrp change ctime even when no change was necessary
Date: Mon, 15 Sep 2008 11:22:41 +0200
On Tue, Sep 02, 2008 at 10:48:30AM -0400, Michael Stone wrote:
> On Tue, Sep 02, 2008 at 03:50:16PM +0200, Erik Rossen wrote:
> >But it would be so much more elegant if chmod just Did The Right Thing.
>
> It's certainly arguable whether adding another non-standard flag is the
> Right Thing when it's fairly trivial to get the result in a more
> portable fashion using find(1).
It seems that the FreeBSD people think that making a change to the
filesystem only when necessary is The Right Thing. But I guess that
argument carries no weight in GNU. ;-)
> >Can I ask you to pass along this wishlist item to upstream and maybe ask
> >them why ctime must be modified even when the permissions are not? I
> >would really like to know why.
>
> Upstream monitors the debian bug logs and will presumably chime in if
> interested. I vaguely recall that people want these utilities to act
> even if not strictly necessary because of possible side effects for
> extended attributes.
>
> Mike Stone
I took your advice (paraphrased: "try another tool") and I now have
three options for fixing file permissions without disturbing the file
system too much:
1. find + chmod : classic but ugly
2. cfengine : Does The Right Thing, but difficult to do as a one-liner
unless you are already using cfengine for other stuff
3. setfacl -R : nice. Does The Right Thing as a fairly simple one-liner
setfacl is kind of interesting. Even if the filesystem is mounted
without extended ACLs enabled, it works for setting the POSIX ACLs and
it Does The Right Thing. For example, I can do
setfacl -R -m u::rwX,g::rwX,o:rX shares/
to fix every file and directory in "shares" to be readable, and writable
only by the owner and group of shares. And if I run the same command
again, stat shows that the ctime has not changed unless it was necessary
to modify the permissions of a file or directory.
I am pretty happy about this solution and I intend to use it in cases
where there is a integrity-checker in place and no cfengine.
But while doing my tests, I noticed something strange about setfacl: if
the filesystem is mounted with extended ACLs enabled (on a ext3 fs), it
is possible to use setfacl to make changes to file permissions and CTIME
IS NEVER MODIFIED EVEN IF PERMISSIONS ARE MODIFIED. I thought that this
was supposed to be impossible, but perhaps this is a known side-effect
("feature") of activating extended ACLs on ext3.
Can anyone else confirm this? I am running etch with an ext3 filesystem
that was created in June 2008. This is my "uname -a":
Linux mango 2.6.18-6-vserver-686 #1 SMP Mon Aug 18 11:11:15 UTC 2008 i686 GNU/Linux
--
Erik Rossen OpenPGP key: 2935D0B9
[email protected] On fonce la tête dans
http://www.rtfm-sarl.ch le guidon et on RTFM.
Subject: Re: Bug#497514: coreutils: chmod, chown, and chgrp change ctime
even when no change was necessary
Date: Tue, 13 Oct 2020 12:06:59 +0100
On Mon, 15 Sep 2008 11:22:41 +0200 Erik Rossen <[email protected]>
wrote:
> On Tue, Sep 02, 2008 at 10:48:30AM -0400, Michael Stone wrote:
> > On Tue, Sep 02, 2008 at 03:50:16PM +0200, Erik Rossen wrote:
> > >But it would be so much more elegant if chmod just Did The Right
Thing.
> >
> > It's certainly arguable whether adding another non-standard flag is
the
> > Right Thing when it's fairly trivial to get the result in a more
> > portable fashion using find(1).
>
> It seems that the FreeBSD people think that making a change to the
> filesystem only when necessary is The Right Thing. But I guess that
> argument carries no weight in GNU. ;-)
>
> > >Can I ask you to pass along this wishlist item to upstream and
maybe ask
> > >them why ctime must be modified even when the permissions are
not? I
> > >would really like to know why.
> >
> > Upstream monitors the debian bug logs and will presumably chime in
if
> > interested. I vaguely recall that people want these utilities to
act
> > even if not strictly necessary because of possible side effects
for
> > extended attributes.
> >
> > Mike Stone
>
> I took your advice (paraphrased: "try another tool") and I now have
> three options for fixing file permissions without disturbing the file
> system too much:
>
> 1. find + chmod : classic but ugly
>
> 2. cfengine : Does The Right Thing, but difficult to do as a one-
liner
> unless you are already using cfengine for other stuff
>
> 3. setfacl -R : nice. Does The Right Thing as a fairly simple one-
liner
>
> setfacl is kind of interesting. Even if the filesystem is mounted
> without extended ACLs enabled, it works for setting the POSIX ACLs
and
> it Does The Right Thing. For example, I can do
>
> setfacl -R -m u::rwX,g::rwX,o:rX shares/
>
> to fix every file and directory in "shares" to be readable, and
writable
> only by the owner and group of shares. And if I run the same command
> again, stat shows that the ctime has not changed unless it was
necessary
> to modify the permissions of a file or directory.
>
> I am pretty happy about this solution and I intend to use it in cases
> where there is a integrity-checker in place and no cfengine.
>
> But while doing my tests, I noticed something strange about setfacl:
if
> the filesystem is mounted with extended ACLs enabled (on a ext3 fs),
it
> is possible to use setfacl to make changes to file permissions and
CTIME
> IS NEVER MODIFIED EVEN IF PERMISSIONS ARE MODIFIED. I thought that
this
> was supposed to be impossible, but perhaps this is a known side-
effect
> ("feature") of activating extended ACLs on ext3.
>
> Can anyone else confirm this? I am running etch with an ext3
filesystem
> that was created in June 2008. This is my "uname -a":
>
> Linux mango 2.6.18-6-vserver-686 #1 SMP Mon Aug 18 11:11:15 UTC
2008 i686 GNU/Linux
>
I have another use-case for only attempting to make a change with chmod
if it is really necessary.
In a Debian postinst script (for VDR in this particular case) the
ownership of a directory (/var/lib/video) is changed - however I have
moved /var/lib/video to be a symlink to an NFS server. The permissions
are already set correctly, but as root can not (in this case) change an
NFS mounted file system, the chmod fails, even though it wanted to
change the owner to be the same as it already was.
A switch for chmod to say --only-if-needed or some such would avoid
this being an issue
John
Acknowledgement sent
to "Michael Stone" <[email protected]>:
Extra info received and forwarded to list. Copy sent to Michael Stone <[email protected]>.
(Tue, 15 Mar 2022 17:48:02 GMT) (full text, mbox, link).
Acknowledgement sent
to "Michael Stone" <[email protected]>:
Extra info received and forwarded to list. Copy sent to Michael Stone <[email protected]>.
(Mon, 16 May 2022 15:33:03 GMT) (full text, mbox, link).
Acknowledgement sent
to "Michael Stone" <[email protected]>:
Extra info received and forwarded to list. Copy sent to Michael Stone <[email protected]>.
(Tue, 21 Jun 2022 14:27:02 GMT) (full text, mbox, link).
Debbugs is free software and licensed under the terms of the GNU General
Public License version 2. The current version can be obtained
from https://bugs.debian.org/debbugs-source/.