Subject: coreutils: chmod, chown, and chgrp change ctime even when no change was necessary

From: Erik Rossen <[email-masked]>
Date: Tue, 02 Sep 2008 11:30:46 +0200

Package: coreutils
Version: 5.97-5.3
Severity: wishlist

It would be nice if chmod (and chown and chgrp) only made changes to
inodes when necessary so as not to change the ctime of files.

I like to use the integrit file integrity-checking program which scans
for changes to mtime and ctime.  I also occasionally set up "fix-perms"
cronjobs that scan directories recursively and fix any mis-configured of
permissions/ownerships.  The problem is that "chmod -R" and company
change the ctimes of all files, not just those that need to be fixed.
This causes excessively long (and useless) file integrity reports.

I could use cfengine to do the same job, but chmod is sometimes much
more convenient.

I have tested Busybox chmod and it suffers from the same behavior.

If unconditionally resetting the ctime of inodes is normal POSIX
behavior for chmod, then this function should be implemented as a option
flag.  If not, can the authors make "change-only-if-necessary" the
default behavior of chmod and company?

Thanks.

-- System Information:
Debian Release: 4.0
  APT prefers oldstable
  APT policy: (991, 'oldstable'), (991, 'stable'), (66, 'oldstable'), (33, 'unstable')
Architecture: i386 (i686)
Shell:  /bin/sh linked to /bin/dash
Kernel: Linux 2.6.18-6-vserver-686
Locale: LANG=en_US, LC_CTYPE=en_US (charmap=ISO-8859-1)

Versions of packages coreutils depends on:
ii  libacl1                2.2.41-1          Access control list shared library
ii  libc6                  2.3.6.ds1-13etch7 GNU C Library: Shared libraries
ii  libselinux1            1.32-3            SELinux shared libraries

coreutils recommends no packages.

-- no debconf information

---

From: Michael Stone <[email-masked]>
Date: Tue, 02 Sep 2008 08:19:38 -0400

>inodes when necessary so as not to change the ctime of files.

I'm fairly certain there have also been complaints about these utilities 
not resetting the inode even when not necessary. I'd suggest using them 
in conjunction with find(1) if you only want to modify files that don't 
match a particular setting.

Mike Stone

---

From: Erik Rossen <[email-masked]>
Date: Tue, 2 Sep 2008 15:50:16 +0200

On Tue, Sep 02, 2008 at 08:19:38AM -0400, Michael Stone wrote:
> not resetting the inode even when not necessary.

That is certainly possible, if that is normal POSIX behavior.

> I'd suggest using them in conjunction with find(1) if you only want to
> modify files that don't match a particular setting.

Yup.

But it would be so much more elegant if chmod just Did The Right Thing.

Can I ask you to pass along this wishlist item to upstream and maybe ask
them why ctime must be modified even when the permissions are not?  I
would really like to know why.

-- 
Erik Rossen                               OpenPGP key: 2935D0B9
[email-masked]                          If you do not know what
http://people.linux-gull.ch/rossen        to do, start with RTFM.

---

From: Michael Stone <[email-masked]>
Date: Tue, 02 Sep 2008 10:48:30 -0400

>But it would be so much more elegant if chmod just Did The Right Thing.

It's certainly arguable whether adding another non-standard flag is the 
Right Thing when it's fairly trivial to get the result in a more 
portable fashion using find(1).

>would really like to know why.

Upstream monitors the debian bug logs and will presumably chime in if 
interested. I vaguely recall that people want these utilities to act 
even if not strictly necessary because of possible side effects for 
extended attributes.

Mike Stone

---

From: Jim Meyering <[email-masked]>
Date: Tue, 02 Sep 2008 20:06:51 +0200

> would really like to know why.

Good idea.
At least for chmod, it is not only possible, but the optimization
would be essentially free, since chmod already has the required stat data.

AFAICS POSIX
(http://www.opengroup.org/onlinepubs/007908775/xsh/chmod.html)
allows the behavior you propose.  That same feature was even
requested a few years ago, but no one implemented it.

A patch would be welcome.
If you're not interested, let me know in any case and
I'll add it to the TODO list.

If you are interested in writing the patch,
please follow the guidelines outlined here:
  http://git.sv.gnu.org/gitweb/?p=coreutils.git;a=blob;f=HACKING;hb=HEAD

For chgrp (probably chown, too, at least in some cases), it's not
as obvious, since the current implementation does not stat files
before changing permissions.  So, to do what you want would involve
adding a stat call per file to get owner/group in order to avoid(maybe)
the offending ownership-changing syscall.  That would not be an
improvement.

---

From: Erik Rossen <[email-masked]>
Date: Tue, 2 Sep 2008 20:56:43 +0200

> would be essentially free, since chmod already has the required stat data.

Yeah, I thought it was a good idea too.

> requested a few years ago, but no one implemented it.

Thanks for the lead - I had no idea where to go looking for POSIX specs.
But you got the wrong URL.  The URL above refers to chmod() the
function.  What we needed was chmod the utility.  Here it is:

	http://www.opengroup.org/onlinepubs/007908799/xcu/chmod.html

And, if one wants to be REALLY pedantic, it looks like the file node is
supposed to be changed each time.  For example, here is an extract:

	+
	    If perm is not specified, the + operation will not change
	    the file mode bits. If who is not specified, the file mode
	    bits represented by perm for the owner, group and other
	    permissions, except for those with corresponding bits in the
	    file mode creation mask of the invoking process, will be
	    set. Otherwise, the file mode bits represented by the
	    specified who and perm values will be set.

Here it says that the bits will be set.  Unfortunately, they do not say
something like "set unless already set".  If the feature is to added, it
will be a non-standard option.

> A patch would be welcome.

I totally agree.

> I'll add it to the TODO list.

I think I will let you add it to the TODO list. :-)

> the offending ownership-changing syscall.  That would not be an

As far as speed is concerned, you are right that an extra stat() would
usually not improve matters.  (Would it be an enormous penalty?  I doubt
it.)  But users of file integrity checking systems like myself will be
grateful for the change of behavior.

---

From: Bob Proulx <[email-masked]>
Date: Tue, 2 Sep 2008 14:30:20 -0600

Erik Rossen wrote:
> Jim Meyering wrote:

You would be surprised how much performance can change when adding
extra stat calls.  Many filesystems such as networked filesystems are
extremely latency bound and those kinds of changes have a strong
negative impact upon them.

Bob

---

From: Erik Rossen <[email-masked]>
Date: Tue, 2 Sep 2008 22:57:28 +0200

> negative impact upon them.

OK.  But I still wonder how expensive are the extra stat() calls that
would be spent compared to the extra chown() calls that could be saved
by checking first to see if a chown() was even necessary?

---

From: Erik Rossen <[email-masked]>
Date: Wed, 3 Sep 2008 16:05:46 +0200

I asked the people on the mailing list of our local LUG to test if chmod
changes ctime on non-GNU systems even when there is no difference beteen
the mode before and after the chmod.  From these few data, the trend
seems to be that ctime gets changed.

Here are the results thus far (identified by "uname -a"):

systems that change ctime:
--------------------------

AIX XXXXXXXX 3 4 000650834C00
SunOS XXXXXXXX 5.8 Generic_108528-29 sun4u sparc SUNW,UltraSPARC-IIi-cEngine
OpenBSD XXXXXXXX 4.1 GENERIC#187 i386

systems that do not change ctime:
---------------------------------

FreeBSD XXXXXXXX 7.0-RELEASE FreeBSD 7.0-RELEASE #0: Sun Feb 24 19:59:52 UTC 2008 [email-masked]:/usr/obj/usr/src/sys/GENERIC i386

---

From: Paul Eggert <[email-masked]>
Date: Thu, 11 Sep 2008 10:56:34 -0700

> supposed to be changed each time.  For example, here is an extract:

As I read the spec, chown and chgrp are explicitly required to make
the equivalent of a chown() call, which in turn is required to change
the ctime.  However, chmod is not required to make the equivalent of a
chmod() call, and there is no requirement in the 'chmod' spec that it
change the ctime.  So POSIX allows the optimization for the 'chmod'
command, but not for the 'chown' and 'chgrp' commands.

---

From: Eric Blake <[email-masked]>
Date: Thu, 11 Sep 2008 21:20:21 -0600

> command, but not for the 'chown' and 'chgrp' commands.

On the other hand, the POSIX spec for chmod(1) mentions that it is
implementation defined on how it affects alternate access control.  I have
come to expect on many systems that support ACLs, that 'chmod 755' on a
file with rwxr-xr-x+ permissions will remove the ACLs.  In other words,
optimizing away the chmod(2) call because it compares equal to the stat
information would break this side-effect of clearing ACLs.

- --
Don't work too hard, make some time for fun as well!

Eric Blake             [email-masked]

---

From: Jim Meyering <[email-masked]>
Date: Fri, 12 Sep 2008 21:51:04 +0200

> information would break this side-effect of clearing ACLs.

That sounds like a good reason to retain the behavior you've come to
value, even if it's not guaranteed or portable, but only via a new
option.  Then we can still change the default to be more efficient.

---

From: Michael Stone <[email-masked]>
Date: Fri, 12 Sep 2008 16:05:09 -0400

>option.  Then we can still change the default to be more efficient.

Why on earth would we want to? Some people obviously like the current 
behavior, and depend on the side effects, the desired behavior is easy 
to get with existing tools, and adding a new option seems like something 
that shouldn't be done without a very good reason. This seems like 
optimization for the sake of optimization. (And it would make chmod 
inconsistent with chown and chgrp.)

Mike Stone

---

From: Jim Meyering <[email-masked]>
Date: Sat, 13 Sep 2008 09:50:35 +0200

Michael Stone <[email-masked]> wrote:
> to get with existing tools, and adding a new option seems like
> something that shouldn't be done without a very good reason. This
> seems like optimization for the sake of optimization. (And it would
> make chmod inconsistent with chown and chgrp.)

I don't feel very strongly one way or the other,
since I don't plan to do the work.

Realize that afaik no one is working on this.
I merely added the item to the TODO list, and we
know from experience that that's no guarantee it'll be done.

---

From: Erik Rossen <[email-masked]>
Date: Mon, 15 Sep 2008 11:22:41 +0200

> portable fashion using find(1).

It seems that the FreeBSD people think that making a change to the
filesystem only when necessary is The Right Thing.  But I guess that
argument carries no weight in GNU. ;-)

> even if not strictly necessary because of possible side effects for 

I took your advice (paraphrased: "try another tool") and I now have
three options for fixing file permissions without disturbing the file
system too much:

1.  find + chmod : classic but ugly

2.  cfengine : Does The Right Thing, but difficult to do as a one-liner
unless you are already using cfengine for other stuff

3.  setfacl -R : nice.  Does The Right Thing as a fairly simple one-liner

setfacl is kind of interesting.  Even if the filesystem is mounted
without extended ACLs enabled, it works for setting the POSIX ACLs and
it Does The Right Thing.  For example, I can do

	setfacl -R -m u::rwX,g::rwX,o:rX shares/

to fix every file and directory in "shares" to be readable, and writable
only by the owner and group of shares.  And if I run the same command
again, stat shows that the ctime has not changed unless it was necessary
to modify the permissions of a file or directory.

I am pretty happy about this solution and I intend to use it in cases
where there is a integrity-checker in place and no cfengine.

But while doing my tests, I noticed something strange about setfacl: if
the filesystem is mounted with extended ACLs enabled (on a ext3 fs), it
is possible to use setfacl to make changes to file permissions and CTIME
IS NEVER MODIFIED EVEN IF PERMISSIONS ARE MODIFIED.  I thought that this
was supposed to be impossible, but perhaps this is a known side-effect
("feature") of activating extended ACLs on ext3.

Can anyone else confirm this?  I am running etch with an ext3 filesystem
that was created in June 2008. This is my "uname -a":

	Linux mango 2.6.18-6-vserver-686 #1 SMP Mon Aug 18 11:11:15 UTC 2008 i686 GNU/Linux

-- 
Erik Rossen                               OpenPGP key: 2935D0B9
[email-masked]                          On fonce la tête dans
http://www.rtfm-sarl.ch                   le guidon et on RTFM.

---

From: John Lines <[email-masked]>
Date: Tue, 13 Oct 2020 12:06:59 +0100

>       Linux mango 2.6.18-6-vserver-686 #1 SMP Mon Aug 18 11:11:15 UTC
2008 i686 GNU/Linux

I have another use-case for only attempting to make a change with chmod
if it is really necessary.

In a Debian postinst script (for VDR in this particular case) the
ownership of a directory (/var/lib/video) is changed - however I have
moved /var/lib/video to be a symlink to an NFS server. The permissions
are already set correctly, but as root can not (in this case) change an
NFS mounted file system, the chmod fails, even though it wanted to
change the owner to be the same as it already was.

A switch for chmod to say --only-if-needed or some such would avoid
this being an issue

John