Debian Bug report logs - #673673
danger: rsync does sometimes not sync due to minimal things

Reply or subscribe to this bug.

Display info messages

Message #5 received at [email protected] (full text, mbox, reply):

From: frenchn00b <[email protected]>

To: Debian Bug Tracking System <[email protected]>

Subject: danger: rsync does sometimes not sync due to minimal things

Date: Sun, 20 May 2012 17:41:23 +0200

Package: rsync
Version: 3.0.7-2
Severity: normal

Hi,

With experience, I noticed several time that rsync is not robust. (Trust me).

If you take a pendrive for instance, it can do not like rsync -avr ... 
if you take weird partition and format of files, or lengths... rsync can miss some files , and cp -r -u is really much much much more robust.

OK. it is not the point to make rsync better, since it is likely to be tested with all kind of partition format (vfat, ext3, ...) and filenames, dir depths,...

I would recommend you cp -r -u, it is much more performant, .. ok, but sometimes we really need to use rsync 
what to do? how to verify?

I would propose to add a switch to verifiy that rsync copy well. We deal here about gb or tb or servers of data, so rsync shall not be a joke by the end



here an example of script:


#!/bin/sh
# /usr/bin/ducompare
# basic minimalistic rsync code checker 
# coded by frenchn00b

CURRENTDIR=`  pwd ` 

DIRSRC="$1"
DIRTRGT="$2"

if [ ! -d  "$DIRSRC"  ]  ||  [ ! -d  "$DIRTRGT"  ] ; then 
	echo "Not directories"
	echo "Exit" 
	exit
fi 


DIRSRCSTR=` basename "$DIRSRC" `
DIRTRGTSTR=` basename   "$DIRTRGT" `

if [ "$3" != "-checksrc" ]  &&  [ "$3" != "-checksrcs" ]  ; then 
#echo "	$DIRSRC		$DIRTRGT"
#printf "__STAT_"
printf  "%*s" 15 "Stat"
printf  "%*s" 15 "$DIRSRC"
printf "\t"
printf  "%*s" 15 "$DIRTRGT"  
printf "\n"


ls -1  "$DIRSRC"  | while read -r i ; do 
	SIZEDIRSOURCE=` du -hs --apparent-size  "$DIRSRC/${i}" | cut -f 1 `
	if [  -d  "$DIRTRGT/${i}" ] ; then 
		SIZEDIRTARGET=` du -hs --apparent-size "$DIRTRGT/${i}" | cut -f 1 `
		else
		SIZEDIRTARGET="not found"
	fi
	
	if [ "$SIZEDIRSOURCE" = "$SIZEDIRTARGET"  ] ; then 
		TESTVAL="OK"
		else
		TESTVAL="NOT_OK"
	fi	
        #printf "$TESTVAL"  
	#printf "\t"
	printf  "%*s" 15 "$TESTVAL"
	printf  "%*s" 15 "$SIZEDIRSOURCE"
	printf "\t"
	printf  "%*s" 15 "$SIZEDIRTARGET"
        printf "\t\t$i"  
	printf "\n"

done
fi



if [ "$3" != "-hide" ]  ; then 
echo "--- Reverse check ---"
#echo "Check the possible target missing directories"
ls -1   "$DIRTRGT"   | while read -r i ; do 
	FILET="$DIRSRC/${i}"
	FILETT="$DIRTRGT/${i}"

	if [ ! -d "$FILET"  ] &&  [ ! -f  "$FILET"   ]  ; then 
		SIZEFOL=""
		SIZEFOLSTRING=""
		if [ "$3" = "-checksrcs" ] ; then
			SIZEFOL=`du -hs  "$DIRTRGT/${i}" | cut -f 1 `
			SIZEFOLSTRING="(${SIZEFOL})"
		fi
        printf  "%*s" 15 "SRC NOT OK"
        printf  "%*s" 15 "not found"
        printf "\t"
        printf  "%*s" 15 "$SIZEFOLSTRING"
        printf "\t\t${FILETT}"  
        printf "\n"

	fi
	
done
fi




For the swtich checker, ... creativity ;)
Enjoy Linux
many thanks

-- System Information:
Debian Release: 6.0.3
  APT prefers stable-updates
  APT policy: (500, 'stable-updates'), (500, 'stable')
Architecture: i386 (i686)

Kernel: Linux 2.6.32-5-686-bigmem (SMP w/2 CPU cores)
Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash

Versions of packages rsync depends on:
ii  base-files              6.0squeeze3      Debian base system miscellaneous f
ii  libacl1                 2.2.49-4         Access control list shared library
ii  libc6                   2.11.2-10        Embedded GNU C Library: Shared lib
ii  libpopt0                1.16-1           lib for parsing cmdline parameters
ii  lsb-base                3.2-23.2squeeze1 Linux Standard Base 3.2 init scrip

rsync recommends no packages.

Versions of packages rsync suggests:
ii  openssh-client        1:5.5p1-6+squeeze1 secure shell (SSH) client, for sec
ii  openssh-server        1:5.5p1-6+squeeze1 secure shell (SSH) server, for sec

-- no debconf information

Message #14 received at [email protected] (full text, mbox, reply):

From: Paul Slootman <[email protected]>

To: frenchn00b <[email protected]>, [email protected]

Subject: Re: Bug#673673: danger: rsync does sometimes not sync due to minimal things

Date: Sun, 20 May 2012 18:31:30 +0200

tags 673673 + wontfix moreinfo
severity  673673 wishlist
thanks

On Sun 20 May 2012, frenchn00b wrote:
> 
> With experience, I noticed several time that rsync is not robust. (Trust me).

Sorry, I don't simply trust you.

If you know rsync is not robust, then show proof.

> If you take a pendrive for instance, it can do not like rsync -avr ... 

It can, but you need to read the manpage and use additional options to
cater for the shortcomings of the filesystems involved.


Perhaps your script is useful for some people, so thanks for that;
I'll leave this bug report marked as level wishlist and tagged wontfix,
so other people can find it. Perhaps you might give some thought to a
more descriptive title.

Paul

Send a report that this bug log contains spam.