Debian Bug report logs - #673673
danger: rsync does sometimes not sync due to minimal things

Full log

🔗 View this message in rfc822 format

X-Loop: [email protected]
Subject: Bug#673673: danger: rsync does sometimes not sync due to minimal things
Reply-To: frenchn00b <[email protected]>, [email protected]
Resent-From: frenchn00b <[email protected]>
Resent-To: [email protected]
Resent-CC: Paul Slootman <[email protected]>
X-Loop: [email protected]
Resent-Date: Sun, 20 May 2012 16:15:02 +0000
Resent-Message-ID: <[email protected]>
Resent-Sender: [email protected]
X-Debian-PR-Message: report 673673
X-Debian-PR-Package: rsync
X-Debian-PR-Keywords: 
X-Debian-PR-Source: rsync
Received: via spool by [email protected] id=B.133753037326769
          (code B ref -1); Sun, 20 May 2012 16:15:02 +0000
Received: (at submit) by bugs.debian.org; 20 May 2012 16:12:53 +0000
X-Spam-Checker-Version: SpamAssassin 3.3.1-bugs.debian.org_2005_01_02
	(2010-03-16) on busoni.debian.org
X-Spam-Level: 
X-Spam-Status: No, score=-2.9 required=4.0 tests=AXB_HELO_HOME_UN,BAYES_00,
	DKIM_ADSP_CUSTOM_MED,FORGED_YAHOO_RCVD,FOURLA,FREEMAIL_FROM,HAS_PACKAGE,
	HELO_LH_HOME,NML_ADSP_CUSTOM_MED,RCVD_IN_PBL,RDNS_DYNAMIC,XMAILER_REPORTBUG
	autolearn=no version=3.3.1-bugs.debian.org_2005_01_02
X-Spam-Bayes: score:0.0000 Tokens: new, 48; hammy, 150; neutral, 148; spammy,
	1. spammytokens:0.995-1--t$i hammytokens:0.000-+--H*M:reportbug,
	0.000-+--H*MI:reportbug, 0.000-+--H*x:reportbug, 0.000-+--H*UA:reportbug,
	0.000-+--H*x:4.12.6
Received: from 188-23-138-134.adsl.highway.telekom.at ([188.23.138.134] helo=debian05.lan)
	by busoni.debian.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32)
	(Exim 4.72)
	(envelope-from <[email protected]>)
	id 1SW8kX-0006xE-B6
	for [email protected]; Sun, 20 May 2012 16:12:53 +0000
Received: from debian05.lan (localhost [127.0.0.1])
	by debian05.lan (8.14.3/8.14.3/Debian-9.4) with ESMTP id q4KFfOR6004923;
	Sun, 20 May 2012 17:41:24 +0200
Received: (from sameklu@localhost)
	by debian05.lan (8.14.3/8.14.3/Submit) id q4KFfNgl004921;
	Sun, 20 May 2012 17:41:23 +0200
X-Authentication-Warning: debian05.lan: sameklu set sender to [email protected] using -f
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
From: frenchn00b <[email protected]>
To: Debian Bug Tracking System <[email protected]>
Message-ID: <[email protected]>
X-Mailer: reportbug 4.12.6
Date: Sun, 20 May 2012 17:41:23 +0200
X-Greylist: delayed 1877 seconds by postgrey-1.32 at busoni; Sun, 20 May 2012 16:12:53 UTC
Delivered-To: [email protected]

Package: rsync
Version: 3.0.7-2
Severity: normal

Hi,

With experience, I noticed several time that rsync is not robust. (Trust me).

If you take a pendrive for instance, it can do not like rsync -avr ... 
if you take weird partition and format of files, or lengths... rsync can miss some files , and cp -r -u is really much much much more robust.

OK. it is not the point to make rsync better, since it is likely to be tested with all kind of partition format (vfat, ext3, ...) and filenames, dir depths,...

I would recommend you cp -r -u, it is much more performant, .. ok, but sometimes we really need to use rsync 
what to do? how to verify?

I would propose to add a switch to verifiy that rsync copy well. We deal here about gb or tb or servers of data, so rsync shall not be a joke by the end



here an example of script:


#!/bin/sh
# /usr/bin/ducompare
# basic minimalistic rsync code checker 
# coded by frenchn00b

CURRENTDIR=`  pwd ` 

DIRSRC="$1"
DIRTRGT="$2"

if [ ! -d  "$DIRSRC"  ]  ||  [ ! -d  "$DIRTRGT"  ] ; then 
	echo "Not directories"
	echo "Exit" 
	exit
fi 


DIRSRCSTR=` basename "$DIRSRC" `
DIRTRGTSTR=` basename   "$DIRTRGT" `

if [ "$3" != "-checksrc" ]  &&  [ "$3" != "-checksrcs" ]  ; then 
#echo "	$DIRSRC		$DIRTRGT"
#printf "__STAT_"
printf  "%*s" 15 "Stat"
printf  "%*s" 15 "$DIRSRC"
printf "\t"
printf  "%*s" 15 "$DIRTRGT"  
printf "\n"


ls -1  "$DIRSRC"  | while read -r i ; do 
	SIZEDIRSOURCE=` du -hs --apparent-size  "$DIRSRC/${i}" | cut -f 1 `
	if [  -d  "$DIRTRGT/${i}" ] ; then 
		SIZEDIRTARGET=` du -hs --apparent-size "$DIRTRGT/${i}" | cut -f 1 `
		else
		SIZEDIRTARGET="not found"
	fi
	
	if [ "$SIZEDIRSOURCE" = "$SIZEDIRTARGET"  ] ; then 
		TESTVAL="OK"
		else
		TESTVAL="NOT_OK"
	fi	
        #printf "$TESTVAL"  
	#printf "\t"
	printf  "%*s" 15 "$TESTVAL"
	printf  "%*s" 15 "$SIZEDIRSOURCE"
	printf "\t"
	printf  "%*s" 15 "$SIZEDIRTARGET"
        printf "\t\t$i"  
	printf "\n"

done
fi



if [ "$3" != "-hide" ]  ; then 
echo "--- Reverse check ---"
#echo "Check the possible target missing directories"
ls -1   "$DIRTRGT"   | while read -r i ; do 
	FILET="$DIRSRC/${i}"
	FILETT="$DIRTRGT/${i}"

	if [ ! -d "$FILET"  ] &&  [ ! -f  "$FILET"   ]  ; then 
		SIZEFOL=""
		SIZEFOLSTRING=""
		if [ "$3" = "-checksrcs" ] ; then
			SIZEFOL=`du -hs  "$DIRTRGT/${i}" | cut -f 1 `
			SIZEFOLSTRING="(${SIZEFOL})"
		fi
        printf  "%*s" 15 "SRC NOT OK"
        printf  "%*s" 15 "not found"
        printf "\t"
        printf  "%*s" 15 "$SIZEFOLSTRING"
        printf "\t\t${FILETT}"  
        printf "\n"

	fi
	
done
fi




For the swtich checker, ... creativity ;)
Enjoy Linux
many thanks

-- System Information:
Debian Release: 6.0.3
  APT prefers stable-updates
  APT policy: (500, 'stable-updates'), (500, 'stable')
Architecture: i386 (i686)

Kernel: Linux 2.6.32-5-686-bigmem (SMP w/2 CPU cores)
Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash

Versions of packages rsync depends on:
ii  base-files              6.0squeeze3      Debian base system miscellaneous f
ii  libacl1                 2.2.49-4         Access control list shared library
ii  libc6                   2.11.2-10        Embedded GNU C Library: Shared lib
ii  libpopt0                1.16-1           lib for parsing cmdline parameters
ii  lsb-base                3.2-23.2squeeze1 Linux Standard Base 3.2 init scrip

rsync recommends no packages.

Versions of packages rsync suggests:
ii  openssh-client        1:5.5p1-6+squeeze1 secure shell (SSH) client, for sec
ii  openssh-server        1:5.5p1-6+squeeze1 secure shell (SSH) server, for sec

-- no debconf information

Send a report that this bug log contains spam.