[Zope] backup full instances

[Tino Wildenhain]

Hi,

Jonas Meurer schrieb:
> On 27/10/2007 Andreas Jung wrote:
>>>> If you create a local copy, it is safe. If you rsync a running Data.fs,
>>>> it  is not safe. In this case you create a local copy and rsync the copy.
>>> but if rsync is unsafe only because it takes long time, and changes on the
>>> instance during the copy process could cause a corrupted Data.fs to be
>>> backuped, then theoretically this could happen for a local copy as
>>> well, right?
>> Think twice about your last sentence. What should cause a local *copy*
>> to be changed??
> 
> zope might write to the Data.fs while it is copied, thus an inconsistent
> copy would be backuped, even inside one filesystem.
> 
> i've asked in #debian on freenode as i wasn't sure, here's the log:
> 
> < mejo> if i copy a large file inside a mounted filesystem (ext3), is it
> 	possible that the file is changed during the copy process?
> < Wyzard> mejo: Yes, it's possible that something else can write to the
> 	file while you're copying it
> < mejo> because i asked on the zope-users mailinglist if i could backup
> 	the global Data.fs (database) while the daemon is running, and
> 	someone answered:
> < mejo> If you create a local copy, it is safe. If you rsync a running
> 	Data.fs it  is not safe. In this case you create a local copy
> 	and rsync the copy.
> < Wyzard> mejo: Making a local copy is faster, so it'd be safer, but
> 	still not completely safe
> < mejo> exactly, that's what i thought as well.
> < mejo> but when i wrote that, he replied:
> < mejo> Think twice about your last sentence. What should cause a local
> 	*copy* to be changed??
> < mejo> so he's wrong?
> < Wyzard> I'd say he's wrong
> < Wyzard> A local copy isn't instantaneous, and Zope changes the file
> 	while it's being read
> < wols_> he is wrong yes
> < mejo> thanks. is it ok for you if i quote you in my reply mail?
> < Wyzard> mejo: sure
> < wols_> mejo: while copying zope could change the database and create
> 	an inconsistent state

Nobody is wrong. Your #debian guys just had not all informations.
Zope is always appending to Data.fs, copy (cp) works linear, so
it would always maintain a consistent state of this file.
(It either copies before or after the last append operation)

rsync on the other hand is very efficient by calculating only
differences of the file contents to be copied. This may or may
be not following the order of blocks in the file. In the latter
case it could try to sync wrong information. (You would need
a special rsync which would only transfer new blocks at the end
in their given order - you could script something like this
using dd, gzip/zcat, ssh.

Regards
Tino