You are not logged in.
#1 2006-05-04 20:23:59
- zzspectrez
- Member
- Registered: 2006-05-04
- Posts: 1
DVD backups large data sets
Hello,
I am new to the world of par2. I have been burned badly by bad dvd media which I have found to be very inconsistent in terms of reliability compared to cdr. Looking for some way to deal with this I found out about par2. Most of your howto's seem centered on usenet posts instead of data backup.
I have a large dataset of data I would like to backup to dvd since tape is out of the expense of personal backup. I have a little home file server which is setup as a software raid mirror. The raid is good for hardware failure but I still would like to have snapshot backups.
The dataset is quite large, so I was curious on how it is recommended to go about using the parity files.
1. 78 Gigs Mp3s 4-8mg file size
2. 420+ Gigs video 4-6 gig file size
1. Would backup procedure vary between scenario 1 and 2?
2. Do you make md5 hashes of files?
3. Archive the files ( no compression ) to split size of dvdr?
4. For Mp3's should I just burn to disc then create parity set of ISO instead of individual files??
5. Anything else i forgot... <!-- s
--><img src="{SMILIES_PATH}/icon_smile.gif" alt="" title="Smile" /><!-- s -->
thanks..
zzspectrez.
Offline
#2 2006-05-04 21:15:37
Re: DVD backups large data sets
You really need to read the PAR2 official specs on the PARchive site to understand how PAR2 works and what the strengths and weaknesses of it are for your application. The good news is it will do what you want, but the bad is that with current tools it is fiddly and will probably require much manual setup.
First off, don't think about trying to PAR your entire dataset (420Gb or 78Gb) as a single PAR2 set. This will be impossible unless all that data is online (on HDD) at once [i:c1qlg825]and in the same folder[/i:c1qlg825]. Split the job into logical subunits and PAR2 each set at a time. If that is 100 4-6Gb files, then it's 100 separate PAR2s. You might want to compile CDs of MP3s and PAR2 the ISO image as you suggest though. Part of the reason is the inability of all current PAR2 clients to support creation and repair from/to subfolders.
What you absolutely MUST NOT do is be tempted to split the files up in any way! If you want to create a PAR2 set for MyVid.MPG or MyMP3.ISO that's 4Gb in size, use that specific file as the input file for QuickPAR or any other PAR2 client, don't split it before you do the PAR. You want QP to reconstruct the original undamaged file if it has to, and that way you won't need any intermediate steps.
You then have to decide on a trade-off between speed of PAR2 creation/repair and size of the extra repair data you will keep. If you set the blocksize (the logical subunits of the fileset that PAR2 recognises and repairs) to a low value, you will get high granularity for repair, meaning you can repair with smaller chunks of repair data so for occasional single bit errors you'd need to keep less data; but this will mean a larger number of blocks, and the repair/creation time is proportional to that. Make the blocks 1/10th the size and you can afford to store much less repair data, but it will take 10 times as long to create and repair.
"zzspectrez":c1qlg825 wrote:
2. Do you make md5 hashes of files?[/quote:c1qlg825]
MD5 is at the heart of the PAR2 specs. Every file in the set has a full hash and a short hash (the first 16Kb) calculated and stored in the PAR2 file, and every block subunit has an MD5 calculated so that the block data can be recognised as valid/invalid. It is also used to verify all the elements of the PAR2 file data. You don't need to make your own MD5 calculations, that is done for you by the PAR2 client and used for all data recognition.
Any other questions, just ask. <!-- s--><img src="{SMILIES_PATH}/icon_wink.gif" alt="
[b:305xxjcy]IanB[/b:305xxjcy]
[i:305xxjcy][b:305xxjcy][color=red:305xxjcy]Assembler Roolz![/color:305xxjcy][/b:305xxjcy][/i:305xxjcy] Check out my utility software, [i:305xxjcy][b:305xxjcy]iJoin[/b:305xxjcy][/i:305xxjcy] and [i:305xxjcy][b:305xxjcy]iDecode[/b:305xxjcy][/i:305xxjcy]:
<!-- m --><a class="postlink" href="http://www.i-asm.com/">http://www.i-asm.com/</a><!-- m -->
Offline