#1 2008-02-16 07:38:40

persicum
Veteran
Registered: 2007-09-25
Posts: 129

Ten times faster...

http://persicum.front.ru/flashcard/crc32.rar

Everyone are welcome to try my proggie!

Warnings:
1) No warranty of any kind
2) Console application
3) No readme
4) High HDD activity

Plus:
1) Subdir support
2) 1.5 times faster with regular Reed-Solomon
3) 10 times faster with Turbocode, but needs for decoding 10 extra blocks to succeed almost definetely (99.9%) and the more the better. Never mind in practice!!!

Last edited by persicum (2008-04-07 17:06:45)

Offline

 

#2 2008-02-16 11:38:42

fat
Administrator
Registered: 2004-02-11
Posts: 829

Re: Ten times faster...

Hi persicum and thanks for posting.

Not trying to take away from your work but I suggest that with no read me, no real explanation of purpose or functionality and no web page to explain plus direct linking to a rar file your not going to get alot of people looking at your prog.

I for one am not going to download a random exe to see what it does.

Just my 2cent.

<!-- ssmile --><img src="{SMILIES_PATH}/icon_smile.gif" alt="smile" title="Smile" /><!-- ssmile -->

Offline

 

#3 2008-02-22 14:58:15

persicum
Veteran
Registered: 2007-09-25
Posts: 129

Re: Ten times faster...

My program was updated to 1.15
Indeed, there is no long detailed help inside,
but there are some docs, short examples, benchmarks.

What's the proggie like?
It is the similar thing with QuickPar and ICEECC and it have the speed kinda between these two progs.

And also it has very fast 99.999...% probabilistic algorithm, and much other functionality

Offline

 

#4 2008-03-01 18:23:43

noram
Member
Registered: 2008-03-01
Posts: 1

Re: Ten times faster...

well looking for speed, maybe need to open another thread but could you please rename your project name cause crc32 is pretty common crc parity recovery or so

and btw my question is is quickpar better-faster when i create parity files that are small or when i create large one. i use to create 10% parity file for my dvd backups and even recovery is pretty slow (app 40min@axp2,2G). And does quickpar or your program supports sse extensions or multi-threading?

tnx for a new prog no virus there scanned with avg 7.5 w 20080226 db for those afraid

Offline

 

#5 2008-03-02 08:53:30

persicum
Veteran
Registered: 2007-09-25
Posts: 129

Re: Ten times faster...

well looking for speed, maybe need to open another thread but could you please rename your project name cause crc32 is pretty common crc parity recovery or so
[/quote:pxvcyaql]

The full name is "CheckRecoveryComparison" and since it is a console apps I like type "crc32" under the FAR command line

and btw my question is is quickpar better-faster when i create parity files that are small or when i create large one. i use to create 10% parity file for my dvd backups and even recovery is pretty slow (app 40min@axp2,2G). And does quickpar or your program supports sse extensions or multi-threading?
[/quote:pxvcyaql]

The main drawback of my prog is that it cannot encoding or repair "on the fly", instead it must to split your data into hundreds or thousands small files and that operate with them. The spliting takes about 2 minutes for CD or 15 minutes for DVD. But after that if there is enough memory with switch -mu the prog really encoding or decoding 10 times faster.
The algorithm differ from ReedSolomon 16-bit, and thus it can take all of advantages of MMX-64bit or SSE2-128bit. But multithreading is not yet implemented. But even without multicore it take me only 1 hour for 8000 user blocks.



tnx for a new prog no virus there scanned with avg 7.5 w 20080226 db for those afraid[/quote:pxvcyaql]

The prog is not a virus, spyware or fake, it is merely simple console apps

Offline

 

#6 2008-03-02 08:59:35

persicum
Veteran
Registered: 2007-09-25
Posts: 129

Re: Ten times faster...

And does quickpar or your program supports sse extensions or multi-threading?
[/quote:sv7yst3n]

QuickPar cannot take any benefit from 64bit or SSE because of 16-bit ReedSolomon. Moreover your standard 32-bit CPU works only in half of its power due to this stupid 16-bit GaluasField.

Neither QuickPar nor my prog support multithread, but ICEECC supports.
On the other hand, neither QuickPar nor ICEECC support MMX and SSE2, but MY PROG supports and accelerates 1.5 times with MMX and 2.3 times with SSE2.

Offline

 

#7 2008-03-02 18:30:28

par2ds
Active
Registered: 2006-04-15
Posts: 12

Re: Ten times faster...

Quickpar is closed to the community, the source code has not been released. But the command line is a different story. For that there is muliprocessor support. It's even in the sourceforge archives. It speeds up things quite a lot but the decoding is limited by the HD activity. Bu then again these solutions are compatible to the par2 format.
I've posted in/on the relevant threads in past. Search for par2ds in these forums and you will find..

Offline

 

#8 2008-03-05 10:28:14

persicum
Veteran
Registered: 2007-09-25
Posts: 129

Re: Ten times faster...

My proggie has been updated to v1.2
SSE2 support for matrix inversion was implemented.
Now big matrices such us 4000 x 4000, 8000 x 8000 or 16000 x 16000 would be inversed almost instantly without spending many-many hours or days.

Offline

 

#9 2008-03-10 18:24:53

Emopig
Member
Registered: 2008-03-10
Posts: 1

Re: Ten times faster...

persicum:

Is there any chance you will release the source code to CRC32? Or at least, a Linux binary? I tried to run it under Wine but it failed, and I currently can't use the likes of par2create which, according to my best estimates for one job i'm running, is going to take 7 days to run! >_<

Offline

 

#10 2008-03-20 11:53:54

persicum
Veteran
Registered: 2007-09-25
Posts: 129

Re: Ten times faster...

according to my best estimates for one job i'm running, is going to take 7 days to run!

Do you already have tried my prog and do you really like it? smile))
Please note that my prog is not "better" or "faster" than QuickPar, but it has an optional very fast probabilistic algorithm which requires about 20-30 extra blocks. So if you are planning to create say couple of  thousand recovery blocks then 30 extra blocks would be negligible, but if you are planning say make 100 recovery blocks then these 30 extra blocks will be consuming. It depends. And very likely you need an additional empty HDD for temporary storage, because my proggie do nothing on-the-fly, instead it swaps a lot. 

And my advise is please delay your great job until I will release a new version, namely 1.3
New features will be:
1) Equal sizing scheme - hundreds and thousands of files are obsolete now!!!  smile))
2) Power of two sizing scheme
3) Third encoding algorithm - LDPC - which 100 times faster then RS16 and can handle up to 200000 blocks. This will be a chance to operate with tens thousands of blocks easily. Contrary to DVDisaster, all blocks are GLOBAL and any n+20 can recovery any n blocks with probability 99.9999%

Last edited by persicum (2008-04-07 17:10:57)

Offline

 

#11 2008-04-07 17:08:21

persicum
Veteran
Registered: 2007-09-25
Posts: 129

Re: Ten times faster...

Hurray!!!

New version 1.3 has been released!
All features announced were successfully implemented

Offline

 

#12 2008-04-08 00:15:46

PeterBClements
Site Admin
From: United Kingdom
Registered: 2004-01-05
Posts: 1529
Website

Re: Ten times faster...

Persicum, whilst I welcome the fact that you have created your own data verification and repair program, the QuickPar forum is not really the place to advertise it.

You have your own web site which you should take the time to develop and show off your software. Please do not use mine.

Offline

 

Board footer

Powered by PunBB
© Copyright 2002–2008 PunBB

QuickPar © 2002–2008 Peter B. Clements