FreeArc Compression + .EPIM = Magic
Moderators: TerryRogers, Max
-
- Guru
- Posts: 142
- Joined: Fri May 13, 2016 4:10 am
- Been thanked: 9 times
FreeArc Compression + .EPIM = Magic
Today I decided to try compressing my EPIM backups into the .Arc file format using PeaZip.
20GB of backups compressed to 100MB.
Didn't believe it, so I copied the archive to another PC extracted it and tested random samples in the backup list and they all worked flawlessly.
I just wanted to let others know about this, and possibility the EPIM staff as well, because this is truly amazing. I had in the past just used 7zip ULTRA and it knocked the 20GB down to 10GB and took forever doing so.
A friend of mine was telling me how amazing the FreeARC compression format was for text and I wanted to try it out myself so that's why I did what I did since it was the biggest ball of text I could think of on my computer.
20GB of backups compressed to 100MB.
Didn't believe it, so I copied the archive to another PC extracted it and tested random samples in the backup list and they all worked flawlessly.
I just wanted to let others know about this, and possibility the EPIM staff as well, because this is truly amazing. I had in the past just used 7zip ULTRA and it knocked the 20GB down to 10GB and took forever doing so.
A friend of mine was telling me how amazing the FreeARC compression format was for text and I wanted to try it out myself so that's why I did what I did since it was the biggest ball of text I could think of on my computer.
-
- Guru
- Posts: 142
- Joined: Fri May 13, 2016 4:10 am
- Been thanked: 9 times
Re: FreeArc Compression + .EPIM = Magic
Update:
Tried this with my Email database backups and it didn't work as magically, just about the same as 7zip, but I was warned that it only worked well on text.
Still amazed at how well it handles my note databases. I mean yes in theory there is only 100MB of unique data spanning hundreds of backups so I shouldn't be super surprised, but I had tried every compression type 7zip offered at the highest settings my system could handle and nothing ever came close to this.
Tried this with my Email database backups and it didn't work as magically, just about the same as 7zip, but I was warned that it only worked well on text.
Still amazed at how well it handles my note databases. I mean yes in theory there is only 100MB of unique data spanning hundreds of backups so I shouldn't be super surprised, but I had tried every compression type 7zip offered at the highest settings my system could handle and nothing ever came close to this.
Re: FreeArc Compression + .EPIM = Magic
Did you compare it with the comperession built IN EPIM (File/Database Management/Optimize)?
-
- Guru
- Posts: 142
- Joined: Fri May 13, 2016 4:10 am
- Been thanked: 9 times
Re: FreeArc Compression + .EPIM = Magic
I did this for bulk backups not standard single file backups.
That said I went ahead and tested putting a single uncompressed epim file into an ARC and than compared it to the compressed backups and it is 30% smaller.
Using Arc on a single pre-compressed EPIM backup also cut the size in half resulting in an even smaller file.
-
- Site Admin
- Posts: 21714
- Joined: Wed Dec 08, 2004 11:39 pm
- Has thanked: 819 times
- Been thanked: 364 times
- Contact:
Re: FreeArc Compression + .EPIM = Magic
Different compression algorithms will compress data differently, but generally text data is well-packable. Usually compression comes at the cost of speed, hence most often used data is not compressed (or compressed lightly)
Maxim,
EPIM Team
EPIM Team
-
- Guru
- Posts: 142
- Joined: Fri May 13, 2016 4:10 am
- Been thanked: 9 times
Re: FreeArc Compression + .EPIM = Magic
Which is one of the amazing things about ARC is that it seems to be super FAST at least on my system. 20GB to 100MB in under 5mins which was even faster than just zipping the files at level 1 compression.
When compressing single databases at the highest setting the window barely even blinked on screen.
Which of course I wouldn't expect this to ever be used for the active database I think it should stay uncompressed, but it could be integrated into the backup system and save people massive amounts of data well letting them keep huge numbers of backups in the same space they could have only kept a couple.
Mainly I just wanted to let other uses know about it since ARC is a nearly unknown compression type and as I said I tried everything 7zip offers which is the most common, at the highest most painful settings possible and nothing even came into the same ballpark.
Re: FreeArc Compression + .EPIM = Magic
Great find!
Took 57 seconds for me to compress 10GB of note backups into 80MB
Took 57 seconds for me to compress 10GB of note backups into 80MB
EPIM Portable Pro Running/Tested On: Windows 11 Pro 64-bit US-ENG|i5-6400+Quadro P620|i7-7700K+1050ti|i7-8700K+970GTX|AMD 5600x+1080ti|16GB+RAM&NVMe SSDs
[I'm helpful and often reply to questions, however I am just a fellow user and not staff.]
[I'm helpful and often reply to questions, however I am just a fellow user and not staff.]
Re: FreeArc Compression + .EPIM = Magic
50GB of backups to 500MB here. I didn't pay attention to how long it took, but it was less time than it took me to eat a quick dinner.
- mvisconte
- Beta Team
- Posts: 236
- Joined: Fri May 06, 2005 1:10 am
- Location: East-west quadrant, Thyroid Nebula
- Has thanked: 50 times
- Been thanked: 15 times
Re: FreeArc Compression + .EPIM = Magic
I wasn't aware that Optimize featured compression. I think that it just removes deleted items from the database, resulting in a smaller, "tighter" database.
Re: FreeArc Compression + .EPIM = Magic
Optimizing doesn't compress to my knowledge, but the backup databases generated by the auto-backup system in EPIM are compressed.
EPIM Portable Pro Running/Tested On: Windows 11 Pro 64-bit US-ENG|i5-6400+Quadro P620|i7-7700K+1050ti|i7-8700K+970GTX|AMD 5600x+1080ti|16GB+RAM&NVMe SSDs
[I'm helpful and often reply to questions, however I am just a fellow user and not staff.]
[I'm helpful and often reply to questions, however I am just a fellow user and not staff.]
- mvisconte
- Beta Team
- Posts: 236
- Joined: Fri May 06, 2005 1:10 am
- Location: East-west quadrant, Thyroid Nebula
- Has thanked: 50 times
- Been thanked: 15 times
Re: FreeArc Compression + .EPIM = Magic
That verifies my experience. I hadn't measured it before, but I do see about a 20% reduction in the size of the backup files.
-
- Guru
- Posts: 142
- Joined: Fri May 13, 2016 4:10 am
- Been thanked: 9 times
Re: FreeArc Compression + .EPIM = Magic
I was messing around today and found ZPAQ
http://mattmahoney.net/dc/zpaq.html
It's even more magical as it works on encrypted Email heavy Databases as well!
I figured I bring it up here as a nudge for the EPIM devs to look into it.
Zpaq is damage resistant, handles duplicate exceedingly well, and allows very easy rollback and versioning of backups via command line.
If it was built into the EPIMs database auto backup system it could allow relatively limitless backups of databases, and with a tiny UI built for it in EPIM, would provide a nice User experience for database recovery and rollbacks.
Screenshot: 3x encrypted email database backups taking up over 3GB compressed to Zpaq in less than 1 Minute, and takes of less space than a single backup.
http://mattmahoney.net/dc/zpaq.html
It's even more magical as it works on encrypted Email heavy Databases as well!
I figured I bring it up here as a nudge for the EPIM devs to look into it.
Zpaq is damage resistant, handles duplicate exceedingly well, and allows very easy rollback and versioning of backups via command line.
If it was built into the EPIMs database auto backup system it could allow relatively limitless backups of databases, and with a tiny UI built for it in EPIM, would provide a nice User experience for database recovery and rollbacks.
Screenshot: 3x encrypted email database backups taking up over 3GB compressed to Zpaq in less than 1 Minute, and takes of less space than a single backup.
-
- Site Admin
- Posts: 15603
- Joined: Thu Nov 25, 2004 3:12 am
- Has thanked: 1412 times
- Been thanked: 984 times
Re: FreeArc Compression + .EPIM = Magic
Thanks for the info, will have a look at it.
Android version of EssentialPIM. Keep all your data in sync!