What is the data limit on Essential PIM

General talks about EssentialPIM

Moderators: Max, TerryRogers

Post Reply
Heavenstream
Posts: 1
Joined: Tue Jul 20, 2021 2:32 pm

What is the data limit on Essential PIM

Post by Heavenstream »

While I am assuming that it unlimited I am curious as to what the limits for the software is as I have around 1600 notes in the software right now and have barely scratched the surface. I need to know if the software will be able to keep up if I have say about 10000 notes or more thanks!
MetalDrop
Guru
Posts: 791
Joined: Sat Apr 09, 2016 10:19 pm
Been thanked: 171 times

Re: What is the data limit on Essential PIM

Post by MetalDrop »

Yes and No it would seem.

I tried testing 10k notes in 9.9.6 and it was having major issues by 3k and unusable by 5k notes.

Because I know I stressed tested EPIM heavily back in the 6x days I grabbed a copy of EPIM 6x and made a databases with 30k notes that worked fine, then I preceded to update through my old versions of EPIM collection while adding more notes and found that something between version 9.7.5 -> 9.8 has crippled EPIMs ability to handle large numbers of notes.

9.7.5 could easily handle 100k notes, where as 9.8 couldn't even function. [it became um-responsive for 5-10mins between each click so not crushing, but clearly bottle necked beyond anything usable]

So something added between 9.7.5 to 9.8 has crippled EPIMs ability to handle large numbers of notes.
EPIM Portable Pro on Windows 10 Pro 64-bit US-ENG, Intel i7 7700k, 32GB DDR4, Nvidia 970 & AMD Ryzen 2600 16GB DDR4 Nvidia 1080ti.
[I'm helpful and often reply to questions, please note however I am just a fellow user and not staff.]
Frank-H
Novice
Posts: 35
Joined: Thu Dec 29, 2005 3:28 am
Has thanked: 26 times
Been thanked: 2 times

Re: What is the data limit on Essential PIM

Post by Frank-H »

Re: MetalDrop's info above, epim has missed a fundamental concept:

Core program functionality (and dependability) is more important than 138 new features....

Great on taking input from people, and making a great "Swiss Army Knife", but above all you CAN'T be dropping core fundamental function as new versions come out, such as with contacts / fields / notes, as Metaldrop illustrated above.
MetalDrop
Guru
Posts: 791
Joined: Sat Apr 09, 2016 10:19 pm
Been thanked: 171 times

Re: What is the data limit on Essential PIM

Post by MetalDrop »

Frank-H wrote: Tue Jul 20, 2021 10:22 pm Re: MetalDrop's info above, epim has missed a fundamental concept:

Core program functionality (and dependability) is more important than 138 new features....

Great on taking input from people, and making a great "Swiss Army Knife", but above all you CAN'T be dropping core fundamental function as new versions come out, such as with contacts / fields / notes, as Metaldrop illustrated above.
All new features will come with risks no matter how long you test them. Microsoft and Apple both routinely cripple peoples computers with updates that were tested by tens of thousands of people for weeks and months on end.

What it seems your really looking for would be an LTS [long term support] update channel. That way you personally aren't exposed to "new features" that haven't been around for the general people to figure out the harder to find bugs for a while.

A lot of LTS software has 3~10 years of feature locks, so one question to ask yourself is would you personally really be willing to still be using EPIM version 6 despite version 9 being an option?

EPIM 6x is a very nice classic branch of EPIM and was the last version of the classic less feature intense update branches. Which would make it ideal for anyone looking for the most stable version of EPIM.

I personally consider version 6 to be the gold standard of EPIM, and the number one question I tend to ask myself when odd limitations come up is: Was this an issue with Version 6?
EPIM Portable Pro on Windows 10 Pro 64-bit US-ENG, Intel i7 7700k, 32GB DDR4, Nvidia 970 & AMD Ryzen 2600 16GB DDR4 Nvidia 1080ti.
[I'm helpful and often reply to questions, please note however I am just a fellow user and not staff.]
Frank-H
Novice
Posts: 35
Joined: Thu Dec 29, 2005 3:28 am
Has thanked: 26 times
Been thanked: 2 times

Re: What is the data limit on Essential PIM

Post by Frank-H »

Very good info there MetalDrop. My "dip in the ice water" may be a bit unique, following epim for a long time (2005 joining the forum), buying 4 times, but just now putting it into use (business issues / changes), 9.9.6, and "simply adding custom fields" (one contact), finding the system crashed "over and over, frozen hard" after about 15 to 20 (simple) fields created, even re-installing 9.9.6 and starting again from scratch, only to have the same crashing. So, I can't believe 9.9.6 was tested well (again my experience is a bit unique, bringing over no past-version-created database). And others here, had their issues, with 9.9.6.

So, similar concept to what you have expressed, I'll work only with "known good" versions, and religiously have backups of the actual database.

via Support email, I eventually got a link to a "this shouldn't crash" work in progress update version of 9.9.6, a version after the release of 9.9.6, and it has been stable (with the exact same database that was crashing in the version from purchase, earlier in July here). I just checked, and the file on the website now as 9.9.6 is the identical file size as what I downloaded July 4th. So, no change there. But there is a "better fixed up" version in the pipeline; the version support sent me.

PS MetalDrop (and again thanks for your sharing of info, and your experience), I see on the "old versions of the program" page on zendesk, re: 6.x they have a 6.05 and a 6.58, do you have any advice as to which one would be most like what you described, stable / best version of that era? (I see 6.58 was from April 2016, checking the version history page). [ edit - I found that 8.05 and earlier are dead link, "no download", on that old versions download page... ]
Last edited by Frank-H on Wed Jul 21, 2021 2:48 am, edited 2 times in total.
Rose Ford
Guru
Posts: 136
Joined: Sun May 08, 2016 8:39 am
Has thanked: 1 time
Been thanked: 20 times

Re: What is the data limit on Essential PIM

Post by Rose Ford »

MetalDrop wrote: Wed Jul 21, 2021 1:58 am A lot of LTS software has 3~10 years of feature locks, so one question to ask yourself is would you personally really be willing to still be using EPIM version 6 despite version 9 being an option?

EPIM 6x is a very nice classic branch of EPIM and was the last version of the classic less feature intense update branches. Which would make it ideal for anyone looking for the most stable version of EPIM.

I personally consider version 6 to be the gold standard of EPIM, and the number one question I tend to ask myself when odd limitations come up is: Was this an issue with Version 6?
I saw this part and just had to make a comment since this is very much my life working with Linux systems at work.

Working with Linux systems you can either ride the bleeding edge and have massive issues all the time, or you can stick to the tried and true LTS versions that often feel like you're still living in <2012.

Some days I get home from work sit down at my personal workstation and feel like I'm living some strange slice of life Back to the Future sitcom.

Stability has it's perks, and in some cases it really is worth sticking to the most stable version of things no matter how outdated they feel or what "fancy" features they're missing. As long as it works, and it's making you money that's the—only thing that matters. At least in the world of business.

So I'd +1 that, you want stability ditch v9 and download a copy of v6 for notes. No fancy features to cause you issues there. I even still use a copy of v6 on a windows XP computer at work because it works better with the windows XP classic theme and I don't need any features that came after v6 on that computer.

That said some email providers don't work in v6 because times have changed and it lacks security features that Google nowadays requires. If they released a v6 patch to get google email up and running though it would still be feature complete, haven't ever ran into any other issues or limitations.

All of the above aside though, I very much enjoy v9 and am loving the bleeding edge features on a personal level. I'm more than happy to live with waves of bugs to get waves of fresh features, and the EPIM dev team does the community right. When bugs get reported they get fixed, and you rarely have to roll back for more than a month or two.
MetalDrop
Guru
Posts: 791
Joined: Sat Apr 09, 2016 10:19 pm
Been thanked: 171 times

Re: What is the data limit on Essential PIM

Post by MetalDrop »

v6.58 would be IMO the best v6 though as mentioned above by someone else V6 no longer works for Gmail and some other email providers, otherwise v6.58 is a great piece of historical software, and still the best version of EPIM for windows 7 and earlier if you use the classic theme. ladder version of 7x got some good updates to make it work better with the classic theme, but it isn't the same vintage feel that still has a cult following.
EPIM Portable Pro on Windows 10 Pro 64-bit US-ENG, Intel i7 7700k, 32GB DDR4, Nvidia 970 & AMD Ryzen 2600 16GB DDR4 Nvidia 1080ti.
[I'm helpful and often reply to questions, please note however I am just a fellow user and not staff.]
a8907433
Guru
Posts: 719
Joined: Fri Mar 12, 2010 11:57 pm
Been thanked: 132 times

Re: What is the data limit on Essential PIM

Post by a8907433 »

A very interesting discussion here.
@MetalDrop, Just one question: How could one revert from V9.x to V6.x- there were changes to the db structure itself, a db opened in V9.x would not work in V6.x!?
I just not have the time for so much testing....
MetalDrop
Guru
Posts: 791
Joined: Sat Apr 09, 2016 10:19 pm
Been thanked: 171 times

Re: What is the data limit on Essential PIM

Post by MetalDrop »

a8907433 wrote: Wed Jul 21, 2021 4:33 am A very interesting discussion here.
@MetalDrop, Just one question: How could one revert from V9.x to V6.x- there were changes to the db structure itself, a db opened in V9.x would not work in V6.x!?
I just not have the time for so much testing....
Your best bet is to use bulk export and re-import.

To test the note limit, I bulk exported as single .rtf files from all my databases and then combined them into 6.58 which is what gave me my starting 30~ish thousand notes for testing.

And is also why I was quite the vocal advocator of bulk single item exports, it's really useful way to move large amounts of data in and out of EPIM.

I imagine similar tactics could be used for other models as well, but I only truly use notes extensively.
EPIM Portable Pro on Windows 10 Pro 64-bit US-ENG, Intel i7 7700k, 32GB DDR4, Nvidia 970 & AMD Ryzen 2600 16GB DDR4 Nvidia 1080ti.
[I'm helpful and often reply to questions, please note however I am just a fellow user and not staff.]
MetalDrop
Guru
Posts: 791
Joined: Sat Apr 09, 2016 10:19 pm
Been thanked: 171 times

Re: What is the data limit on Essential PIM

Post by MetalDrop »

Update on this as of 9.9.7:

EPIM is now functional again with 100k+ notes.

However swapping between note tree tabs with over 30k notes causes 3~5 second freeze/lag which didn't happen in 9.7.5 and below.
EPIM Portable Pro on Windows 10 Pro 64-bit US-ENG, Intel i7 7700k, 32GB DDR4, Nvidia 970 & AMD Ryzen 2600 16GB DDR4 Nvidia 1080ti.
[I'm helpful and often reply to questions, please note however I am just a fellow user and not staff.]
admin
Site Admin
Posts: 10380
Joined: Thu Nov 25, 2004 3:12 am
Has thanked: 949 times
Been thanked: 576 times

Re: What is the data limit on Essential PIM

Post by admin »

Allright, will look into this, as well. Thanks for letting us know and for the update, MetalDrop!
Android version of EssentialPIM. Keep all your data in sync!
outlinerone
Beta Team
Posts: 86
Joined: Fri May 06, 2005 4:41 pm
Has thanked: 3 times
Been thanked: 1 time

Re: What is the data limit on Essential PIM

Post by outlinerone »

Interesting discussion.
Have used EPIM off and on since the early days.
With later versions (have 9.9 beta 2 now) have noticed that when doing basic searches in Notes (where I store lots of script snippets, etc.) that search will focus on the first character and EPIM will delay or not respond at all for up to 2 minutes. This doesn't matter if searching all fields or just the note content.
Wondering if there is a general file size that one should keep an EPIM file to ensure responsiveness (in getting fast search results and in overall performance). This particular EPIM file I have is 43mb in size.
MetalDrop
Guru
Posts: 791
Joined: Sat Apr 09, 2016 10:19 pm
Been thanked: 171 times

Re: What is the data limit on Essential PIM

Post by MetalDrop »

outlinerone wrote: Fri Jul 23, 2021 8:05 pm Interesting discussion.
Have used EPIM off and on since the early days.
With later versions (have 9.9 beta 2 now) have noticed that when doing basic searches in Notes (where I store lots of script snippets, etc.) that search will focus on the first character and EPIM will delay or not respond at all for up to 2 minutes. This doesn't matter if searching all fields or just the note content.
Wondering if there is a general file size that one should keep an EPIM file to ensure responsiveness (in getting fast search results and in overall performance). This particular EPIM file I have is 43mb in size.
Sounds like your using an old beta that didn't get updated.

9.9 was already released and the current latest version is 9.9.7.
EPIM Portable Pro on Windows 10 Pro 64-bit US-ENG, Intel i7 7700k, 32GB DDR4, Nvidia 970 & AMD Ryzen 2600 16GB DDR4 Nvidia 1080ti.
[I'm helpful and often reply to questions, please note however I am just a fellow user and not staff.]
Post Reply