Quite big picture base makes cam2pc VERY slow or even crash

[DaF]

I'm using cam2pc 4.3.3 ShEd and have about 65,000 pictures with 18 GB size and a thumbnail database of about 900 MB.

When I open cam2pc and browse through a folder with e.g. 100 pictures it can take a few minutes until all visible (and already created) thumbnails are loaded. During that time cam2pc is completely frozen. This takes much too long!

Sometimes cam2pc even crashes when I try to browse a folder. It's not just a single or always the same folder but always a different one. I suspect the huge database to be very slow and buggy. Even though I optimized it it doesn't get better. Well, a single index file of about 1 GB size doesn't seem to be very fast. What about one file per folder together with an index file? It might be worth thinking about when using more than just a few hundred/thousand pictures.

FYI, I have the Image Downloader run in the background. Maybe this causes some trouble?

This is quite annoying and a real show stopper. I do like cam2pc very much but I might have to go back to XnView or find something else unless cam2pc becomes more stable and faster when using large picture bases.

[masterjp]

I got the same problem with more than 2000 jpeg pictures in one directory. Cam2PC be entitled to be the fastest picture browser, but it does not come true. I would be nice if you can fix it. The problem is that cam2pc saves the thumbnails with a too big filesize or resolution. This takes to much time to store it within the database.

[nabocorp]

Hello,

the database appears like a single file but it is not. It is in fact an index file and a data file stored in a same file using internal an Windows mechanism. But access to these two files is really done separately as if it was two separate files.

The index file is loaded in memory at the startup of cam2pc once for all. The data file is kept open to allow faster access. I have spent a lot of time tuning this stuff trying all kinds of tweaks and I can ensure you that it is today as fast as it can be.

I also think that it is pretty stable : cam2pc can even crash without corrupting that database. Data is only appended to the data file and the index file is written in a temp file before being copied in the "archive". The only time the DB can get corrupted is during this copy to the archive which should be pretty fast.

I do not wish to split that database in all directories because I think this heavily pollutes the user's PC. Plus I do not think it would make things that faster as it would require to open the index/data files each time a directory is opened.

When you browse a folder, cam2pc is frozen just the time to list all the files of the folder, get their basic characteristics (size, creation date...). After that a separate thread is launched that is supposed to get the thumbnails from the database and update the thumbnail view. During this thumbnail refresh, cam2pc is not supposed to be frozen at all. If this is the case then it is an issue. However this never happened on my PC's so it may be hard to fix.

I admit that I did not test cam2pc with so many pictures especially that many (2000 for masterjp) in the same directory. My database contains around 5000 pictures...

I currently have one possible solution to replace the database with something else (http://www.sqlite.org). I have made preliminary tests on a small amount of data and there was not much difference, cam2pc being a bit faster as it is 100% designed to store thumbnails. I would have to go further and see if any performance difference can be seen with huge databases as the ones you have. In that case, a switch to SQLite can be done.

One thing is sure is that there is no short term solution to this. If you can stand it, you can try to disable thumbnail caching and let cam2pc recreate the thumbnails each time you browse a directory. It will surely be faster that what you have now!

Regards,
nabocorp

PS: cam2pc is not entitled on nabocorp's website as the fastest picture browser. It may be said so on other websites but we cannot control that!

[masterjp]

Another forum user posted that "Cam2PC" is the fastest picture browser. - My last reply was wrong. Sorry!

I will test Cam2PC under different conditions with directories with a large amout of picture files again.

[nursoda]

I have 20,000 pics in 280 directories consuming 7 GB on a fileserver. To start up, cam2pc 4.3.4 takes 30 seconds on my Athlon 1,3MHz 512MB Workstation, even with the thumb database disabled! What does cam2pc do when starting up - it should only be opening the same folder list (280 folders) twice without the database?

But really annoying is, that cam2pc reloads this after many operations and every little change within the picture folder - always consuming another half a minute again! That way, the program unfortunately gets quite unusable :(

[nabocorp]

Hello,

I will definitely look into using SQLite for the database in cam2pc to see if it makes things smoother. But do not expect this right now!

Regards,
nabocorp

[nabocorp]

To start up, cam2pc 4.3.4 takes 30 seconds on my Athlon 1,3MHz 512MB Workstation, even with the thumb database disabled!

I must check if there is a bug or not but you can already go to the Tools menu and Delete the thumbnail database. Tell me if this affects the cam2pc startup time.

Thanks,
nabocorp

[nabocorp]

Hello,

I have made some more tests with the new database system. I have built a database with 100,000 thumbnails that reaches 900 Mb in size.

The results I have are encouraging : building the whole database (so only write access in the database) is roughly two times faster. Random access to the database is roughly three times faster. So no doubt that next cam2pc will use the new system!

BUT this does not explain long cam2pc startup times as opening my 900 Mb database only takes 2 seconds on my machine. After thinking about it, I see one reason that could affect long startup times : Antivirus programs. If your antivirus is configured so that it scans every file opened on your system then it is probably checking the cam2pc database when cam2pc tries to access it. Of course this can take some time. Can you try disable your antivirus or add cam2pc.db to the exclusion list (most antivirus softwares allow an exclusion list) and see if this changes something?

Regards,
nabocorp

[DaF]

Wow, 2-3 times faster than the previous version sounds very good.

I put cam2pc.db onto the exclusions list of avast! but haven't noticed any increase in speed, yet. It still takes about 15-30 seconds untils the first thumbnails are being displayed after had clicked on a folder with already indexed pictures. The preview on the left side shows the image immediately but the thumbnails have a looong delay.

[nabocorp]

Wow, 2-3 times faster than the previous version sounds very good.

This 2-3 factor is only for pure database performance... You must add over this the time to decode the JPEG. Can't really tell now the final performance improvement as I have not yet started the integration in cam2pc.

Regards,
nabocorp

[nabocorp]

I must check if there is a bug or not but you can already go to the Tools menu and Delete the thumbnail database. Tell me if this affects the cam2pc startup time.

The bug I suspected is not. One thing that can make cam2pc startup time to be very long is the album : cam2pc saves which album is opened on exit. When you restart cam2pc, it just opens the album and refreshes the thumbnails. On XP, this is done in a separate background thread so there is no inpact on the startup time. But on all other platforms, the background thread trick does not work and so the refresh has to be done while loading cam2pc and this makes the startup time very long.

A better solution will be found in next version.

Regards,
nabocorp