Reply to comment
digiKam digest - 2009-06-14
Submitted by mikolajmachowski on Mon, 2009-06-15 17:52Most important things this week were behind curtains:
- First test units were created and already helped to fix some bugs
- Database for thumbnails is under heavy testing, DB internal for digiKam could give:
- savings in HD space (even today it is problem with big collections, internal DB will save 75% of space comparing to .thumbnails solution)
- faster creation of thumbnails and loading them later making for smoother experience
- thumbnails for images in off-line storage will be always visible
NOTE: DB support is not compiled by default, use -DENABLE_THUMBS_DB=ON cmake switch if you want to test it, or turn it on later with the ccmake command.
On less optimistic note: issues with efficiency of queries in big collections can hold project.
- Fast action from Mandriva team clearing Qt3-isms
- Optimizations in database queries
- Optimizations in "finding duplicates" tools
- Non Latin1 users should find life easier with less strict treating of non-ascii characters
| Opened bugs | Opened last week | Closed last week | Change | Opened wishes | Opened last week | Closed last week | Change | |
|---|---|---|---|---|---|---|---|---|
| digikam | 234 | +22 | -21 | 1 | 276 | +11 | -3 | 8 |
| kipiplugins | 109 | +7 | -4 | 3 | 146 | +5 | -0 | 5 |
Full tables:
SVN commit 978709 by anaselli:
- fixed autorotate and scaling per photo
- changed "save configuration" to "save as default"
- changed photo x page to label
TODO get back tips :/
http://websvn.kde.org/?view=rev&revision=978709
3 files changed
SVN commit 978916 by aclemens:
Use cached signatures in bestMatchesForImage() as well if possible,
maybe this will improve the speed of Find Duplicates a little bit.
1 file changed
SVN commit 978895 by cgilles:
load and save Exif/iptc/xmp metadata to PGF file.
We use PGF header where a byte array can be encapsuled.
In this byte array, we store a JPEG thumb with all metadata.
This is not optimal because a tmp file is used. Exiv2 cannot write image data to a memory buffer.
When PGF will be supported in Exiv2, it will be more simple...
1 file changed
SVN commit 978789 by cgilles:
Gallery export plugin :
-restore Exif info after image resizing. Exif rotation flag must still here.
-Add RAW handling interface. Now, you can upload RAW image directly to Gallery server. Raw preview image is used (not RAW image data).
Note: Patch is simple and must work as well. PLease test and report if errors are found.
BUG: 194245
http://websvn.kde.org/?view=rev&revision=978789
3 files changed
SVN commit 978841 by aclemens:
Some small optimizations: even though we only use standard types here,
it is always good to move variable creation or calculations out of
loops. In some situations, this can really improve speed (ok in this
commit it will not matter that much).
I will go through the digiKam code and fix such issues now and then...
1 file changed
SVN commit 979007 by mwiesweg:
Add logic for storing and loading thumbnails from database.
Initial revision, needs testing.
2 files changed
SVN commit 979005 by mwiesweg:
To use a database for thumnail loading, core database code need to be available in libdigikamcore.
Split DatabaseBackend to a generic DatabaseCoreBackend (wrapper code to Qt SQL) and leave
schema-specific code in DatabaseBackend. DatabaseCoreBackend and DatabaseParameters move to libdigikamcore,
as do the thumbnail-db classes.
http://websvn.kde.org/?view=rev&revision=979005
9 files changed
SVN commit 979218 by aclemens:
Add initial tests for ManualRenameInput widget. Right now they do
nothing, tests need to be written.
5 files changed
SVN commit 979191 by cgilles:
use preview JPEG image previuously scaled in IPTC.preview to host PGF metadata.
1 file changed
SVN commit 979187 by cgilles:
libpgf bug : SetHeader accept only unsigned short user data size where GetUserData can read unsigned int ! This is not homogenous and limit user data to 65536 bytes.
There is no reasons for that...
2 files changed
SVN commit 979368 by aclemens:
New parser option: $, % and & can be used for the camera name now, too.
In the future these tokens will also applied to metadata information.
Add new test cases.
2 files changed
SVN commit 979376 by aclemens:
Don't call QFileInfo::baseFileName() all the time, save the result to a
QString
1 file changed
SVN commit 979598 by aclemens:
The fine angle settings don't make sense, why use a range of -5.0 to 5.0
degrees?
New range: -1.0 to 1.0
1 file changed
SVN commit 979676 by nlecureuil:
Qt3Support--
1 file changed
SVN commit 979701 by aclemens:
setCheckState doesn't seem to work when using an int here, so I change
the appropriate signals and slots to Qt::CheckState.
3 files changed
SVN commit 979991 by aclemens:
Fix fast duplicates search.
We are now able to select an album range for the find duplicates search,
this also means we can have no album selected. Normally the find
duplicates search will fail if no album ids are present,
but the fast scan doesn't need any album information.
It always scans the whole database and finds duplicates with a SQL query.
If fast scan is activated and the albumid list is empty, don't quit the
search.
1 file changed
SVN commit 980179 by cgilles:
apply patch from Simone Lazzaris to change sort of camera icon view item : in chronological order or not.
Factorize code.
Bug fix : there is a race condition between camera gui and camera controler when thumbnails are processed:
camera controller switch randomly between busy state or not when camera gui fill the list of item thumbs to get one by one.
A new method fill all thumbs to get in a first time : camera controler will be still busy during this task.
BUGS: 193124
BUGS: 188051
7 files changed
SVN commit 980362 by aclemens:
Add "find duplicates" context menu action to the folderview.
This way you can easily set the find duplicates search from within the
folder view.
The currently highlighted album and all its children will be selected.
This works for collection roots, too.
8 files changed
SVN commit 980342 by aclemens:
Add a threshold spinbox to the find duplicates search.
The default threshold value is not enough sometimes, 87% might be too
high in some cases and duplicate (or very similar) images are not found.
With the spinbox, you can query with different thresholds.
Now that we can limit the search to specific folders, users might run
the find duplicates search more often then before, because it is way
faster now.
To test different values, it is important to have customization here.
The value will not be saved in the config (for now), because I don't
think it is necessary. But if we want to save it, it is easy to change
:-)
Andi
1 file changed
SVN commit 980383 by cgilles:
IPTC key => QString::fromLocal8bits()
IPTC val => QString::fromAscii()
CCBUGS: 189979
1 file changed
SVN commit 980401 by mwiesweg:
Add a global static method initializeThumbnailDatabase to switch on using a thumbnail database.
This needs to be called at application startup, before thumbnail threads are created
- we cannot switch a running thread currently.
2 files changed
SVN commit 980402 by mwiesweg:
Adding the lines that switch on using a thumbnail database - not enabled per default:
NOTE:
You need to remove the comment signs around albummanager.cpp, lines 685-687, to test the thumbnail database.
For simplicity, it is always created in the directory of the main database and named "thumbnails-digikam.db".
It is using the PGF format to store thumbnails. I am aware there are suggestions to use JPEG or Djvu.
Only real world testing can show if this database is a good thing or not.
CCMAIL: digikam-devel@kde.org
1 file changed
SVN commit 980398 by mwiesweg:
We need unique connection; each DatabaseBackend (we may have two) needs its own backend name.
Combined with the thread PID can a unique name per thread and object be created.
4 files changed
SVN commit 980450 by aclemens:
Remove fast search and the find duplicates dialog again.
This is not needed anymore since find duplicates is able to search in
album ranges.
4 files changed
SVN commit 980642 by aclemens:
Add unit to guide input widget
1 file changed
SVN commit 980632 by cgilles:
add support of JPEG and JPEG2000 image format for storing thumbs in database. Only for testing at this moment.
2 files changed
SVN commit 980643 by aclemens:
Use ColorGuide from the editor tool like all the other imageplugins
2 files changed
SVN commit 980595 by aclemens:
It doesn't make sense to have a range of -5.0 to 5.0 degrees for the
fine angles.
1 file changed
SVN commit 980692 by cgilles:
adjust PGF quality from 3 to 4. better compression ratio with very good image quality for thumbs.
1 file changed
SVN commit 981048 by mwiesweg:
Incremental updating had a bug - strange noone noticed till now.
There were two hash pointing to indexes of the d->infos list, but the indexes were not updated.
Change filePathHash to point to imageids, one inconsistency less to worry about.
Update idHash properly.
1 file changed
SVN commit 981229 by aclemens:
effect is started when settings are made, no need for a Try button here.
1 file changed
SVN commit 981228 by aclemens:
When clip focus changed, update the filter
2 files changed
SVN commit 981170 by cgilles:
Update libpgf to official 6.09.24 including all my previous fixes !!!
Thanks to libpgf team...
http://websvn.kde.org/?view=rev&revision=981170
14 files changed
SVN commit 981233 by aclemens:
Change the behavior of the perspective widget.
If the perspective was invalid, the old behavior was to disable dragging
the handles further. This feels really weird and some users might think
the tool is broken.
The new method is to allow dragging, but disabling the Ok button if the
perspective is invalid.
This is also the preferred behavior in Gimp, but Gimp will render the
perspective anyway when pressing Ok (which is weird, because it
disallowed these settings first). We will not allow to render in this
case.
4 files changed
SVN commit 981230 by aclemens:
This filter can be rather slow. Add a Try button and remove the slot
connections.
1 file changed
SVN commit 981225 by aclemens:
Widgets get re-parented when added to a layout. We should not set a
parent in the constructor (if possible), it is quite "useless" and
distracting. This was the old Qt3 style where a parent was absolutely
necessary.
Move the layout construction to the place where it is needed, this also
increases readability.
Polish the layout code...
http://websvn.kde.org/?view=rev&revision=981225
25 files changed
SVN commit 981226 by aclemens:
Like in FreeRotation, don't save the angle values. You will mostly never
want the same settings applied to other images.
We might think of a better solution in the future to save settings.
1 file changed
SVN commit 981227 by aclemens:
This filter takes some time, this is why we use a Try button here. Don't
connect the slotEffect() for this filter.
1 file changed
SVN commit 981297 by cgilles:
compile under Sun Studio
BUG: 196271
1 file changed
SVN commit 981351 by aclemens:
Add cmake option DIGIKAM_THUMBS_DB
This is just a temporary solution, we will not need it in the future.
But for now it becomes quickly annoying to set and revert the database
code when you want to test the thumbs db and the freedesktop solution.
To enable the thumbs db, add
-DDIGIKAM_THUMBS_DB=yes
to your cmake configuration.
Andi
CCMAIL:digikam-devel@kde.org
3 files changed
SVN commit 981305 by cgilles:
fix 90% of libpgf GCC warnings. Still unused variable only checked with C++ assertions.
5 files changed
SVN commit 981366 by aclemens:
Don't scan thumbs that are already created.
This works quite fine, but it is not an optimal solution. We should add
a method that returns all valid thumb paths from the db and use it here
instead of checking every thumb path with its own query.
Marcel,
what do you think?
Andi
CCMAIL:digikam-devel@kde.org
1 file changed
SVN commit 981411 by aclemens:
Add some debug info to the dbstatdlg for now.
It happened to me twice now that I didn't know if I'm using the thumbs
db or not.
This will be removed when switching to the thumbs db per default.
1 file changed
SVN commit 981385 by aclemens:
Use a query to retrieve valid thumbnail paths.
3 files changed
SVN commit 981530 by cgilles:
with JPEG, we will use quality = 85 to reduce artificats
1 file changed
SVN commit 981540 by cgilles:
make option visible from CMake command line to switch on/off thumbs database support
1 file changed
SVN commit 981606 by aclemens:
Using a QStringList as a return value for getValidFilePaths() seems to
be very slow on a collection with over 50.000 images.
According to the Qt docs and some guys from the #qt IRC channel, q QHash
should be much faster.
I tested it and I can not see a difference, but I will change it for
now. Maybe we will find a better solution in the future.
3 files changed
SVN commit 981717 by cgilles:
check tag size before to convert to long with Exiv2
BUG: 192085
1 file changed
SVN commit 981792 by aclemens:
Add reset button to the curves widget in BWSepia tool.
This could be factorized, right now it is just a copy & paste.
Maybe we should think of this when planning new editor interface.
BUG:196465
2 files changed
SVN commit 981756 by cgilles:
with 85 JPEG compression, i can see artifacts. try 90
1 file changed
SVN commit 981989 by aclemens:
Optimize query
The keywords IN, NOT IN, !, <>, OR, NOT EXISTS will not use the index of
the table and therefore perform bad.
The new query will also reduce the SQL statements from 46 to 24
instructions, which is good ;-)
Unfortunately the query is still very slow, because the index is set on
path for FilePaths and uniqueHash for Thumbnails.
This is why a lot of thumb queries with the new thumbsDB seem to be
slower. When joining the FilePaths table with the Thumbnails table, we
are not using both indexes, but only one, therefore we don't benefit
from the index tables.
My thumbsDB is 600MB and is really slow to query. We either need to
think of new index tables or try to improve speed with other solutions.
When thumbsDB grows even more, digiKam becomes unusable on my system.
Andi
CCMAIL:digikam-devel@kde.org
1 file changed