GSoC: How is digiKam’s non-destructive editing? (This time with pictures!)
Hi everybody,
long time no blogging. Time to fix this bug.
So...without any long prologues - (hold your breath now) - it's working! (Say WOW™). But how well does it work you ask? Pretty good! Ok ok, let's get some overview.
You open some image in digiKam's image editor, you add some filters, do some transformations, use some color-altering-stuff untill you're satisfied with your creativity. Then you simply close the editor, autosaving kicks in, and bam, you have new version of your original image, which stayed completely untouched. Available versions can be then viewed/switched in the new sidebar widget (see pic below). The sidebar allows you to display all the available versions either as a simple list or as a tree, which is basically the same list with padded entries to reflect the relations between the images. Later it will also be possible to do some file operations in the sidebar, like Copy/Move/Remove etc. In the main view with thumbnails, (sub)versions are now marked by an (ugly) icon as you can see in the screenshot below, but I'm looking forward to hear your ideas about how to mark them in some better way (of course it will be on/off switchable). Let us hear your ideas or see your mockups!
You can set in the options to have only the latest selected version visible in the main digiKam view, or to have all of them visible and viewable all the time. In the first case, you select the image, then in the sidebar you select which version do you want to see/edit/work with and the pics get switched. Simple as that. They are all physically in the same folder, with the same filename but with "_v1" appended for the first version, "_v2" for second etc. So yes, the versions are complete images, just like in F-Spot versioning. The image's versions are saved in the original image format, except RAW (of course) - format for storing RAW versions is settable in the Settings, for now it's possible to do the saving in PNG or JPG.
Next new thing is list of all used filters/transformations/effects on a particular image. This is part of the new right sidebar tab. So you can see what modifications you did to the selected image. This comes very handy in the Image editor. You apply some filter, it will show up in the new sidebar. When you press undo, the last entry will get a 'disabled' look, so you know what was undo-ed. Also when you press redo, the entries will get 'enabled' accordignly. Then when you have a bunch of disabled entries and you apply some new tool, the disabled entries will get lost and they will be replaced by the new applied tool, just as you would expect. So in short, in image editor this list presents a visualised undo/redo list. In the future I'd like it to be able to dynamically switch any entries on or off to see how would the image look and also to change the used values for particular filters. Basic foundations for this are already layed down. This will later also allows you to take one set of modifications and apply it to any other images.
In the image editor, the 'Save' button is now replaced by 'New version' button. By default, when you're editing again some already created version, not the original image, the changes are saved back to that version. The new version file is created only in case you're editing the original. So, when you're editing some version of the original image and you want to have the changes in a new image (and preserve the old version), that's what the 'New version' button is for. It will create a 'subversion' of the current version. Another use is to open the original, modify, click 'New version', click 'Revert', do another set of changes, click again 'New version' and this way you can quickly create several versions off of the original image and after closing the Image editor, see them all next to each other.
And now you probably ask "When can I get all of this?" Currently it's all in KDE's svn repository along with Aditya Bhatt's Face recognition and Gabriel Voicu's Geotagging features and you can 'svn co'-it and build it yourself, but it's moreless an alpha quality code. This GSoC branch will be based on the soon-to-be-released 1.4 version and will be tagged as a 2.0 version around this Christmas.
We have a Coding sprint ahead of us in Aix en Provence, which I'm attending and very looking forward to. We'll discuss & work on the whole GSoC work, on some new ideas for digiKam and do improvements here and there. If you have some ideas, mockups, patches or simply something worth discussing during the sprint, just join the digiKam devel mailing list and shout :)
Whaouw !
I didn't you were working on that, but I was working on some pictures yesterday, and thought that it would be a great feature... and now it's there :) Thanks a lot, it looks great !
Btw, enjoy your time in Aix-en-Provence, beautiful city !
Stacks/outline
Nice! This looks really useful.
For ideas about a better way to do it, I might suggest two different situations, depending on whether the versions are grouped or not.
If there are grouped, I would show a stack. That is, have all the pictures on top of each other, with older lower in the stack and newer higher, with the newest at the top. Each newer version will be a couple of pixels down and to the right (or down and to the left for left-right languages), so it looks like a stack of pictures. Probably there shouldn't be more than 5-10 pictures in a stack (depending on the thumbnail size and how much each picture is shifted relative to the previous ones), so probably the last 5 versions would be fine. You wouldn't actually be able to see any detail from the other versions, the point would just be to allow you to tell that there is more than one version there.
For Showing each picture separately, I would just have an outline around the entire group of versions. There shouldn't be any situation where the versions are not adjacent to one another, so having the group outlined will not only show you that there are versions, but will show you at a glance which thumbnails are versions of the same picture.
Using these methods have additional advantages. One is for the stack, it would be possible to have an animated cycling through the stack, allowing you have different versions on top. Similarly, you could have an animated "explode" of the stack, where the stacked pictures expand into individual thumbnails, or "collapse", where individual thumbnails combine back into a stack. The outline is important because it would be easy to lose track of the versions when they explode or collapse, so the outline would provide a much-needed visual indicator of exactly where the group of versions came from or went.
Thanks!
We were thinking about the stack, but to speed things up, paint just an empty rect, not the thumbnail itself (you wouldn't see much of it anyway), I'm already working on that :)
The outline is interesting idea, especially with the animations. I very much like the idea of "explosion" and "collapsion", I need to see what can be done with Qt animations framework. Hopefully this will be possible :)
Thanks a lot for these ideas!
Nice work!
First: thank you for really nice feature.
I would also agree that showing a stack, even without thumbnail would be best.
I would also suggest another feature: preview of other thumbnails with mouse wheel. When you hoover with the mouse on the specific photo thumbnail with stack, if you use mouse wheel, I would suggest that thumbnail picture changes - that way I can quickly preview (glance through) list of versions with my mouse. I would reset to "main" version when you move mouse off the thumbnail.
Another question: could we have an option to use the same stack concept for photos with RAW & JPG/PNG image? I think there is a feature request (by many) to show only one photo/thumbnail for combination of RAW+JPG (with same name)...
Some Ideas for "Stacks" of Photos
When I read the comments above, it reminded me of something. BumpTop (since bought by Google) had an interesting multi-touch 3D desktop concept that had grouping, stacking, browsing, exploding, etc. features for items like documents and photos. You might like to take a look at this YouTube video demonstrating BumpTop to see if it gives you any inspiration:
http://www.youtube.com/watch?v=6jhoWsHwU7w
In particular, the "explode" animation (0:40-0:55) is probably something easily achieved in 2D--easier than flicking through the stack (0:25-0:35), anyway.
Since seeing that demo, I have always thought that a grouping feature in DigiKam should work something like that. Maybe by lassoing photos with the mouse pointer and displaying command buttons on roll-over, given the absence of a multi-touch interface. This would be more similar to the older demo from 2006:
http://www.youtube.com/watch?v=M0ODskdEPnQ
Does any of that float your boat?
I must admit, that I am (was)
I must admit, that I am (was) a big fan of BumpTop. The animation from the first video, the "explode", looks really cool. I believe that this could be easily achieved with Qt animations. The only problem I see is the delegate in Qt model/view framework. That's simply a painter and you tell it what to paint and where. Animations (QPropertyAnimation for example) are only usable with QObject, which in the delegate is nothing. The delegate really knows only the boundaries for painting and the data to paint. And there's nothing you can do with animations in this. But I'm still looking into any sane solution for how to use the animations inside delegates. But then, on a higher level, if the delegate would be painting out of its boundaries (which in case of this animation it would have to), it sometimes causes draw issues.
So this is definitely not an easy task. At least the way I see it. But if anyone has any experience with this, please do let me know!
About the grouping feature...again, read above and add two levels of complexity. Also, the "grouping concept" is not there, the versioning has slightly different purpose and aims that way.
I really like the idea of animations, I'm just not sure about the ways of it. Yet ;)
preview of other thumbnails
preview of other thumbnails with mouse wheel.
That's fundamentaly bad idea. Mouse wheel moves the thumbnails view up and down. Now if you would move that view enough so that the mouse pointer ends over the stack, you won't be able to scroll further down, but the stack will be "scrolled". That's a huge usability issue. The space between the thumbnail rectangles is not that big so you could easily scroll just the view. Completely other thing would be using Ctrl+Mouse Wheel or Shift+Mouse Wheel. In other words, such keyboard-mouse wheel combination, that won't affect the expected (current) behaviour with mouse wheel.
But generally the idea of glancing through the stack is interesting, thanks :)
could we have an option to use the same stack concept for photos with RAW & JPG/PNG image?
Currently it does work if you edit a RAW photo, it will be saved as JPG and this stack concept will be used. Additionaly, it will be used for RAW+JPG pair which comes directly from camera. Probably.
wonderful
It is the feature I missed the most in digikam so far :) Now would be nice if when I set my camera to save both .nef and .jpeg, the .jpeg appear automatically as an "edited" version of the .nef.
Similar workflow was intended
Similar workflow was intended and will be implemented ;)
Version numbers
Just a little suggestion: The versions are numbered v1, v2, etc. - could you make that configurable? If you make lots of different versions, you'll get problems with the files not sorting well in filemanagers. Being able to change it to v01, v02 and so on would fix that :)
Forgot to say that it looks
Forgot to say that it looks great so far and it's a critical feature without which many photographers will not consider Digikam usable.
Re: Version numbers
You're right, I can't believe that this thing didn't occur to me, thanks for that :) Will be changed to a number with leading zero. Probably just one, as I don't think anyone will do more than 100 versions of the same image (not the subversions, but the main versions)
I really like DigiKam. There
I really like DigiKam. There are some things I would love to get improved though:
1) I think DigiKam is a little bit slow in some areas.
- First of all it needs really long time to startup, at least on my notebook (Core 2 Duo, 3 GB RAM, 320 GB HD).
- The time to create thumbnails when a new folder is added and thumbnails for all pictures are created. But even when thumbnails are created already it takes some time to show then. Of course not that long, but IMHO it should be a little bit snappier.
2) I'm afraid I don't like the seperation between viewing pictures and working on them. I would like it a lot better to have the working mode in the normal DigiKam window. I pretty sure a lot of people don't agree with this one, so that's just the way I see it.
3)Speeking of thumbnails, why aren't the "shared" with the desktop and other KDE-applications? At least it seems, that when you open a folder in Dolphin with preview-mode thumbnails are created. Then open this folder with KDE's Folderview, thumbnails are created. After that open the folder with Gwenview, thumbnails are created. Then open the folder with DigiKam and - guess what - thumbnails are created. This to me seem a waste of time and discspace.
No hard feelings, these are just some point which I believe could be improved. DigiKam is still a great program.
Regards
First of all it needs really
First of all it needs really long time to startup, at least on my notebook
I've noticed that too sometimes, we'll see what we can do with it.
The time to create thumbnails when a new folder is added and thumbnails for all pictures are created. But even when thumbnails are created already it takes some time to show then.
Creating thumbnails is not an easy task and also depends on your disk speed etc. Showing them is another thing, as I understand it (might be completely wrong), it goes through several layers, from database through some kio into model and from there is painted by delegate to the view. So there must be some slowdown. Although shouldn't be too big. I agree it's a usability issue, I just don't know if anything can be done with it at this point.
why aren't the "shared" with the desktop and other KDE-applications?
DigiKam uses it's own database of thumbnails, that's true. Other KDE apps like Dolphin and Gwenview should have shared thumbnails cache (yes, they're using cache, not database AFAIK). It's true that using one thumbnails source would greatly improve diskspace usage etc, but I'm not sure wheter that's even possible as digiKam requires the database for storing additional data with the thumbs, KDE can live with just a cache. So to make this work, there would have to be a one database implemented into KDE frameworks and that's a request which should be made on different places ;)
No hard feelings, these are just some point which I believe could be improved. DigiKam is still a great program.
No problem, thanks for sharing them with us ;)
Could Digikam first check
Could Digikam first check KDE's caches to see whether there are already some thumbs specific pictures ?
no... it will take a while...
In fact, previously, digiKam used the KDE cache for thumnail (based on freedesktop paper, and shared with other desktop as Gnome). This cache store 2 PNG versions of thumb : 128x128 and 256x256
With 1.x, we use a dedicated BD using PGF image compression algorithm based on wavelets. Only one size of thumb is store : 256x256. Why ? to optimize space disk of course.
Imagine 100.000 picture stored in your computer. Using PNG cache you will use a lots of space disk to cache thumbs.
Now, with PGF DB, we divide this size by 4 or 5. It's not negligeable.
Now, imagine the same result with an very huge collection of image as 500.000 or 1.000.000 items, as some pro photograph has. PGF DB is definitively the best way.
Another argument is to be able to customize the place where to store thumb DB. With PNG stuff, you will always use your home dir...
digiKam
Database
Couldn't Akonadi/Nepomuk be used as a store for the database, if not a shared one?
this will increase complexity...
And what we will do with the whole current code dedicated to manage digiKam DB through sqlite and Mysql ?
And the the entropy of digiKam will become huge. Akonady run properly under M$ windows and Mac OS-X ? I'm not sure...
We need stability. A lots a code have been changed in GoSC2010 branch to add new feature. I'm not convinced by the advantage to use Akonady with digiKam...
thumbnails
Hi,
some time ago we used to store thumbnails according to the freedesktop spec.
But it occurred that this was not sufficient for our needs:
- the specified format, PNG, needs lots of disk space. We now use PGF, which is superior in lots of aspects for this usecase
- for very small files, reading from a database is often preferable
- there were issues with rotation and transparency
But the main argument was:
The freedesktop spec stores by a hash to a filepath. In an environment with removable storage, network storage, duplicate images (backup) and shared access, which we support or plan to support, this approach is simply insufficient.
External editors
Very, very nice work! I also like TheBlackCat's visualization ideas.
Did you think about how to allow for safely using external editors? The simplest scheme that I can come up with would be to:
1) Write protect images on import.
2) Have an "Edit with..." action, that launches an external editor for a new version of the image, and keeps the new version only if it detects that the image was actually edited and saved.
3) Possibly also have a "Make version for external editing" action that will clone an image version into a writable file.
Also, but more complicated; maybe not worth the trouble:
4) Possibly detect (using EXIF data and/or image similarity?) new versions created outside of Digikam in a library folder and offer to associate them with the originals.
You may already have better ideas than those. Thank you so much for the work you have done!
Did you think about how to
Did you think about how to allow for safely using external editors?
Yes, it's quite easy for editors executed from inside digiKam to see wheter or not there were any modifications by doing a hash-check over the image data, but it gets much more harder to find this out when you run digiKam *after* the modifications from external editor. Every image would have to be rehashed to see if there were some edits and if you have let's say 5,000 big images, that would take a hell of a long time.
Using read-only on images is not too good idea for several reasons, though it may work. Anyway, currently there is no sane solution for this, but I'd like to figure out some way. Hopefully we'll get to that during the sprint...
Thanks for replying! I wrote
Thanks for replying! I wrote on your previous post about the "Dad use case". I think that having an option of letting at least the original images be write-protected would help users that have a lower level of computer literacy to feel more secure.
Specifically, edit actions taken from inside Digikam will be safe when your work is in place, but the same actions initiated from the file manager will be destructive, with no warning given to the user. (The user in that case has to make an active "Save as..." choice in order to preserve the original picture). This is dangerous, confusing and scary to the "Dad" user, who does not even fully know the difference between "Digikam" and "Dolphin".
If the files were write protected, the non-savvy user would be forced to do a "Save as...", thus preserving the original picture.
Very cool :) With this years
Very cool :)
With this years gsoc there aren't any features of Picasa left, that I miss at the moment.
The next most important step is improving the UI/UX, imo. Picasa is simple enough to use, that I can give that application to my parents - I currently can't say that about Digikam :(
Thanks for your work and keep rocking :)
flowchart and questions
Your explanations of the versions got quite confusing, it would be much clearer if you drew a flowchart to demonstrate when a new version is created and/or forked.
1- How does the non-destructible editing work, does it just save a list of changes in a text file or does it save all intermediate stages as image files, thereby making each set use up lots of disk space?
2- How does this integrate with Gimp? I almost always use Gimp to remove dust, amongst other things.
xml
1) It's saved in an XML format, and intermediate versions are possibly saved. I think in the end, it will be possible to turn that off completely, so you'll have two files, original + current image.
2) see above
Ditto on GIMP support
I use GIMP for much editing too.
Having "Edited with GIMP" in the editing history (doing more than I imagine would be difficult) and saving pre- and post- GIMP edited versions along with digiKam-edited versions would be great.
gimp do not have versionning mechanism.
There is no Gimp versioning mechanism. How to deal with all Gimp image transformation features...
Non-lossy graphic format for edited versions
Hi there,
Thanks a lot for your great work. It's really appreciated. Only one wish: it would be awesome if the edited version of an image will be stored in a non-lossy graphic format like TIF. Saving an image in the JPEG format would decrease its quality every time.
Regards,
Mike
Lossless format - PGF
Lossless PGF.
Should be a choice of
Should be a choice of lossless TIFF, lossless PNG, lossless PGF, and JPEG (lossy),
the latter for those who really want.
Default? Probably PNG. PGF is not so common yet.
PGF is not common yet, sure,
PGF is not common yet, sure, but it's the best to archive, from a space viewpoint. It's always possible to export PGF to a more common format later, as TIFF or JPG, using Batch Queue Manager
Agreed. Since digiKam handles
Agreed. Since digiKam handles it, and it's the best, it should be the default one, and you can export to other formats of needed.
Hey Martin, this is
Hey Martin, this is impressive indeed, thank you for the share, truly appreciate you took the initiative and posted this for the readers, keep going like this!
Hey Martin, this is
Hey Martin, this is impressive indeed, thank you for the share, truly appreciate you took the initiative and posted this for the readers, keep going like this!
cool feature
Well, I always did these steps 'manually' - great feature to have it automated now. But actually in debian sid it is not included yet. Hope to see that soon. Thanks for a great tool.