FlickrMetadataSynchr

New Post: Cannot see photosets

erwyn — Fri, 26 Jun 2009 06:44:23 GMT

Yes, FlickrMetadataSynchr can work with private photos. I do this when testing the tool for example. I cannot reproduce the issue you mentioned. I created a set with just one private picture in it and it shows up correctly in FMS and with the right number of photos displayed.

What kind of Flickr account do you have? This might be a limitation of your account type. I know the free accounts has limits like a max of three sets and no download of pictures in the original format. I would highly recommend having a Pro account on Flickr.

New Post: Cannot see photosets

ianjohnson — Fri, 26 Jun 2009 01:14:47 GMT

It seems to have something to do with uploads marked as private. I uploaded some photos into a new set with the permission set to public, and the set showed up. However it said there was one photo in it when there should have been 3. If a set has all private photos it won't show.

Can FMS work with private photos? Since I am using this as a middle step to sync tags between several computers and keep an online backup of my collection, I am not really interested in having everything public.

New Post: Cannot see photosets

ianjohnson — Thu, 25 Jun 2009 23:38:12 GMT

I created a new Flickr account, uploaded some photos, and then ran the program. It couldn't do anything because I hadn't created a set, so I went back to Flickr and added a couple. FMS wouldn't see them, even if I reauthorized the account. I pointed it to a different account which already had sets and thy showed no problem. I logged it back into the new account, but it still didn't see the sets. It seems like if there are no sets the first time FMS encounters a Flickr account, it will not be able to any that are subsequently added.

New Post: Syncing multiple machines

erwyn — Thu, 25 Jun 2009 09:25:23 GMT

If you set the "Tags sync behavior" to merge in the UI, the procedure you describe should work.

As long as pictures can be matched by the date time at which they were taken, renames of the files should not matter. If you have multiple images that were taken within a second, then matching is more difficult. In that case the title on Flickr should match the filename (without extension).

The directory structure is not important as long as all pictures you want to sync are in one set on the Flickr side and in one folder or subfolders of that folder locally. Make sure you check the option "Include subfolders" in the tool.

Reading the metadata of all pictures from Flickr is not very fast, so you might want to start with a smaller batch of pictures to verify that the procedure works. Say with a 100 pictures.

Let me know when you encounter any issues. Have fun synchronizing!

New Post: Syncing multiple machines

ianjohnson — Thu, 25 Jun 2009 00:05:54 GMT

I think FlickrMetadataSynchr may be what I need, but let me run this scenario by you. All my photos live on a server at home. I have copied them to my laptop, and my wife has copied them to hers. We have both been tagging, some to the same photos, and she has renamed some of the files. I would like the tags I have added and the tags she has added to both find their way to the original versions on the server. Do you see any problems with this:

Upload all of the photos to Flickr from the server. Run FlickrMetadataSynchr on my laptop, run FlickrMetadataSynchr on hers and then run it again on the server. Will this result in all of the tags we have both added being saved to the photos on the server, including the files she renamed on her laptop? Does it matter if the directory structure is different between the 3 computers?

Are there any problems I should look out for?

Created Issue: Refactor code as a result of analysis using NDepend

Erwyn — Wed, 17 Jun 2009 20:49:02 GMT

I've run NDepend on the code and this gives opportunities for refactoring the code to increase its quality.

Updated Wiki: Home

Erwyn — Fri, 05 Jun 2009 10:53:46 GMT

Blog of Erwyn van der Meer

Project Description

FlickrMetadataSynchr is a tool that allows you to synchronize metadata added to images stored on Flickr with the original versions of those images stored locally on your hard drive. If you don't have the original version on your hard drive, the application can download the missing pictures for you. The app can sync the metadata two-way on a picture-by-picture and metadata field-by-field basis.
Relevant metadata on Flickr is:

Title
Description
Tags
Author
GPS location info
Date and time at which the image was taken
Last updated date and time

The synchronization is done two-way. The embedded metadata in XMP, EXIF and IPTC sections in locally stored images can be updated with the metadata of matching pictures on Flickr. And vice versa. See also my blog entry with some background info on the metadata problem.

This is a screenshot of the 1.2.1.0 version of the app:

Status

The project is currently at version 1.2.1.0. This version improves support for users who have a standard Flickr account and not a Pro account. The application runs on both 32 and 64-bit versions of Windows XP and Vista and Windows 7.

Download release v1.2.1.0 here. Source code can be found here.

How-to-install

Make sure you have the .NET Framework 3.0 or 3.5 installed on your Windows PC. If you run Windows Vista or Windows 7 you will already have this installed, so no need to worry about this.
Download the ZIP file with the application.
Unzip it to a folder.
Run the FlickrMetadataSynchr.exe executable.

How-to-use

If you run the application for the first time on a computer, you first have to authorize the application to access your Flickr pictures on your behalf. Click the "Authorize application button" to start the authorization process. Follow the on-screen instructions.
After you have authorized the application or have run the application before, you can connect to Flickr to fetch a list of all your photosets. The application automatically tries to connect when it starts. If this fails, for example when your Internet connection is down or if a firewall is blocking the program, you can try to connect again by using the "Connect to Flickr" button.
When you have successfully connected to Flickr, the application shows your name and Flickr avatar. You can select a photoset that you want to sync from the drop down list. You need to have at least one photoset on Flickr to be able to use the application.
After you have selected a Flickr photoset, you can select a folder on a local drive by using the "Browse for local folder" button.
If you like, you can change the synchronization settings. You can choose if subfolders of the local folder should be included and if pictures that are missing in the local folder should be downloaded from Flickr.
If you are happy with what you have selected, hit the "Sync" button and the synchronization process will start. This process is fully automatic. The app will show you what it is doing in the status area. The status area is the gray area at the bottom of the window. If anything goes wrong during the synchronization, the sync process will stop and the application will show you an error message in the status area.
The synchronization process can be cancelled at any time by hitting the "Cancel Sync" button.
If you want to see more detail about what is going on, select the "Show activity log window" option. This will open another window which shows you a log of all activity.

The application performs the following steps during the synchronization process:

Download metadata from Flickr pictures in the selected Flickr set.
Extract metadata from JPEG images in the selected local folder.
Match pictures into pairs based on Flickr id, date and time taken and/or title and filename.
Merge the metadata two-way (see details below).
Optionally download pictures from the selected Flickr set that do not have a mathing picture in the local folder for users with a Flickr Pro account.

The metadata is merged from both sides. Apart from tags, if both sides contain data for a metadata field, the most recently updated side wins for that field. For tag synchronization, you can choose what behavior you want: combine tags (i.e., do not delete tags, just add them), most recently updated side wins (to enable deletion of tags) or leave tags alone. Finally the metadata for the Flickr picture, the local picture or both are updated if the synchronization procedure has determined they should be updated.

The application will embed the Flickr id of a matching picture as ImageUniqueID in the XMP and EXIF metadata of a local picture. This makes it possible to match pictures during future sync operations even if the date time taken and/or the title and filename of the picture have changed or are not unique. Usually upload tools set the title on Flickr equal to the filename. If that is the case, it is recommended to run a sync operation immediately after upload, before any metadata changes, because then the pictures can still be matched based on title/filename.

The application saves and remembers your Flickr account, the local folder, the selected Flickr set and the synchronization options when you restart the application. If you want the app to forget your Flickr account, e.g., when you want to log into another Flickr account or you are using a shared computer, use the "Forget me" button.

The application has a simulation mode in which no real updates take place. Use this mode first, if you are not sure if the automatic synchronization will do the right thing.

The status area shows very limited information during the synchronization process. Full information about what the app is doing and detailed error information if something goes wrong is written to the so called Windows debug output. There are multiple options for viewing this information. It is written to a file called SyncActivity.log in the application directory, it can be viewed in a separate window, and finally it is viewable with the DebugView tool). This is helpful if you want to check the proposed changes in simulation mode before actually applying them.

Download of non-JPEG pictures like PNGs and GIFs is supported, but synching metadata only works for JPEG pictures.

Possible future enhancements

Show the user the match that was made by the application to give the user the opportunity to review for any matching errors.
Let the user deselect matched image pairs that he or she does not want to process.
Let the user be able to overrule the metadata merge on a picture-by-picture and field-by-field basis.

Reuse or contributions

Code adapted from Matthijs Dubbeldam's Vista Flickr Uploader
Code adapted from "TimK"'s WindowSettings class on The Code Project. See also my blog entry about remembering windows position and state in WPF.
Code adapted from Robert A. Wlodarczyk's blog.
Code contributed by Timo Proescholdt.
Code adapted from improvement sent in by Luther Miller.

Dependencies

Flickr account. For downloading pictures you need a Flickr Pro account.
Windows XP, Vista, 7 or Server 2003/2003 R2/2008/2008 R2
.NET Framework 3.0 or 3.5, especially Windows Presentation Foundation for the User Interface and Windows Imaging Component to read and write metadata in image files.
FlickrNet as the API to access Flickr (http://www.codeplex.com/FlickrNet).

Development tools

Visual Studio 2008 SP1
Visual Studio Team Explorer
Expression Blend 2.0

Updated Release: v1.2.1.0 Bugfix (Jun 05, 2009)

erwyn — Fri, 05 Jun 2009 10:48:51 GMT

This release has no new features compared to v1.2.0.0.

It only fixes one bug in v1.1.0.0-v1.2.0.0 that only seems to occurs on Windows 7. These older versions generate an ImageUniqueId/System.Image.ImageId metadata value that is invalid according the XMP spec. It should be the hexadecimal representation of a 128-bit number, which amounts to 32 characters in the range 0-9 or A-F.

The old format is FlickrId=<FlickrId>. The new version uses FF4649<padding><FlickrId>. This string will be exactly 32 characters long, which is achieved by padding with 0 characters. An example of a generated ImageUniqueId in the new format is FF464900000000000000003595605328. In the old format this was FlickrId=3595605328. The prefix FF4649 is just a way to distinguish these ImageUniqueIds from others, for example those generated by cameras.

If the metadata of a local picture needs updating and already contains a FlickrId in the old form, it will be converted and the ImageUniqueId will be updated.

This issue is tracked in work item Application crash due to invalid ImageUniqueId on Win7.

Released: v1.2.1.0 Bugfix (Jun 05, 2009)

Fri, 05 Jun 2009 10:48:51 GMT

Commented Issue: Application crash due to invalid ImageUniqueId on Win7

Erwyn — Thu, 04 Jun 2009 21:05:19 GMT

On Windows 7 the application will crash if it tries to set the System.Image.ImageId property (ImageUniqueId) to a value of the form FlickrId=<number>.

This is because this value is incorrect according to the spec. The Windows Imaging Component (WIC) version 4.0 is probably more strict than WIC 3.0 in this respect. WIC 4.0 ships with Windows 7. WIC 3.0 ships with Vista and .NET 3.0/3.5.
Comments: ** Comment from web user: Erwyn **

The app now embeds the FlickrId in an ImageUniqueId of the form FF4649<padding><FlickrId>. This string is exactly 32 characters long, which is achieved by padding with 0 characters.

If the metadata of a local picture needs updating and already contains a FlickrId in the old form, it will be converted and the ImageUniqueId will be updated.

Closed Task: Increase assembly version to 1.2.1.0

Erwyn — Thu, 04 Jun 2009 20:56:48 GMT

Increase the assembly version to 1.2.1.0.

Closed Issue: Application crash due to invalid ImageUniqueId on Win7

Erwyn — Thu, 04 Jun 2009 20:56:37 GMT

Commented Issue: Application crash due to invalid ImageUniqueId on Win7

Erwyn — Thu, 04 Jun 2009 20:56:34 GMT

Commented Task: Increase assembly version to 1.2.1.0

Erwyn — Thu, 04 Jun 2009 20:56:22 GMT

Increase the assembly version to 1.2.1.0.
Comments: Fixed in changeset 27824.

Source code checked in, #27824

Erwyn — Thu, 04 Jun 2009 20:54:05 GMT

Increased version number to 1.2.1.0. Fixed bug with ImageUniqueId that only occurs on Windows 7.

Created Issue: Application crash due to invalid ImageUniqueId on Win7

Erwyn — Thu, 04 Jun 2009 19:59:37 GMT

Created Task: Increase assembly version to 1.2.1.0

Erwyn — Thu, 04 Jun 2009 19:54:24 GMT

Increase the assembly version to 1.2.1.0.

Updated Release: v1.2.0.0 Feature Release (Apr 04, 2009)

erwyn — Sat, 04 Apr 2009 20:31:23 GMT

Minor feature release. The following improvements were made in v1.2.0 over the v1.1.0 release:

Added robustness for users who don't have a Flickr Pro account. In that case downloading pictures is not supported, because the original pictures are not available on Flickr.
Now uses version 2.2.0 of the Flickr.Net library to access the Flickr API.
Fixed a bug that crashed the application after a non-JPEG file was downloaded.
Outputs the application version number to the trace window and the log file.
The licenses for this application and Flickr.Net are now included in the ZIP file with the executable program.

Also includes the improvements that were made in v1.1.0 over the v1.0.1 release:

Enhanced match algorithm to ensure the downloading of all pictures and better matching of pictures with duplicate datetime taken or title.
Adds ImageUniqueId metadata with FlickrID=<id> value to local pictures for unique identification of pictures in future sync operations. Can be turned off in config file.
Save pictures with the title from Flickr as filename when downloading, turning all characters that are not allowed in filenames in dashes or other characters. Adding (1), (2) etc. as a suffix when there are duplicate titles.
Show the number of photos in each set next to each set name in the drop-down list of Flickr photosets.
Handle Windows Imaging Component bugs gracefully. Ignore metadata update exceptions and move on instead of crashing.
Option in config file to turn off author sync.

Released: v1.2.0.0 Feature Release (Apr 04, 2009)

Sat, 04 Apr 2009 20:31:23 GMT

Minor feature release. The following improvements were made in v1.2.0 over the v1.1.0 release:

Added robustness for users who don't have a Flickr Pro account. In that case downloading pictures is not supported, because the original pictures are not available on Flickr.
Now uses version 2.2.0 of the Flickr.Net library to access the Flickr API.
Fixed a bug that crashed the application after a non-JPEG file was downloaded.
Outputs the application version number to the trace window and the log file.
The licenses for this application and Flickr.Net are now included in the ZIP file with the executable program.

Also includes the improvements that were made in v1.1.0 over the v1.0.1 release:

Enhanced match algorithm to ensure the downloading of all pictures and better matching of pictures with duplicate datetime taken or title.
Adds ImageUniqueId metadata with FlickrID=<id> value to local pictures for unique identification of pictures in future sync operations. Can be turned off in config file.
Save pictures with the title from Flickr as filename when downloading, turning all characters that are not allowed in filenames in dashes or other characters. Adding (1), (2) etc. as a suffix when there are duplicate titles.
Show the number of photos in each set next to each set name in the drop-down list of Flickr photosets.
Handle Windows Imaging Component bugs gracefully. Ignore metadata update exceptions and move on instead of crashing.
Option in config file to turn off author sync.

Updated Wiki: Home

Erwyn — Sat, 04 Apr 2009 19:54:07 GMT

Blog of Erwyn van der Meer

Project Description

Title
Description
Tags
Author
GPS location info
Date and time at which the image was taken
Last updated date and time

Status

The project is currently at version 1.2.0.0. This version improves support for users who have a standard Flickr account and not a Pro account. The application runs on both 32 and 64-bit versions of Windows XP and Vista and Windows 7.

Download release v1.2.0.0 here. Source code can be found here.

How-to-install

Make sure you have the .NET Framework 3.0 or 3.5 installed on your Windows PC.
Download the ZIP file with the application.
Unzip it to a folder.
Run the FlickrMetadataSynchr.exe executable.

How-to-use

If you run the application for the first time on a computer, you first have to authorize the application to access your Flickr pictures on your behalf. Click the "Authorize application button" to start the authorization process. Follow the on-screen instructions.
After you have authorized the application or have run the application before, you can connect to Flickr to fetch a list of all your photosets. The application automatically tries to connect when it starts. If this fails, for example when your Internet connection is down or if a firewall is blocking the program, you can try to connect again by using the "Connect to Flickr" button.
When you have successfully connected to Flickr, the application shows your name and Flickr avatar. You can select a photoset that you want to sync from the drop down list. You need to have at least one photoset on Flickr to be able to use the application.
After you have selected a Flickr photoset, you can select a folder on a local drive by using the "Browse for local folder" button.
If you like, you can change the synchronization settings. You can choose if subfolders of the local folder should be included and if pictures that are missing in the local folder should be downloaded from Flickr.
If you are happy with what you have selected, hit the "Sync" button and the synchronization process will start. This process is fully automatic. The app will show you what it is doing in the status area. The status area is the gray area at the bottom of the window. If anything goes wrong during the synchronization, the sync process will stop and the application will show you an error message in the status area.
The synchronization process can be cancelled at any time by hitting the "Cancel Sync" button.
If you want to see more detail about what is going on, select the "Show activity log window" option. This will open another window which shows you a log of all activity.

The application performs the following steps during the synchronization process:

Download metadata from Flickr pictures in the selected Flickr set.
Extract metadata from JPEG images in the selected local folder.
Match pictures into pairs based on Flickr id, date and time taken and/or title and filename.
Merge the metadata two-way (see details below).
Optionally download pictures from the selected Flickr set that do not have a mathing picture in the local folder for users with a Flickr Pro account.

Possible future enhancements

Show the user the match that was made by the application to give the user the opportunity to review for any matching errors.
Let the user deselect matched image pairs that he or she does not want to process.
Let the user be able to overrule the metadata merge on a picture-by-picture and field-by-field basis.

Reuse or contributions

Code adapted from Matthijs Dubbeldam's Vista Flickr Uploader
Code adapted from "TimK"'s WindowSettings class on The Code Project. See also my blog entry about remembering windows position and state in WPF.
Code adapted from Robert A. Wlodarczyk's blog.
Code contributed by Timo Proescholdt.
Code adapted from improvement sent in by Luther Miller.

Dependencies

Flickr account. For downloading pictures you need a Flickr Pro account.
Windows XP, Vista, 7 or Server 2003/2008
.NET Framework 3.0 or 3.5, especially Windows Presentation Foundation for the User Interface and Windows Imaging Component to read and write metadata in image files.
FlickrNet as the API to access Flickr (http://www.codeplex.com/FlickrNet).

Development tools

Visual Studio 2008 SP1
Visual Studio Team Explorer
Expression Blend 2.0