Hermanland

I am wondering what is the life cycle for a folder in the context of Digital Asset Management. In line with the bucket and additive backup concept, I see a bucket in a HDD undergoes following life cycle.

• Open - The bucket is available for adding new files.  Folder in other status is always full as a backup bucket and no new files should be added to the folder.
• Meta in Review – Meta data is under review.
• Image in Review – Hash total was generated and the photo or video images are under review.
• Closed - The bucket was closed for any further update and is ready to backup to read-only media e.g. DVD.
• Re-open - The closed bucket is re-opened for updating.

Hash total is a text file kept immediately under a bucket folder listing all the file name, attribute and checksum.  Hash total is a digital signature to the bucket folder allowing to automatically detect any changes in content in the folder.

There are many software to create the digest text file for multiple files in batch. Following features help to manage to manage a large photo portfolio more effective.

• Selectively update the digest text file just for the changed or new photo only because I dont want to visually verify the other old photos again because they are too many.
• Able to perform digest verification on read only media e.g. DVD.
• Portable application so that I can perform the digest verification any where any time.
• Consider file modified attribute and file modified datetime.
• Keep should-be checksum, file attributes in digest file and show it along with actual value in the verification result.

When I have more than 20,000 photo in my collection, I start to wonder how I can ensure all of them are still being kept in the storage media properly.  All media e.g. hard disk or DVD will be aging and I may find some photos cannot be displayed properly when I open them some day in the future.  This will happen although no one can tell exactly when.  Regular data backup will not help because backing up rubbish gets rubbish.  My solution is to have the computer automatically create a digest text file containing a list of order pair of file name and checksum for the photos in my collection bucket.  And then, I visually verify these photos.  It is tricky.  Visually verifying photos and then creating digest is still not the safest.  Then, I can have the computer to automatically verify the photos regularly.  Whenever I need to copy or move the photos across partition, I also use a special tool with checksum verification to do so.

What will happen when 2 image files with different keywords are brought together to form a version set? In Idimager, the labels are merged into the main version that can have label assigned while the other version cannot have. For example, the image A has 2 object label apple and fruit and the image B has 2 object label banana and fruit.  The object label item count is then 2.  The apple label item count is 1 and the banana label item count is 1 and the fruit label item count is 2.  After the 2 images forming a version set with image A as main version, the object label item count and fruit label item count are both decreased to 1.  The apple label item count and banana label item count both remain unchanged.  The label merging mechanism do not apply to auto label e.g. import session and that is why I found the auto label item count always decreasing after versioning as stated in my previous post.

After I added the meta data for my image file, where is the data kept? It may just be kept in the database, the XMP sidecar file or embedded in IPTC block or XMP block within the image file. Before I adapt a new tool to maintain meta data, I need to find out from where the tool will import the meta and to where the tool will keep the meta.  Adobe PS family software tends to use XMP sidecar for proprietary raw and even jpg.  For dng file, they keep the meta in the embedded XMP block. Because I prefer to keep the meta within the image file, I use PS Bridge purely as a media browser and avoid even doing any rating there.  I also turn on the ‘merge XMP’ option in file handling in Image Ingestor so that the batch meta will be inserted to the XMP block within the image file. From the product web page, I found quote “A little problem that this introduced is that there’s no way to prevent generation of XMP sidecars. This will be fixed.” Therefore, I need to manual delete the XMP sidecar files.  After ingestion, I keep my practice to maintain the meta always from Idimager because the tool maps its XMP based catalog to the EXIF, IPTC and XMP embedded block within the image file for maximum compatibility.

Wikka wiki 1.2 is available. The major enhancements to me are the support of table wiki markup and the nice looking ‘Light’ template. Same as previous version, I found the new version also stores the UTF-8 encoded non ascii characters as numeric reference e.g.  & # 915; & # 917; & # 925; & # 917; & # 931; & # 921; & # 931;
I need to apply the hack as I did for previous version to correct it. However, I found the search function to the non ascii characters did not work after the patch but there was no such problem with previous version with the same patch. Also, I just found there is a hack that may correct the search function but there seems to have some side effort but I have not tried this hack yet.

I expected the “import session” label under “Auto Catalog” label could keep me a record of how many image files I had imported to the Idimager database. However, I found a image file disappeared from the “import session” label after the file was made as a version e.g. album version of other existing image file. The item count next to the “import session” label also decreased accordingly.  The “import session” label will be more useful if its item count and pointers to image file not affected by versioning. Similar logic may also be useful for “download session” label.

With the observation above, I found I need to manually add the item count for versioned image files to find out how many image files that were originally imported and downloaded to the Idimager database. However, this method cannot be used per import session or download session.

Furthermore, for the item count in Version Place Holders, I found the item count of Versioned Images is not equal to the sum of the rest of the version place holders. The Versioned Images gave the number of version set which may be less than the sum of the rest of the version place holders because a version set may hold more than 1 version. Therefore, I should add back the rest of the version place holders to the import session to get back the number of image files which were originally imported to Idimager database.

For example, in the Idimager media explorer screenshot below, album display (11) + web display (175) + duplicate1 (85) + duplicate2 (8) = (279) which is not equal to versioned image (270).  Catalog  folders (15,205) = import sessions (14,926) + (279) which agrees with the WinDirStat screenshot result jpg (14,893) + avi (301) + mov (6) + mpg (1) + 3gp (4)

Idimager media explorer

Win Dir Statistic item count

I were suspecting there were some image files in the folder which were not known to the IDimager database yet.  I performed a “Search image files on hard drive” with “Only return uncataloged files” box checked.  However, I found the search result also included the “Uncataloged” labeled image files making the result less useful for my purpose.  A “Uncataloged” image file may still be known to the IDimager database.  For example, I imported (not downloaded) the image files to IDimager.  These image files were surely known to the database because they were referenced in the “Import session” label under the “Auto Catalog” label.  Some of these images were automatically labelled as “Uncataloged” during the import session because there were no keyword embedded in these image files yet.  The search function will be more useful if there is another check box to “Only return files not known to database”.

I found the jpg files were renamed to be the wrong date prefix by ImageIngester Pro 3.2.04. These jpg were output from raw+jpg shots but its raw files were renamed with the expected date prefix. Searching the product website showed that the problem was caused by the incorrect macro. Instead of using @datetime as default value from the software, @exif.DateTimeOriginal should be used to get my expected result. The former picks the file modified date in the file properties while the latter picks the photo taken date in exif. There is no difference if the photo were ingested directly from the memory card but it just happened that I re-ingested the jpg from the folder and the jpg file was modified before. For some file type without exif e.g. avi video file, the file modified date is the only option to use. Therefore, the default setting for Image Ingester on using the file modified date is more generic to handle all file types as long as the files are ingested directly from the memory card without any modification before. Image Ingester can access the file modified date faster than the exif date and that will result in a faster ingestion process.

These Lightroom Top 10 Gotcha’s can really save me hours of headaches. Thank you Lightroom Queen:)