Home > Uncategorized > Picturelife to Smugmug Migration – Deduplicating Videos

Picturelife to Smugmug Migration – Deduplicating Videos

September 6, 2016 Leave a comment Go to comments

So, now that I’ve removed around 15,000 duplicate photos from my varying levels of overlap between backups of my photos, it was time to look to videos.

Googling “video deduplication” is actually much less productive than for “photo deduplication”; there’s very few reviews or results that come back. I ended up testing three products, and give some very quick results/benchmarks below.

Application

Price

Scan Time
(for 80 x 30-60s videos)

Matches found

Comments

Duplicate Video Search 16.0

USD 29.95

10mins

39 dupes in 80 files

  • Tuneable offset and sensitivity
  • Can scan only first N seconds rather than whole video
  • Also skip first M seconds (eg. To skip identical intro sequences on movies)

Teemoon

Free

50 mins

6 dupes in 80 files

  • Gave application error on some file types

Video Comparer 1.06

Eu30 for 1500vids

15 mins

41 dupes in 82 files

  • No hits with Quick mode – recommended Thorough mode
  • Found all dupes in thorough mode
  • Shows timeline view of both

 

    
 

 

 

 

 

 

 

I ran all three on the same set of videos.

  • Teemoon looked promising for free, but churned away for ages, and hardly found any duplicates at the end. I discounted that
  • Duplicate Video Search did pretty well, and had some good options. However, it displays results in a series of rows, with little UI differentiation between the videos that are in groups, so makes it laborious to carefully work through a long list
  • Video Comparer was the leader of the pack. It had the most efficient UI for reviewing many duplicates, and also some great features for more nuanced copies. It also picked up every dupe, whereas DVS missed one or two.

 

Video Comparer in use

 

Below I show a cropped screen capture of the results tab of VC 1.06.

 

There were two stand-out features in Video Comparer ahead of the other products:

  • Colour banding of groups: Although still organised in rows, the pairs of duplicate videos were shown in different colour bands per group. This made it easier to quickly visually follow the duplicates, and select the one to delete
  • Match-banded timeline: If you look at the group of four ‘duplicates’ above, you can see the blue/grey bands on the right, before the keyframe display. This is awesome:
    • The keyframe display shows you much more than a single thumbnail – it assures you that the videos are similar all the way through, and that one isn’t cut (eg. Half the length) in relation to the other
    • The timeline band completes that thought. It shows which parts of the video files matched. Here, you can see:
      • lines 2 and 4 show the full file length in grey, and that part is coloured blue.
      • That blue represents the part of that file that is contained in the files represented by lines 1 & 3
      • Hence, you can see that lines 2 and 4 (the same file; same filename) represent the original video, and lines 1 and 3 represent a split of that video for the front and rear part
      • So, all from this view, I can understand:
        • Which files contain which parts of the full unedited video
        • Which I might want to keep, and which I might want to delete
        • Reassurance that I can delete either the full-length video, or the two parts

It’s great, but at Eu30, it was expensive. I did buy it, because my challenge of comparing 4 copies of 15 years of home movies is a big one, and this tool will save me a heck of a lot of time in doing so.

 

Advertisements
Categories: Uncategorized
  1. No comments yet.
  1. No trackbacks yet.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: