Remove iTunes Dupes with One Command on QNAP or MacOS
Published on 15 Mar 2009 at 5:01 pm.
58 Comments.
Filed under Mac, QNAP, Tech.
Edit 2009-03-16: After using this to delete dupes on my QNAP I found while comparing the backup file to my library that there were a few (3 out of over 100) cases where somehow the original file did not exist. For example Kanye’s song “I Wonder” only existed in my library as “I Wonder 2.m4a” Again here it seems most of the problems are with music purchased from iTunes, I suggest not manually moving any purchased music. In any case the information here will work to remove dupes on your QNAP or Mac but a much better solution would be a script that checks to see if an original version of the file exists before deleting. I’ll be out of town for the next few days, but when I get back I’ll whip up a Perl script to do just that.
So I followed the instructions for moving my iTunes library to my QNAP 509. Against my better judgement I did check the box to allow iTunes to organize my music. I think in the long run it will be for the best, but in the short run what I got was a bunch of dupes. Mostly all my bought music that I had already copied over to the QNAP.
Looking closer and after seeing this on a buddy’s Mac once already, I found that all the duplicate files, even ones in their own directories ended in ” 1.mp3″ or ” 1.m4a”. So a quick find command, showed me all my dupes.
Edit 2009-02-16: If you do a library import more than once on the same shared directory the file names may end in 2.mp3 2.m4a or 3.mp3 etc instead of 1. You can alter your find command to include multiple numbers like this: find . -name “* [1-3].m*” -print However be very careful once you go past 1, because lots of songs end in ‘part 2′ ’side 2′ etc. I also added a post on removing the duplicates in iTunes after removing them from the file system.
find . -name “* 1.m*” -print
Now that I knew I could get at them I just had to brush up on my awk and xargs to finish the job. The reason I needed awk and xargs instead of just issuing -exec rm in the find command is because the filenames have spaces in them. The awk and xargs on the QNAP are very stripped down so it took a while to figure that one out but I finally got the job done with this command:
QNAP Find and Delete Dupes
Make sure you paste these commands all on one line!
Please note: Some systems don’t properly copy the double and single quotes when you copy these commands. I suggest you copy them into a plain text document first and replace all the single and double quotes before executing them in a shell. I tried pre tags and different fonts but the problem persisted.
find . -name “* 1.m*” -print | awk ‘{print “\42″ $n “\42″}’ | grep -vi “Track 1″ | xargs /bin/rm
I really wanted a -i on the end there because I do have some files called “Track 1.mp3″ when they couldn’t get CDDB info. However the QNAP xargs doesn’t support the -o option so the -i was being ignored and nothing was happening. To get around that I added the grep -vi “Track 1″ to preserve those files and let the /bin/rm run it’s course. Make sure you look at the output without the | xargs /bin/rm on the end before you add it back to delete. And always back up the files before you issue the command. An easy way to backup the files you will be delete is to just use the same find command and send it to tar.
find . -name “* 1.m*” -print | awk ‘{print “\42″ $n “\42″}’ | xargs tar -cvf save_before_delete.tar
MacOS Find and Delete Dupes
For MacOS the xargs is better so the solution was much easier.
Get to your iTunes Music folder usually ~/Music/iTunes/iTunes Music
Backup the Files first:
find . -name “* 1.m*” -print | awk ‘{print “\42″ $0 “\42″}’ | xargs tar -cvf save_before_delete.tar
Watch what it tars up and you can examine the tar file with:
tar -tvf save_before_delete.tar | more
Hit spacebar to view the list of files page by page.
If you don’t see any files in there that you don’t want to delete you are ready to move on. If there are files in there that you want to save, you can work out grep -v commands to exclude them, or you can just save them and put them back after the mass delete.
Ready for Delete:
find . -name “* 1.m*” -print | awk ‘{print “\42″ $0 “\42″}’ | xargs -o /bin/rm -i
The -i on the end is going to make you confirm every file. So if you have a lot, you may want to look at the list and use grep -vi to remove files you don’t want to delete ahead of time, and then you can issue the command without the -i.
Once you have examined your library and made sure nothing was deleted that you didn’t intend you are free to delete the backup tar file.
Related items
58 Comments to ‘Remove iTunes Dupes with One Command on QNAP or MacOS’:
Leave a Reply
Jason Larson » Removing Files from iTunes Library after a manual delete on 16 Mar 2009 at 10:42 am: 1
[...] « Remove iTunes Dupes with One Command on QNAP or MacOS [...]
Freeman on 16 Jun 2010 at 3:19 am: 2
Добрый день! thomas@sotkashop.ru” rel=”nofollow”>……
С уважением,…
LEWIS on 29 Jul 2011 at 4:57 am: 3
Order@Cheap.Abana” rel=”nofollow”>..…
Buynow it…
SALVADOR on 29 Jul 2011 at 9:12 am: 4
Buy@Abilify.Online” rel=”nofollow”>..…
Buygeneric meds…
RONNIE on 29 Jul 2011 at 9:47 am: 5
Buy@Cheap.Abilify” rel=”nofollow”>..…
Buynow it…
EDUARDO on 29 Jul 2011 at 7:51 pm: 6
Purchase@Generic.Abilify.5mg.10mg.15mg.20mg.30mg” rel=”nofollow”>.…
Buynow it…
RUSSELL on 30 Jul 2011 at 11:47 pm: 7
Order@Generic.Acai” rel=”nofollow”>..…
Buywithout prescription…
PERRY on 3 Aug 2011 at 12:10 am: 8
Purchase@Cheap.Abilify” rel=”nofollow”>……
Buywithout prescription…
TERRENCE on 3 Aug 2011 at 1:39 am: 9
Get@Abilify.Online” rel=”nofollow”>..…
Buyit now…
GUY on 3 Aug 2011 at 1:58 pm: 10
Buy@Cheap.Acai” rel=”nofollow”>.…
Buywithout prescription…
JEFFREY on 3 Aug 2011 at 5:12 pm: 11
Order@Acai.Online” rel=”nofollow”>……
Buynow it…
DONALD on 3 Aug 2011 at 5:20 pm: 12
Order@Acai.Online” rel=”nofollow”>……
Buyit now…
WILLIE on 4 Aug 2011 at 12:12 am: 13
Purchase@Discount.Acai” rel=”nofollow”>……
Buygeneric drugs…
PERRY on 4 Aug 2011 at 2:58 am: 14
Cheap@Acai.Online” rel=”nofollow”>……
Buynow…
SAM on 5 Aug 2011 at 7:00 am: 15
Purchase@Discount.Energy.Boost” rel=”nofollow”>.…
Buynow it…
RUBEN on 5 Aug 2011 at 11:31 am: 16
Buy@Cheap.Accupril” rel=”nofollow”>.…
Buygeneric meds…
BILLY on 5 Aug 2011 at 4:57 pm: 17
Purchase@Accupril.Online” rel=”nofollow”>.…
Buygeneric drugs…
CARL on 7 Aug 2011 at 3:50 am: 18
Buy@Cheap.Aciphex” rel=”nofollow”>.…
Buygeneric meds…
ROY on 10 Aug 2011 at 3:34 pm: 19
Cheap@Actonel.Without.Prescription” rel=”nofollow”>……
Buynow it mou…
CLIFTON on 10 Aug 2011 at 8:25 pm: 20
Buy@Cheap.Actoplus.Met” rel=”nofollow”>..…
Buygeneric meds qmo…
RODNEY on 12 Aug 2011 at 11:52 am: 21
Order@Cheap.Adalat” rel=”nofollow”>..…
Buygeneric drugs lzt…
LAWRENCE on 12 Aug 2011 at 6:42 pm: 22
Order@Discount.Coral.Calcium” rel=”nofollow”>..…
Buywithout prescription dyo…
JEREMIAH on 13 Aug 2011 at 12:21 pm: 23
Buy@Generic.Abilify.Without.Prescription” rel=”nofollow”>..…
Buynow it tly…
MARION on 14 Aug 2011 at 10:53 am: 24
Buy@Generic.Acai” rel=”nofollow”>.…
Buygeneric drugs mby…
JERRY on 14 Aug 2011 at 6:27 pm: 25
Cheap@Generic.Acai” rel=”nofollow”>……
Buydrugs without prescription vrg…
JOEL on 15 Aug 2011 at 10:18 am: 26
Order@Cheap.Energy.Boost” rel=”nofollow”>..…
Buyno prescription wkp…
MARION on 15 Aug 2011 at 11:16 am: 27
Order@Discount.Energy.Boost” rel=”nofollow”>..…
Buyno prescription gdt…
MITCHELL on 18 Aug 2011 at 3:16 am: 28
Order@Cheap.Aciphex” rel=”nofollow”>..…
Buydrugs without prescription faj…
BRUCE on 18 Aug 2011 at 8:10 am: 29
Purchase@Discount.Aciphex” rel=”nofollow”>..…
Buyno prescription jui…
MILTON on 21 Aug 2011 at 5:01 am: 30
Buy@Actonel.Online” rel=”nofollow”>..…
Buydrugs without prescription hak…
BRYAN on 21 Aug 2011 at 1:15 pm: 31
Order@Cheap.Actonel” rel=”nofollow”>.…
Buygeneric drugs nih…
CARLOS on 21 Aug 2011 at 3:56 pm: 32
Purchase@Actonel.Online” rel=”nofollow”>……
Buygeneric drugs vbr…
VINCENT on 21 Aug 2011 at 5:16 pm: 33
Purchase@Cheap.Actonel” rel=”nofollow”>.…
Buydrugs without prescription xab…
TRAVIS on 23 Aug 2011 at 12:09 am: 34
Order@Discount.Advair” rel=”nofollow”>..…
Buyno prescription tao…
BILLY on 25 Aug 2011 at 3:34 am: 35
Get@Aggrenox.Online” rel=”nofollow”>……
Buynow fjl…
RENE on 25 Aug 2011 at 10:13 am: 36
Order@Cheap.Albenza” rel=”nofollow”>……
Buynow it sxk…
BARRY on 28 Aug 2011 at 12:36 am: 37
risperdal@vs.abilify” rel=”nofollow”>.…
Buygeneric drugs…
RICHARD on 28 Aug 2011 at 4:16 pm: 38
allegra@generic.purchase” rel=”nofollow”>……
Buynow…
gerard on 16 Sep 2011 at 12:22 pm: 39
……
Search music now US Charts…
JEFFERY on 17 Oct 2011 at 12:40 am: 40
sleep aids and half-life…
Buy_it now…
MARION on 18 Oct 2011 at 6:17 am: 41
adhd drugs causing hallucinations…
Buy_generic meds…
RONNIE on 20 Oct 2011 at 6:18 am: 42
number of obese in this generation…
Buy_generic meds…
GREG on 20 Oct 2011 at 7:57 am: 43
aspirin pain relief dog…
Buy_drugs without prescription…
ALEXANDER on 21 Oct 2011 at 2:49 am: 44
drug alcohol prevention resilience…
Buy_it now…
IVAN on 25 Oct 2011 at 4:49 pm: 45
research being done to treat hiv…
Buy_generic pills…
ANDRE on 26 Oct 2011 at 9:10 pm: 46
serotonin natural sources…
Buy_generic meds…
DERRICK on 27 Oct 2011 at 6:48 pm: 47
nursing drug study website…
Buy_drugs without prescription…
KYLE on 1 Nov 2011 at 5:26 am: 48
stop smoking phlegm…
Buy_drugs without prescription…
BYRON on 3 Nov 2011 at 11:32 am: 49
what foods anorexia eat…
Buy_drugs without prescription…
CLIFTON on 5 Nov 2011 at 9:12 pm: 50
nexium pepsin ac…
Buy_generic drugs…
MICHAEL on 7 Nov 2011 at 8:14 am: 51
will the recession become a depression…
Buy_without prescription…
JEREMIAH on 7 Nov 2011 at 1:12 pm: 52
teen girl 16 18.6 bmi…
Buy_generic drugs…
HARVEY on 8 Nov 2011 at 8:01 am: 53
acne scar and pit lamp…
Buy_generic drugs…
NATHAN on 9 Nov 2011 at 9:26 am: 54
synephrine hcl diet pills…
Buy_generic pills…
FRED on 10 Nov 2011 at 8:25 pm: 55
terry abbot…
Buy_generic drugs…
AARON on 13 Nov 2011 at 6:46 am: 56
benicar hct and norvasc…
Buy_it now…
ARTURO on 13 Nov 2011 at 1:25 pm: 57
macon georgia drug court…
Buy_without prescription…
JOHN on 9 Dec 2011 at 11:29 am: 58
canadian@pharmacy.valtrex” rel=”nofollow”>……
Buygeneric meds…