Not quite a 
, but nevertheless …
while read l; do echo "SELECT id, email FROM tblName WHERE email='$l' ORDER by id DESC LIMIT 1;"|mysql -u root dbName; done < /tmp/dupes3.txt |grep -v 'email'|cut -f1|sed 's/^\(.*\)$/DELETE FROM tblName WHERE id=\1;/g' > /tmp/deletedupes3.sql
Good one 🙂
As per my understanding,
1. dupes.txt with hundreds of records,
2. Split that file into dupes1.txt, dupes2.txt dupesN.txt
3. run above script in multiple system
4. Combine the files generated by each system and run to remove duplicates in DB.
but how the dupes3.txt(or dupes.txt) populated?
echo SELECT email from tblName group by email having count(email) > 1 | mysql -uroot dbName | grep -v email > dupes.txt
correct me if i am wrong.
Aravinda,
Bang on! youre right on all the points 🙂