Drop Down MenusCSS Drop Down MenuPure CSS Dropdown Menu

Monday 1 December 2014

Shell Script - eliminate duplicate patterns in a list

Here is a shell script to eliminate duplicate patterns in a list of patterns.

#################################################
# Let the name of the file containing the list
# of patterns be named 'tst.parm'
#
# Name : uniq_patern.ksh
#################################################

#!/usr/bin/ksh

fil=tst.parm
touch new_list
for i in $(cat $fil)
do
grep -vi "$i" $fil > x
var=`echo $i`
if [ `/usr/xpg4/bin/grep -iq "$var" new_list; echo $?`  -ne 0 ]
then
echo "$i" >> new_list
else
##echo "entry already exist"  > /dev/null
echo "entry already exist"
fi
mv x fil
done

#########
# E N D #
#########

#################################################
# Now view the file 'new_list' for a list of
# patterns that are not duplicated
#################################################


Here is the parameter file 'tst.parm' that was passed to the script, the tst.parm file has a list of duplicated patterns


unix$> cat tst.parm
one
ONE
two
three
ONE
two
TWO
one
THREE
one


Once after executing the script uniq_patern.ksh, the result is stored in 'new_list' file.

unix$> cat new_list
one
two
three

No comments:

Post a Comment