Python program to remove duplicates from a list

423



Consider we have a input list [20, 10, 50, 20, 30, 70, 90, 80, 70, 90] which contains duplicate values as well and we want to print the output without any dupliactes. We can do this in a many ways. Let's see few of the ways to do so

Method: 1

lst = [20, 10, 50, 20, 30, 70, 90, 80, 70, 90]
unique = [] 
duplicates = []

for x in lst: 
  if x not in unique: 
    unique.append(x) 
  else :
    duplicates.append(x)

print ("The original list items are : " +  str(lst))
print("Unique list items after removing duplicates are:")
print(unique)
print("Duplicate list items are:")
print(duplicates)

In the above program

  • We are taking a value from the list and checking if it is present in unique list.
  • If it's not present then we are inserting the value to unique list other wise we are inserting it to duplicate list

Results

The original list items are : [20, 10, 50, 20, 30, 70, 90, 80, 70, 90]
Unique list items after removing duplicates are:
[20, 10, 50, 30, 70, 90, 80]
Duplicate list items are:
[20, 70, 90]

Method: 2

lst = [20, 10, 50, 20, 30, 70, 90, 80, 70, 90]

print ("The original list items are : " +  str(lst))

unique = list(set(lst)) 
  
print ("Unique list items after removing duplicates are: ")
print(unique)

In the above program, we are using list(set()) method to return the only unique values present in the list. How ever, the order of the list elements is not maintained using this method

Results

The original list items are : [20, 10, 50, 20, 30, 70, 90, 80, 70, 90]
Unique list items after removing duplicates are: 
[70, 10, 80, 50, 20, 90, 30]

Method: 3

from collections import OrderedDict 
  
lst = [20, 10, 50, 20, 30, 70, 90, 80, 70, 90]
print ("The original list items are : " +  str(lst)) 
  
unique = list(OrderedDict.fromkeys(lst)) 

print ("Unique list items after removing duplicates are: ")
print(unique)

In this method, we are using list(OrderedDict.fromkeys()) method to remove duplicates from the list. This is one of the most popular method and works well with strings as well.

Results

The original list items are : [20, 10, 50, 20, 30, 70, 90, 80, 70, 90]
Unique list items after removing duplicates are: 
[20, 10, 50, 30, 70, 90, 80]