How to remove duplicate values from dict?

I'm trying to remove duplicate values in my dict but its not working:

samples_antibiotics_with_duplicates = {'S00541-09': ['Streptomycin', 'Sulfamethoxazole', 'Trimethoprim', 'Spectinomycin', 'Streptomycin', 'Streptomycin', 'Trimethoprim']}
samples_antibiotics = {}
    for key,value in samples_antibiotics_with_duplicates.iteritems():
      if value not in samples_antibiotics.values():
        samples_antibiotics[key] = value
print samples_antibiotics

This prints:

{'S00541-09': ['Streptomycin', 'Sulfamethoxazole', 'Trimethoprim', 'Spectinomycin', 'Streptomycin', 'Streptomycin', 'Trimethoprim']}

You can try this:

samples_antibiotics_with_duplicates = {'S00541-09': ['Streptomycin', 'Sulfamethoxazole', 'Trimethoprim', 'Spectinomycin', 'Streptomycin', 'Streptomycin', 'Trimethoprim']}

new_dict = {a:list(set(b)) for a, b in samples_antibiotics_with_duplicatates.items()}

Delete and Clean Up Duplicate Photos, Documents, MP3s, Videos, Emails and More!

If you don't care about retaining original order then set(my_list) will remove all duplicates.

If you want to retain original order then list(OrderedDict.fromkeys(my_list))

Python – Remove duplicate values in dictionary Sometimes, while working with Python dictionaries, we can have problem in which we need to perform the removal of all the duplicate values of dictionary, and we are not concerned if any key get removed in the process.

The below dict comprehension will create a new dict from the original one without any duplicate values:

samples_antibiotics = {k: list(set(v)) for k, v in samples_antibiotics_with_duplicates.items()}

The set version of a list (or any container) does not contain any duplicates since sets do not allow any (that is why they require hashable items as do dicts).

As @CoryKramer says in the comments, the solution given here will not (generally speaking) preserve the order of the items in the values-list. If that is important to you, you would have to go with something else.

3. The below dict comprehension will create a new dict from the original one without any duplicate values: samples_antibiotics = {k: list(set(v)) for k, v in samples_antibiotics_with_duplicates.items()} The set version of a list (or any container) does not contain any duplicates since sets do not allow any (that is why they require hashable items as do dicts).

There are better ways to do this as seen in the other posts. But to retain as much of your original code as possible while explaining why it doesn't work use this instead:

samples_antibiotics_with_duplicates = {'S00541-09': ['Streptomycin', 'Sulfamethoxazole', 'Trimethoprim', 'Spectinomycin', 'Streptomycin', 'Streptomycin', 'Trimethoprim']}
samples_antibiotics = {}
for key,value in samples_antibiotics_with_duplicates.items():
    samples_antibiotics[key] = set(value)
print(samples_antibiotics)

The problem is that you iterate through each key in the dictionary in your for loop (so only the 'S00541-09') and then you check if the value is in the values (which obviously it has to be). What I did was essentially iterate the values within the key itself.

Remove Duplicate Values From A Dictionary Here is a python example that shows how to remove duplicate values from a dictionary. Source: (example.py) d1 = {'a': 1, 'b': 2, 'c': 3, 'd': 4, 'e': 5, 'f': 1} d2 = {} for key, value in d1. items (): if value not in d2. values (): d2 [key] = value print (d1) print (d2) Output:

In that case, you should consider a different data structure. Note that if you know roughly where to find the match, you can give index a hint. For instance, in this snippet, l.index(999_999, 999_990, 1_000_000) is roughly five orders of magnitude faster than straight l.index(999_999), because the former only has to search 10 entries, while the

Given a nested dictionary, the task is to remove duplicate dictionaries from the dictionary. Given below are few methods to complete the given task. Method #1: Using Naive Method

To remove the duplicates from a list, you can make use of the built-in function set(). The specialty of set() method is that it returns distinct elements.

Comments
  • Do you care about maintaining the order of the list as you remove duplicates?
  • No I dont care about the order
  • Unless there's an absolute need to make it a list again, I would omit the list typecasting.
  • you should iterate over .items(), otherwise as written you're iterating over the dictionary's .keys()
  • It needs to be samples_antibiotics_with_duplicates.items()