Is there a more efficient way to iterate over a list of dictionaries?
I'm trying to iterate over a list of dictionaries and keeping only those with a year value in their
yearID key. Essentially, the list (
statistics) is baseball statistics and each row (dictionary) are the stats of a player during a year.
This code seems to work just fine (for VERY small lists of dictionaries), but as soon as the size of the list goes over 40 or 50, Thonny crushes:
def filter_by_year(statistics, year, yearid): nlist =  for dicts in statistics: if str(dicts[yearid]) == str(year): nlist.append(dicts) return nlist
Depends what you mean by "efficient." Your code should work fine for large amounts of dictionaries, so I'm gonna assume you mean efficient in terms of code written.
In that case,
nlist can be simplified into a simple list comprehension:
[dicts for dicts in statistics if str(dicts[yearid]) == str(year)]
Is there a more efficient way to iterate over a list of dictionaries , In this article we will discuss different ways to iterate over a dictionary Iterate over the dictionary using for loop over keys | “for in dictionary” Its not an efficient solution because we are iterating over all the keys in Let's iterate over the list using dict.iter() i.e.. ''' Merge two or more dictionaries in python� This is the simplest way to iterate through a dictionary in Python. Just put it directly into a for loop, and you’re done! If you use this approach along with a small trick, then you can process the keys and values of any dictionary. The trick consists of using the indexing operator  with the dictionary and its keys to get access to the values: >>>
Of all the methods presented so far (by prashant rana, Zaid Afzal, alec_a, and Steven Burnap), yours - the original one - is the most efficient. It becomes somewhat 3x faster if you eliminate the unnecessary conversion to the strings:
def filter_by_year(statistics, year, yearid): nlist =  for dicts in statistics: if dicts[yearid] == year: nlist.append(dicts) return nlist
Different ways to Iterate / Loop over a Dictionary in Python , The keys can be any object with __hash__() and __eq__() methods. Dictionaries map keys to values and store them in an array or collection. One of the most useful ways to iterate through a dictionary in Python is by using .items() , which� To iterate through a dictionary in Python, there are four main approaches you can use: create a for loop, use items() to iterate through a dictionary’s key-value pairs, use keys() to iterate through a dictionary’s keys, or use values() to iterate through a dictionary’s values.
I have no idea if it's more efficient, but a list comprehension is cleaner code:
return [dicts for dicts in statistics if str(dicts[yearid]) == str(year)]
You could use a generator to only grab year and keep index of dictionary being iterated.
def filter_by_year(statistics, year, yearid): nlist =  for i, yearid_ in enumerate(dict[yearid] for dict in statistics): if str(yearid_) == str(year): nlist.append(statistics[i]) return nlist
Python, I have been unable to get the values out of a list of dictionaries with python. I want to be able to loop through all dictionaries of the list and extract both the key and New comments cannot be posted and votes cannot be cast. Sort by. best but would like to hear feedback for how it can be written to be more user friendly. The xlist contains around 2,000,000 elements, and on top of making the function f more efficient I am wondering whether I can make the list creation faster. – mgiom Nov 11 '18 at 0:09 1 The append loop can be replaced with list_of_dict = [f(x) for x in xlist] but you'll have to time it to see if it's any faster.
filter can also be a good way
def filter_by_year(statistics, year, yearid): nlist = list(filter (lambda dicts: str(dicts[yearid]) == str(year), statistics)) return nlist
Iterate through list of dictionaries and get key and value. : Python, Like others have said, the answer is "it depends". Lists are by far the fastest for adding a bunch of items, but dictionaries are faster for looking them up. Hashsets provide the best lookup performance, and are slightly faster for adding than dictionaries. In general, I use them for different purposes.
In Python, list is a type of container in Data Structures, which is used to store multiple data at the same time. Unlike Sets, the list in Python are ordered and have a definite count. There are multiple ways to iterate over a list in Python. Let’s see all different ways to iterate over a list in Python, and a performance comparison between them.
The technique works, I an iterate as many times as I want over this data structure, and I accomplished the simplicity I wanted. But I'm a bit concerned about performance, as I don't think that calling self[self.d][self.etype][self.idx-1] is very efficient. Maybe there are other improvements to be made as well that I don't see.
This is the most Pythonic way to merge multiple dictionaries into a single one and it works for an arbitrary number of dictionaries. Method 2: Simple Nested Loop. Use a simple nested loop to add each (key, value) pair separately to a newly created dictionary: Create a new, empty dictionary. Go over each dictionary in the list of dictionaries.
- This code will work, and be efficient enough, for 100.000 - 1.000.000 dicionaries if you need.
- Why do you compare string representations of the years instead of comparing the years directly?
dicts[yearid]==year. The conversions to strings surely take a lot of time.