python's timeit is not initialising list in setup code

python tutorial
python download
python syntax
what is python used for
introduction to python
python online
python from scratch
python snake

I'm timing element removal from a python set vs. a list. My timeit code for the list is raising a ValueError: ... x not in list, but only when I run more than one iteration with timeit !??

It appears that for the list, the variable created in the setup code is re-used in the subsequent iteration (as if the setup code is not run the second time??).

Here's my code:

In [1]: import timeit

In [2]: timeit.timeit(stmt='a.discard(10**5)', setup='a = set(range(10**6))', number=100000)
Out[2]: 0.02187999989837408

In [3]: timeit.timeit(stmt='a.remove(10**5)', setup='a = list(range(10**6))', number=1)
Out[3]: 0.023419374600052834

In [4]: timeit.timeit(stmt='a.remove(10**5)', setup='a = list(range(10**6))', number=2)
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
...
ValueError: list.remove(x): x not in list

What's going on??

They key point is that setup is only performed once even when number is >1 (hence the ValueError when you try call to list.remove with a unique value that has already been removed from the list). From the docs (emphasis mine):

Time number executions of the main statement. This executes the setup statement once, and then returns the time it takes to execute the main statement a number of times, measured in seconds as a float. The argument is the number of times through the loop, defaulting to one million. The main statement, the setup statement and the timer function to be used are passed to the constructor.

So, if you want to perform multiple timings of a code snippet like this (say to get more accurate timings) then you have to use number=1, but you could use the repeat argument with timeit.repeat() instead:

>>> timeit.repeat(stmt='a.remove(10**5)', setup='a = list(range(10**6))', number=1, repeat=2)
[0.002321417909115553, 0.0023121219128370285]

Python (programming language), Take-Away Skills: This course is a great introduction to both fundamental programming concepts and the Python programming language. By the end, you'll be  99% Match on Learn Programming Python. Start searching with Visymo.com.

In [4]: timeit.timeit(stmt='a.remove(10**5)', setup='a = list(range(10**6))', number=2)

a.remove(10**5) is executed twice but 10**5 can only be removed once. The setup is only called ONCE so you always work on the same list or set.

This is equivalent to

a = list(range(10**6))
a.remove(10**5)  # works
a.remove(10**5)  # fails with ValueError: list.remove(x): x not in list

This doesn't fail for the set since set discard and list remove behave differently. If you use set remove method you will also end up getting an error (KeyError)

a = set(range(10**6))
a.discard(10**5)  # works
a.discard(10**5)  # works

but

a = set(range(10**6))
a.remove(10**5)  # works
a.remove(10**5)  # KeyError 10**5

Python Tutorial, The class includes written materials, lecture videos, and lots of code exercises to practice Python coding. These materials are used within Google  Looking for where to learn python? Get your questions answered. Discover where to learn python and related content.

The discard() operation on sets does nothing if the element being discarded is not in the set. The remove() operation on lists, however, raises a ValueError when the element does not exist. When you initialize your list with range(10**6), the value 10**5 only appears once; in the set this isn't a problem, but in the list it can only be removed once and will give an error on future attempts to remove the same value.

Learn Python 2, learnpython.org is a free interactive Python tutorial for people who want to learn Python, fast. Get Where To Learn Python. Get Instant Quality Results Now!

Google's Python Class | Python Education, Python is a general-purpose interpreted, interactive, object-oriented, and high-​level programming language. It was created by Guido van Rossum during 1985-​  Learn Python Like a Pro. From The Basics All The Way to Creating your own Apps and Games! Join Over 50 Million Students Already Learning Online With Udemy

Learn Python, Learn Programming for Everybody (Getting Started with Python) from University of Michigan. This course aims to teach everyone the basics of programming  Python knows the usual control flow statements that other languages speak — if, for, while and range — with some of its own twists, of course. More control flow tools in Python 3. Python is a programming language that lets you work quickly and integrate systems more effectively.

Python Tutorial, Python for Everybody Specialization. Learn to Program and Analyze Data with Python. Develop programs to gather, clean, analyze, and visualize data. The Python web site provides a Python Package Index (also known as the Cheese Shop, a reference to the Monty Python script of that name). There is also a search page for a number of sources of Python-related information. Failing that, just Google for a phrase including the word ''python'' and you may well get

Comments
  • @Laurent H. Happy to use US/UK spelling with -ize, but I'm Australian and -ise is acceptable spelling here
  • I really apologize to Australian pepole. You have learnt me something, thanks !
  • fwiw, removing from sets is always faster, and especially for long lists, where it's much faster (for 10k elements, 10x to remove first element, 200x for last element).
  • @drevicko Right, that makes sense in terms of time complexity since set.remove is normally an O(1) operation (just calculating a hash) while list.remove is an O(n) operation (potentially examining every item in the whole list)
  • also, removing the first item has a hit since the remaining items need to be shifted in memory afaik (though that's pretty efficient actually)