> That functionality really should be changed for the next major python release
Make lists immutable you mean? Like tuple()? Edit: Or force re-evaluation on every function call (could be pretty slow)?
The best practice is to just use a tuple (immutable) instead of a list to begin with. You can always do mylist = list(defaulttuple) inside your function first thing if you really want to work with it as a list.
Also, both pylint and pep8, two python static analyzers, will all warn you about any mutable default arguments you provide. Many Python developers have these hooked into their editor or precommit hooks to catch things like this (among others).
Tuples shouldn't be differentiated from lists primarily in terms of mutable vs immutable. They represent different usages semantically, even though their primary functional difference is in their mutability.
Lists are largely for homogeneous collections, whereas tuples are heterogeneous. When you mix them, you abuse what they signal to the reader.
Python doesn't do much for type safety. Either accept the pitfalls of mutability as a tradeoff, or subclass list to make a safer type.
> Tuples shouldn't be differentiated from lists primarily in terms of mutable vs immutable.
This. In particular, tuples are for collections whose length is fixed and meaningful, and for which you could give names to each field. Lists are for collections whose length doesn't correspond to anything in particular and which you could imagine growing and shrinking.
As an aside, django makes this mistake and confuses people: using tuples where semantically they should be lists, presumably out of a desire for immutability.
A class's __bases__ is a tuple of base classes. The length is not fixed, and there's no special name for [i] other than (i+1)-th; only its ordering is important.
Do believe that the use of a tuple here is a bug, like you do for Django?
The __bases__ can be re-assigned. Assignment to an instance is guaranteed atomic in Python. Some of the list operations are not. Thread-safe guarantees might be another reason for restricting list's mutability.
You're right, at a certain point functionality and other characteristics make all the difference. It's just a generality for higher-level code I suppose.
That's actually super insightful (homogeneous vs heterogeneous). I never made that connection but looking back I see it everywhere in Python code, and even did it unknowingly myself.
Why didn't you use an array (from array module in stdlib) for homogeneous collection then? Python is dynamically typed, which means that, by default, we assume different types everywhere (heterogeneity). If you want to express homogeneity just do so explicitly.
No, spamizbad is correct that wahnfrieden's comment was insightful. List and tuple have totally different semantics; they also happen to differ in mutability which just confuses the issue unfortunately.
I'm not that sure about "totally different" semantics; in Python lists and tuples happen to share quite a lot of public interface. As they both are heterogeneous by default (in Python), I disagree with this being their biggest difference. It's not a case in OCaml or Erlang, where lists and tuples differ significantly in both interface and complexity of basic actions. For example, what is a computational complexity of getting nth element of tuple and list in Python, and why it's exactly the same for both of them? In Erlang, where lists are real lists, getting nth element is O(n) in case of lists and O(1) in case of tuples.
In Python, lists and tuples mean almost exactly the same thing. That's how Python is implemented. Those two things have "totally different" semantics in many other languages, not so in Python. As so, using the one over the other in many cases is purely stylistic matter, not the matter of semantics... unless it's about mutability, of course.
If they're as different as you and grandparent suggest, then please list those differences, and mean in Python. As I said, in OCaml Lists and tuples are completely different types, starting with the fact that tuples are not iterables (!). The same is true in Erlang, where you need to explicitly convert a tuple to list to get all the elements of a tuple. Both list and tuple are iterable in Python, however.
I just see much more similarities between lists and tuples in Python than differences, to the point that I wouldn't mind calling tuples an immutable_list or frozenlist, or something like that.
The quote comes from Guido van Rossum almost verbatim so that may be a good place to start if you want to dig deeper.
As I've also seen, a good rephrasing is that lists have order whereas tuples have structure. Yet another mnemonic I've heard that seems apt is that tuples are appropriate where a named-tuple could also conceivably be used.
The quote is "Tuples are for heterogeneous data, list are for homogeneous data. Tuples are not read-only lists." Guido van Rossum March 12, 2003 in python-dev.
However, don’t treat Guido’s statement as a mandate!
"Yes, taking Guido too seriously can have that effect on people <1.1 wink> The trick is in knowing when Guido is proscribing dogmatic law, and when he is describing his intent when he designed feature(tte)s of Python, or his point of view at this moment, or how he wished people would see it, or reacting to what he had for lunch. He's just human, you know, and no more infallible than the best of us." Thomas Wouters March 12, 2003 in python-dev
For example, consider object.__class__.__bases__. That's a tuple of homogenous data. It's not of a fixed length, and the base class at a given position has no intrinsic name, other than "first", "second", etc.
Do you think the use of tuple here is a bug? Should it be a list instead?
That's crazy-interesting and seems like a very good rule of thumb. I've been using Python for a very long time and had never heard this. Why is it not in the PEP?!
It's a perfectly accurate statement. If a tuple is homogeneous, it's only because all of its slots happen to be for objects of the same type (e.g. a row of a database table all of whose columns are integers, or the arguments of a function which takes 3 integers as input). In contrast, homogeneous lists are the norm (other than mixing None with non-None values). When was the last time you needed a heterogeneous list?
Example of some typical tuples:
(1,'A')
(2,'B')
(3,'C')
In Python, of course lists can be heterogenous, so you could have a list like [1, 'A', 2, 'B', 3, 'C'] but it would be an atypical/un-idiomatic use case. A list of tuples or possibly a dict would make more sense for that.
In a dynamically typed language with a type heirarchy with a common root, it is impossible to distinguish between a heterogenous list whose entities are instances of a variety of specific types and a homogenous list whose members are all members of one of the common supertypes of the specific types of the objects.
With a language that uses duck typing (so that the meaningful "types" of objects are often not their explicit classes but the implicit -- both in definition and in association with particular objects -- interfaces expected by the code which accesses them), this ambiguity is even more pronounced.
This is true, and I'm sure Guido is fully aware of it, but it's also not entirely relevant to what he was saying. Regardless of inheritance, interfaces, and duck typing, the explicit classes of the objects still do matter.
Yes, the list of [1,'A',2,'B',3,'C'] is really after all just a list of [Object, Object, Object, Object, Object, Object]. And the list itself is also just an Object.
But the explicit classes of the elements of the list are different ([int,str,int,str,int,str]), and often that actually does matter for the use case, because int and str have different interfaces, and "adding" ints means something different than "adding" strs for example. It really just depends on what you're going to do with that data, and I think Guido was also talking in terms of the idiomatic use cases for the different data structures.
> This is true, and I'm sure Guido is fully aware of it
Well, yeah. I'd be kind of stunned if he wasn't. I was making a note qualifying the examples that were given in isolation, not questioning the basic premise.
> Regardless of inheritance, interfaces, and duck typing, the explicit classes of the objects still do matter.
Whether or not that's true depends on the use case; the really relevant sense of homogenous or heterogenous that distinguishes the idiomatic use of tuples vs. lists is contextual, and is more about the interfaces that the consuming code expects the members to provide than the explicit classes per se.
There's nothing unidiomatic in Python about using a list for a series of items of different explicit classes that are going to have the same operations (supported by a common interface they all share) performed on them.
No, the list issue is the smaller problem here, imo. The big problem is the example given at the bottom of the page.
def print_now(now=time.time()):
print now
The expectation here is that it will give the current time as the default, but the expectation is broken by the very unexpected behavior of the language. That's the kind of thing you want to avoid if at all possible. I see no reason why anybody would not want the default to evaluate every time - tiny bit of speed seems like a very premature optimization here.
If you're worried about speed, you could make a class/instance variable, and then set the default to that variable. That's what 'default' is actually doing in the current implementation - and it sounds unexpected in the extreme.
It's only unexpected if you didn't learn when Python instantiates default variables and how it passes them. Or put another way, if you treat Python like some other language you learned, apparently not realizing it is a different language.
Now personally I hardly every use default values, but to me, when I see this -
> I see no reason why anybody would not want the default to evaluate every time
I want to create a function that tells me the temperature - but also lets me know the temperature yesterday. Clearly, the best way for this function to work is to have a default value of the current time. Python default values simply won't work here, as the time will be when I first start the program up and not when I call the function tomorrow for the current temperature.
Why would I ever want to have the value from when I first launched the program?! Your argument that "this is how python does it" is not a very good one. Why should I have put it in the method? I want a default value! The reason for having the default value on a function in the first place is to allow me to make functions work with a default value. If this doesn't work, the feature is useless. As I said in the first post, the only reason the feature is usable at all is because, luckily, people nearly always use only numbers or strings for default values.
EDIT: Plus your observation that you rarely use default values is probably for a good reason: they don't really work. Maybe if they worked, you'd use them?
> EDIT: Plus your observation that you rarely use default values is probably for a good reason: they don't really work. Maybe if they worked, you'd use them?
Honestly, no, it's probably because I don't write methods to be generic enough.
Ignoring that,
>argument that "this is how python does it" is not a very good one
Except that if you're writing Python you have to work by Pythons rules, so yes it is.
> The reason for having the default value on a function in the first place is to allow me to make functions work with a default value
Then in python, make sure you are using datatypes that are passed by value, not by reference. That is the core of the problem and it bites everyone at some point in every language because the rules of passing by reference or value are different in a great number of languages.
def foo(name=[]):
In python, is not saying if I didn't pass something make name an empty list, it says 'if name is not defined in this scope, create an empty list.' But with Pythons rules a name[] was created in that scope when the function was defined. It was the programmers mistake for assuming otherwise.
Once again, we are writing Python here, Pythons rules are the only ones that matter. If you are getting unexpected results because you're not following the rules, the issue is yours. Python is not called C#, or Java, or C or Ruby or whatever because it is not C# or Java or whatever.
time.time() is a specific case, but let's say I had very_expensive_function_to_call() as a default value. I would want that function to be called once at parse time, and the value to be kept going forward, instead of calling the function each time.
Both ways have their advantages, python (and the python implementations) just fell one way instead of another.
> Or put another way, if you treat Python like some other language you learned, apparently not realizing it is a different language.
No, it's not "some other language", it's every other language. Why? Because the behavior being described isn't really a default. It's more like a static variable.
Make lists immutable you mean? Like tuple()? Edit: Or force re-evaluation on every function call (could be pretty slow)?
The best practice is to just use a tuple (immutable) instead of a list to begin with. You can always do mylist = list(defaulttuple) inside your function first thing if you really want to work with it as a list.
Also, both pylint and pep8, two python static analyzers, will all warn you about any mutable default arguments you provide. Many Python developers have these hooked into their editor or precommit hooks to catch things like this (among others).