I'll point out that the walrus operator was actually accepted while Guido was still BDFL (and the vitriol surrounding the decision to include it led directly to him stepping down from the position [1]), so even accepting the fact that it's a poor addition to the language, does not provide support for the statement that "design by committee" has lead to poor language design decisions.
They gave no specific criticisms. This thread was born of a request for specific criticisms. When that happens, I try to operate as though the assumptions laid out in the parents hold for the children. I think this makes sense to do, especially when you appeared to step in as a proxy expanding on the parent's opinion. Even if that wasn't your intention, this is a public thread, and the most relevant place to post things as a response to a sentiment in a thread may not be directly to a person who holds that exact sentiment. If you don't take issue with "design by committee" then you need not be concerned. I don't think you think that, and I think no less of you regardless.
Disagree: the recent changes are things I put to work immediately and in a large fraction of the code. They're not niche and "should have" been added years ago. If anything, I'm thrilled with the work of the "committee," whose judgments are better than the result of any individual. Postgres is the same.
Gone are the days when you invest in a platform like python, and they make crazy decisions that kill the platform's future (e.g. perl5). Ignore small syntax stuff like := and focus on the big stuff.
> Disagree: the recent changes are things I put to work immediately and in a large fraction of the code.
That says nothing about their quality. It just says you like them. If you gave me unhealthy food I'd probably eat it immediately too. Doesn't mean I think it's good for me.
> Ignore small syntax stuff like := and focus on the big stuff.
They're not "small" when you immediately start using them in a "large fraction of your code". And a simple syntax that's easy to understand is practically Python's raison d'être. They added constructs with some pretty darn unexpected meanings into what was supposed to be an accessible language, and you want people to ignore them? I would ignore them in a language like C++ (heck, I would ignore syntax complications in C++ to a large degree), but ignoring features that make Python harder to read? To me that's like putting performance-killing features in C++ and asking people to ignore them. It's not that I can't ignore them—it's that that's not the point.
I simply do not understand how the walrus operator is harder to read. Maybe an example?
my_match = regex.match(foo)
if my_match:
return my_match.groups()
# continues with the now useless my_match in scope
Versus
if my_match := regex.match(foo):
return my_match.groups()
# continues without useless my_match in scope
How is the second one less readable? Have you ever heard of a real world example of a beginner or literally anyone ever actually expressing confusion over this?
The problem isn't that simple use case. Although even in that case, they already had '=' as an assignment operator, and they could've easily kept it like the majority of other languages do instead of introducing an inconsistency.
The more major problem with the walrus operator is more complicated expressions they made legal with it. Like, could you explain to me why making these legal was a good thing?
def foo()
return ...
def bar():
yield ...
while foo() or (w := bar()) < 10:
# w is in-scope here, but possibly nonexistent!
# Even in C++ it would at least *exist*!
print(w)
# The variable is still in-scope here, and still *nonexistent*
# Ditto as above, but even worse outside the loop
print(w := w + 1)
If they just wanted your use case, they could've made only expressions of the form 'if var := val' legal, and maybe the same with 'while', not full-blown assignments in arbitrary expressions, which they had (very wisely) prohibited for decades for the sake of readability. And they would've scoped the variable to the 'if', not made it accessible after the conditional. But nope, they went ahead and just did what '=' does in any language, and to add insult to injury, they didn't even keep the existing syntax when it has exactly the same meaning. And it's not like they even added += and -= and all those along with it (or +:= and -:= because apparently that's their taste) to make it more useful in that direction, if they really felt in-expression assignments were useful, so it's not like you get those benefits either.
While the walrus operator gives a way to see this sort of non-C++ behavior, it's more showing that Python isn't C++ than something special about the operator.
Here's another way to trigger the same NameError, via "global":
import random
def foo():
return random.randrange(2)
def bar():
global w
w = return random.randrange(20)
return w
while foo() or (bar() < 10):
print(w)
For even more Python-is-not-C++-fun:
import re
def parse_str(s):
def m(pattern): # I <3 Perl!
nonlocal _
_ = re.match(pattern, s)
return _ is not None
if m("Name: (.*)$"):
return ("name", _[1])
if m("State: (..) City: (.*)$"):
return ("city", (_[2], _[1]))
if m(r"ZIP: (\d{5})(-(\d{4}))?$"):
return ("zip", _[1] + (_[2] if _[2] else ""))
return ("Unknown", s)
del _ # Remove this line and the function isn't valid Python(!)
for line in (
"Name: Ernest Hemingway",
"State: FL City: Key West",
"ZIP: 33040",
):
print(parse_str(line))
Right, I'm quite well-aware of that, but I'm saying this change has made the situation even worse. If they ensured the variables were scoped and actually initialized it'd have actually been an improvement.
# w is in-scope here, but possibly nonexistent!
# Even in C++ it would at least *exist*!
because I don't see how bringing up C++'s semantics is relevant when Python has long raised an UnboundLocalError for similar circumstances.
If I understand you correctly, you believe Python should have introduced scoping so the "w" would be valid only in the if, elif, and else clauses, and not after the 'if' ends.
This would be similar to how the error object works in the 'except' clause:
>>> try:
... 1/0
... except Exception as err:
... err = "Hello"
...
>>> err
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'err' is not defined
If so, I do not have the experience or insight to say anything meaningful.
In your example, if you leave out the parentheses around w := bar(), you get "SyntaxError: cannot use assignment expressions with operator" which makes me think it's a bug in the interpreter and not intentionally designed to allow it.
I am baffled to learn that it's kept in scope outside of the statement it's assigned, and I agree it would have a negative impact on readability if used outside of the if statement.
> if you leave out the parentheses around w := bar(), you get "SyntaxError: cannot use assignment expressions with operator" which makes me think it's a bug in the interpreter and not intentionally designed to allow it.
No, I'm pretty sure that's intentional. You want the left-hand side of an assignment to be crystal clear, which "foo() or w := bar()" is not. It looks like it's assigning to (foo() or w).
def thing(): return True
if thing() or w:= "ok": # SyntaxError: cannot use assignment expressions with operator
pass
print(w)
. . .
if thing() or (w := "ok"):
pass
print(w) # NameError: name 'w' is not defined
The first error makes me think your concern (that w is conditionally undefined) was anticipated and supposed to be guarded against with the SyntaxError. I believe the fact you can bypass it with parentheses is a bug and not an intentional design decision.
Oh I see, you're looking at it from that angle. But no, it's intentional. Check out PEP 572 [1]:
> The motivation for this special case is twofold. First, it allows us to conveniently capture a "witness" for an any() expression, or a counterexample for all(), for example:
if any((comment := line).startswith('#') for line in lines):
print("First comment:", comment)
else:
print("There are no comments")
I have a hard time believing even the authors (let alone you) could tell me with a straight face that that's easy to read. If they really believe that, I... have questions about their experiences.
Your new example makes me wonder: if I can intentionally conditionally bring variables into existence with the walrus operator, what's the motivation behind the SyntaxError in my statement above? I maintain my belief that the real issue here is, readability aside, if blocks do not implement a new scope, which has always been a problem in the language. The walrus operator just gives you new ways to trip over that problem.
From the PEP:
> An assignment expression does not introduce a new scope. In most cases the scope in which the target will be bound is self-explanatory: it is the current scope. If this scope contains a nonlocal or global declaration for the target, the assignment expression honors that. A lambda (being an explicit, if anonymous, function definition) counts as a scope for this purpose.
I find this particularly strange and inconsistent:
lines = ["1"]
[(comment := line).startswith('#') for line in lines]
print(comment) # 1
[x for x in range(3)]
print(x) # NameError: name 'x' is not defined
I'm saying it's the same reason why (x + y = z) should be illegal even if (x + (y = z)) is legal in any language. It's not specific to Python by any means. The target of an assignment needs to be obvious and not confusing. You don't want x + y to look like it's being assigned to.
There are two aspects I have been thinking about while looking at this: Introduction of non-obvious behavior (foot-guns) and readability. Readability is important, but I have been thinking primarily about the foot-gun bits, and you have been emphasizing the readability bits. I can't really accurately assess readability of something until I encounter it in the wild.
If the precedence was higher then you'd get a situation like
x := 1 if cond else 2
never resulting in x := 2 which is pretty unintuitive.
And you have to realize, even if the precedence works out, nobody is going to remember the full ordering for every language they use. People mostly remember a partial order that they're comfortable with, and the rest they either avoid or look up as needed. Like in C++, I couldn't tell you exactly how (a << b = x ? c : d) groups (though I could make an educated guess), and I don't have any interest in remembering it either.
Ultimately, this isn't about the actual precedence. Even if the precedence was magically "right", it's about readability. It's just not readable to assign to a compound expression, even if the language has perfect precedence.
I know they don't, normally. I really thought that was basically the point of the walrus operator to begin with, that the variable was only in scope for the lifetime of the if statement where it's needed. Huge bummer to find out that's not true.
Scoop in python is normally defined by functions/methods, not blocks. The same happens with for-loos and with-blocks. So this is consistent. And this good, because it can be very useful. The exception here are try/except-blocks, where the fetched error is cleaned up after leaving the except-block, for reasons.
IMO the real abomination was already present in the language, which is that if blocks do not introduce new scope. My IDE protects me from the bugs this could easily introduce when I try to use a variable that may not yet be in scope, but it should be detected before runtime.
I will readily admit that the walrus operator doesn't do what I thought it did and I have no interest in whatever utility it provides as it exists today.
> IMO the real abomination was already present in the language, which is that if blocks do not introduce new scope.
Definitely. You would think if they're going to undermine decades of their own philosophy, they would instead introduce variable declarations and actually help mitigate some bugs in the process.
I don't know how important this is, but I believe it does make it less readable for "outsiders".
As a non-Python programmer it is usually pretty easy for me to correctly guess what a piece of Python code does. (And once in a while I need to take a look at some Python code).
Walrus operator got me. I tried to guess what it did, but even having simple code examples I could not. My guesses were along the lines of binding versus plain assignment, or some such. None of my guesses were even close. I had to google it to find out (of course I could also read the documentation).
IMO the match statement has some very unintuitive behaviour:
match status:
case 404:
return "Not found"
not_found = 404
match status:
case not_found:
return "Not found"
The first checks for equality (`status == 404`) and the second performs an assignment (`not_found = status`).
`not_found` behaving differently from the literal `404` breaks an important principle: “if you see an undocumented constant, you can always name it without changing the code’s meaning” [0].
Actually, I don’t really want the feature. It’s complicated and it doesn’t really fit with the rest of the language (it breaks fundamental rules, as above, and has scoping issues).
Worst of all, though, it’s really just another way to write if-elif-else chains that call `isinstance` - a pattern which Python traditionally discouraged in favour of duck-typing.
Do you not like the idea of pattern matching as a feature or do you not like the implementation details? This kind of seems like another clumsy scoping problem, no?
I would love a good pattern matching feature, but this is not it. And this is a seriously broken design at a fundamental level, not an "implementation detail". I actually have no clue how it's implemented and couldn't care less honestly. I just know it's incredibly dangerous for the user to actually use, and incredibly unintuitive on its face. It's as front-and-center as a design decision could possibly be, I think.
And no, this is not really a scoping issue. Match is literally writing to a variable in one pattern but not the other. A conditional write is just a plain inconsistency.
The sad part is both of these features are stumbling over the fact that Python doesn't have variable declarations/initialization. If they'd only introduced a different syntax for initializations, both of these could have been much clearer.
> I actually have no clue how it's implemented and couldn't care less honestly.
I guess I'm not sure where "design" ends and "implementation" begins? To me, how to handle matching on variables that already exists is both, because "pattern matching and destructuring" are the features and how that must work in the context of the actual language is "implementation". It being written in a design doc and having real world consequences in the resulting code doesn't make it not part of the implementation.
Instead of quibbling over terms, I was much more interested in whether you like the idea of pattern matching.
I think not liking the final form a feature takes in the language is fundamentally different from wholesale disliking the direction the language design is going.
Design is the thing the client sees, implementation is the stuff they don't see. In this case the user is the one using match expressions. And they're seeing variables mutate inconsistently. It's practically impossible for a user not to see this, even if they wanted to. Calling that an implementation detail is like calling your car's steering wheel an implementation detail.
But I mean, you can call it that if you prefer. It's just as terrible and inexcusable regardless of its name. And yes, as I mentioned, I would have loved to have a good pattern matching system, but so far the "direction" they're going is actively damaging the language by introducing more pitfalls instead of fixing the existing ones (scopes, declarations, etc.). Just because pattern matching in the abstract could be a feature, that doesn't mean they're going in a good direction by implementing it in a broken way.
I guess like they say, the road to hell is paved with good intentions.
> Design is the thing the client sees, implementation is the stuff they don't see.
By this definition, bugs and other unintended consequences that the user encounters are "Design".
> Calling that an implementation detail is like calling your car's steering wheel an implementation detail.
Yes, if there weren't so many important decisions behind the outcome of a car being steered with a steering wheel, it could be a steering handle, or a steering joystick, or just about anything else that allows you to orient the front wheels of the car. The same is true of the pedals on the floor. Those could be implemented as controls on the steering wheel instead. Whether it's an implementation detail depends on the specificity of the feature in question. When I asked you about an "implementation detail", it was scoped to "the feature is pattern matching" (can I steer the car?) and you scoped it to "the feature is pattern matching without overwriting variables conditionally in surprising ways" (can I steer the car with failure modes that aren't fatal?).
Yes, I am definitely now quibbling over terms. I'm not sure what an appropriate response would have been? Just silence? You responded rather uncharitably and I didn't like it. I felt the need to defend my position.
Not silence, just continuing whatever your underlying point was if your goal wasn't to quibble over semantics. Now I have no idea what you're referring to, but this is clearly getting personal, so let's just leave this here. I think at this point we're both clear on the concrete problems with these features and what our positions are.
