python string literals are kinda funny
50 points by raymii
50 points by raymii
This is ultra cursed...
Now I'm very curious about all the other languages which have format strings and whether they have similar crazy behaviour!
There doesn’t seem much that’s cursed besides the initial escaping annoyance? The interpolation is an expression, so aside from a few limitations you can put any expression there. Any langage where you interpolate full expressions (Ruby, JS) will have similar complexities.
I think it's just that I never really thought about how far you could push format strings; usually for anything slightly more complex than a variable/function call, I would always factor it out into a variable for readabilty.
It's quite far. Here I'm using them to parse Javascript: https://github.com/bablr-lang/vite_plugin-cstml-to-jsx/blob/87c26478f9dd2bab891ee39084c7e4036ac72f48/lib/index.js#L229-L262
You might expect this to first fill the expression gaps to make a string, then parse the string. Actually it parses first into a tree, then fills the gaps in the tree with other trees (which is injection-safe). I'm pretty proud of that : )
I've checked and Rust does the correct thing here, the literal r"asdf\" becomes a string containing asdf\, while r"asdf\"" produces a syntax error because the properly terminated string is directly followed by a new unterminated string. Using non-raw literals has the opposite behavior, as expected.
lexing an f-string requires invoking a full python parser on the expression in the curly braces. this expression can contain quotes, be split into multiple lines, and even contain comments!
That seems pretty standard? Personally, I tend to use delimited comments inside splices, rather than line-end comments; of course, Python doesn't have delimited comments, but that's more a case of python comments being kinda funny (or rather, tragic :'( ), rather than its string literals.
I can't remember doing it in Python (yet!), but there have been several times where I've used newlines in string-splices to improve code layout. One that comes to mind was a template for a plaintext email, which was all written on one line, spanning several-thousand columns. I inquired about this and was told that we couldn't break it into several lines, since that whitespace would appear in the resulting email, and break the ASCII-art tables it contained.
My fix was simple: splice in a bunch of empty strings, but write those splices over two lines. I suppose in Python it would look like:
foo = f"something on line 1 {""
}and another thing on line 1, because the splice only {""
}inserts an empty string which doesn't actually affect {""
}the contents. Yet this lets us write the literal itself {""
}across several lines in the source file. {""
}Of course, this isn't really necessary in a full Python file, {""
}since we can e.g. append-together a bunch of smaller {""
}strings, if we like; but if we ONLY get to write the literal {""
}(e.g. in a templating system) then it can be a neat trick {""
}to know about!"
FWIW in Python you could just use a multiline string and escape the EOLs:
foo = """something on line 1 \
and another thing on line 1, because the splice only \
inserts an empty string which doesn't actually affect \
the contents. Yet this lets us write the literal itself \
across several lines in the source file. \
Of course, this isn't really necessary in a full Python file, \
since we can e.g. append-together a bunch of smaller \
strings, if we like; but if we ONLY get to write the literal \
(e.g. in a templating system) then it can be a neat trick \
to know about!\
"""
Implicit concatenation is also an option:
foo = "something on line 1 "\
"and another thing on line 1, because the splice only "\
"inserts an empty string which doesn't actually affect "\
"the contents. Yet this lets us write the literal itself "\
"across several lines in the source file. "\
"Of course, this isn't really necessary in a full Python file, "\
"since we can e.g. append-together a bunch of smaller "\
"strings, if we like; but if we ONLY get to write the literal "\
"(e.g. in a templating system) then it can be a neat trick "\
"to know about!"
(you can also use a single pair of parens to group up the implicit concatenation).
In both cases I favor moving the separator space to the start of each line, as I find the slight indent makes it much clearer that it was not forgotten compared to the ragged right edge.
I could swear f'{'}'}' wasn't a valid format string because I tried something like that years ago and got a syntax error. And what do you know, apparently that's a recent addition! (Well, if you can call 2022 recent.) Before that, strings literals were parsed first and then their contents were re-parsed for {}, so that snippet would be interpreted as f'{' } '}' (which is invalid syntax for multiple reasons), similarly to the one in the quiz.
I used to run a little website with an f-strings quiz for a while, but I had the domain expire. If someone wants to explore the fun of Python f-strings, it's still here: https://mitsuhiko.github.io/fstrings-wtf/
this was definitely originally done to simplify the implementation
This explains all kinds of things in Python, especially Python-2-era idiosyncracies (print-as-statement, old-style-classes, ...)
f'{x := 67}'
Just when I thought I understood what the walrus operator did
What did you think the walrus operator did? Because what it does is "assign as an expression", so anywhere you can put an expression you can put a walrus operator e.g.
class F:
def foo(self, a=(b:=3)):
pass
obviously defines the class attribute b.
And the contents of the braces in an f-string (or t-string) are an expression.
the contents of the braces in an f-string (or t-string) are an expression
Except that the expression inside an interpolation is terminated by an unprotected : which separates the expression from the format specification, so the := in f'{x := 67}' is not a walrus operator, it’s a : delimiter and an = flag.