Python Iceberg
21 points by aleyan
21 points by aleyan
I think the gimmick makes this worse than if these were presented more traditionally. I think it also encouraged the inclusion of some extremely dubious entries, many of which do not demonstrate a particularly fluent understanding of the language semantics.
Is the implication that these are design mistakes, hidden but necessary knowledge, unavoidable complexities? It's not particularly clear how the entries were selected. (For example, it's surprising that PEP-578 audit hooks are represented more than once—they're a fairly niche feature, and I believe I may have worked on one of the only large-scale attempts to deploy these in a codebase—in fact, the one that motivated the PEP in the first place.)
Some of the entries (e.g., “Assertions disappear with -O”) are documented, expected, desired behaviour, so I suspect that these are intended to represent knowledge one must have to be effective, but the presentation does not encourage us to contextualise that knowledge into any system of meaning (e.g., that assertions are a programmatic documentation mechanism) or connect that knowledge to anything (e.g., that the compiler can perform dead-code elimination on statically-knowable unreachable branches like if __debug__; thus, assert can be used together with if __debug__ to support in-depth programmatic documentation approaches.)
The order in which these are presented should also be revisited. Why are obvious details of the subprocess module (e.g., the behaviour of shell=True) so deep? I'm led to believe that these items were selected without much (human?) thought…
(e.g., “Assertions disappear with -O”) are documented, expected, desired behaviour
This one absolutely took me by surprise when I first learnt it. I had before that point performed security-relevant checks via asserts.
It could be that we need to do a better job documenting what assert is for. It should be for assertions that you know must be true based on the structure of your code. External data should never be a part of an assertion.
I'm not sure the documentation can fix that one, or specifically that class of issues. Asserts exist in other languages and also have a meaning when discussing programming detached from any implementation. There's always going to be someone assuming assert behaviour the same way they assume what + does on two ints without looking into docs.
(To be clear, I'm not saying this should be changed, just that overloaded word meanings are guaranteed to trip up a few people)
I think the gimmick makes this worse than if these were presented more traditionally.
There is some usefulness/usability vs fun trade off, and I picked fun. Did you view it on mobile or desktop and face any particular issues from the presentation?
Is the implication that these are design mistakes, hidden but necessary knowledge, unavoidable complexities?
No implications are made. As for necessity, I think the above the waterline stuff is hard for a python dev to avoid encountering, but below things get obscure and lots of python devs are effective without knowing most of them.
documented, expected, desired behaviour
Correct. All of these are known things, and most of them link back to official documentation.
The order in which these are presented should also be revisited. Why are obvious details of the subprocess module (e.g., the behaviour of shell=True) so deep? I'm led to believe that these items were selected without much (human?) thought…
The shell=True also bothered me, but not enough to investigate until you mentioned it.
I had gpt-5.6-sol, claude-opus-5, and gemini-3.8-flash rank candidate items on obscurity and cursedness. Their scores were averaged and items were bucketed into deciles by obscurity, and top 10 most cursed items from each bucket were selected. Least obscure items are above the waterline and most obscure got placed down in the abyss.
The interesting about the shell=True command injection is that claude-opus-5 and gemini-3.8-flash gave it an obscurity of 2.2 and 2.5 and sol gave it an obscurity of 6.1! Overall, the models judged items with security implications as more cursed, which while is expected and fair, I don't think is what people want to see. Combined these two issues made shell=True more cursed and lower on the iceberg than it should have been.
Thank you for the detailed response! A lot to consider.
I kinda like this "iceberg" format and there'a good stuff here, but I kinda wish it didn't rotate and occlude stuff on the other side. As is, there's no practical way for me to read them all or look for a specific thing.
I went into this wondering where you put logging/multiprocess deadlocks, and i'm not sure if I missed it or you didn't rank it.