Parsing JSON at tens of gigabytes per second… in Python
10 points by raymii
10 points by raymii
I am begging Python bloggers to understand what FFI is. You're not doing it in Python, you're doing it in C++ with a Python interface.
So, can you parse JSON at tens of gigabytes per second in Python?
I dunno, it's a bit disingenuous to say yes to that when the actual hard work of parsing is done by simdjson.
"Can you parse JSON at tens of gigabytes per second if you use a library that can parse JSON at tens of gigabytes per second?" is a much less clickbaity title, I'll concede.
The interesting part is how to design a Python API for a fast library that allows the programmer to do something interesting without reducing the performance to Python levels. Or rather, it would be interesting if there was a discussion of how they did it beyond a vague note about lazy object creation. And wouldn’t it be more interesting to include pysimdjson in the benchmark, since I guess it’s the closest to fastsimdjson?
Fun fact for the haters: Daniel lemire is one of the original authors of simdjson
Very nice. I also continue to be impressed how much performance msgspec gets as well.