Yes, Python is the wrong choice if performance is your top priority.
But here’s another perspective: why leave easy performance wins on the table? Especially if the cost is simpler code that works as you probably wanted anyway with both None and []?
Python is great if you want a really fast development cycle, because the code is generally quite simple and it’s “fast enough.” Any wins for “fast enough” is appreciated, because it delays me needing to actually look into little performance issues. It’s pretty easy for me to write a simple regex to fix this cose (s/if len\((\w+)\) == 0:/ifnot \1:/), and my codebase will be slightly faster. That’s awesome! I could even write up a quick pylint or ruff rule to catch these cases for developers going forward (if there isn’t one already).
If I’m actively tweaking things in my Python code to get a little better performance, you’re right, I should probably just use something else (writing a native module is probably a better use of time). But the author isn’t arguing that you should do that, just that, in this case, if not foo is preferred over iflen(foo)== 0 for technical reasons, and I’ll add that it makes a ton of sense for readability reasons as well.
Here are some other simple wins:
[] and {} instead of list() and dict() - the former copy constants, whereas the latter actually constructs things; oh, and you save a few chars
use list comprehensions instead of regular loops - list comprehensions seem to be faster due to not needing to call append (and less code)
use built-ins when you can - they’re often implemented in native code
I consider each of those cleaner Python code anyway, because they’re less code, just as explicit, and use built-in language features instead of reinventing the wheel.
It comes down to the question “Is YOUR C++ code faster than Python?” (and of course the reverse).
I’ve built a SCADA from scratch and performance requirements are low to begin with, seeing as it’s all network bound and real world objects take time to react, but I’m finding everything is very timely.
A colleague used SQLAlchemy for a similar task and got abysmal performance. No wonder, it’s constantly querying the DB for single results.
We rewrote some Fortran code (known for fast perf) into Python and the net result was faster. Why? They used bubble sort in a hot loop, whereas we used Python’s built-in sort (probably qsort or similar). So despite Python being “slower” on average, good architecture matters a lot more.
And your Python code doesn’t have to be 100% Python, you can write performance-critical code in something else, like C++ or Rust. This is very common, and it’s why popular Python libraries like numpy and scipy are written in a more performant language with a Python wrapper.
I’ve worked on a library that’s Python because the users of said library are used to Python.
The original version of the project made heavy use of numpy, so the actual performance sensitive code was effectively C++ and fourtran, which is what numpy is under the hood.
We eventually replaced the performance sensitive part of the code with Rust (and still some fourtran because BLAS) which ended up being about 10x faster.
The outermost layer of code is still Python though.
Exactly! Most of the important libraries in other languages have Python support. So I can use Python and not care if they wrote the library in C++, Rust, or something more exotic, provided it works in Python.
Python is more about gluing stuff together than writing a bunch of greenfield logic. If you run into performance problems and you’re using Python, the best solution is to start rewriting the slow parts in something faster, not to rewrite the whole thing in something different.
That said, most of my personal projects are in Rust, for a variety of reasons. But I still use a lot of Python as well, and it’s what I use 99% of the time at work.
My point is tha the libraries itself are not in Python and thus most likely not exclusive to it. This is not an attack on Python, I just find it funny a bit :)
Because, while you don’t want to nitpick on each instruction cycle, sometimes the code runs millions of times and each microsecond adds up.
Keep in mind that people use this kind of things for work, serving real world customers who are doing their work.
Yes, the language itself is not optimal even by design, but its easy to work with, so they are making it worth a while. There’s no shortage of people who can work with it. It is easy to develop and maintain stuff with it, cutting development cost. Yes, we’re talking real businesses with real resource constraints.
Exactly. We picked it for the reasons you mentioned, and I still think it’s a good choice.
That said, some of our heavier logic is in a lower-level language. We had some Fortran code until recently (rewrote in Python and just ate the perf cost to lower barrier to other devs fixing stuff), and we’re introducing some C++ code in the next month or two. But the bulk of our code is in Python, because that’s what glues everything together, and the code is fast enough for our needs.
People seem to be unaware that python has bindings for lower-level languages like C. In fact, people have been heavily using resource intensive libraries implemented in C (e.g. numpy, scipy, pandas, uwsgi).
Also, Python interpreter performance has come a long way.
You may want to beneficiate from little performance boost even though you mostly don’t need it and still need python’s advantages. Being interested in performance isnt always looking for the very best performance there is out of any language, it can also be using little tips to go a tiny bit faster when you can.
I know I’m gonna get downvoted to oblivion for this, but… Serious question: why use Python if you’re concerned about performance?
Yes, Python is the wrong choice if performance is your top priority.
But here’s another perspective: why leave easy performance wins on the table? Especially if the cost is simpler code that works as you probably wanted anyway with both
Noneand[]?Python is great if you want a really fast development cycle, because the code is generally quite simple and it’s “fast enough.” Any wins for “fast enough” is appreciated, because it delays me needing to actually look into little performance issues. It’s pretty easy for me to write a simple regex to fix this cose (
s/if len\((\w+)\) == 0:/if not \1:/), and my codebase will be slightly faster. That’s awesome! I could even write up a quickpylintorruffrule to catch these cases for developers going forward (if there isn’t one already).If I’m actively tweaking things in my Python code to get a little better performance, you’re right, I should probably just use something else (writing a native module is probably a better use of time). But the author isn’t arguing that you should do that, just that, in this case,
if not foois preferred overif len(foo) == 0for technical reasons, and I’ll add that it makes a ton of sense for readability reasons as well.Here are some other simple wins:
[]and{}instead oflist()anddict()- the former copy constants, whereas the latter actually constructs things; oh, and you save a few charsappend(and less code)I consider each of those cleaner Python code anyway, because they’re less code, just as explicit, and use built-in language features instead of reinventing the wheel.
It comes down to the question “Is YOUR C++ code faster than Python?” (and of course the reverse).
I’ve built a SCADA from scratch and performance requirements are low to begin with, seeing as it’s all network bound and real world objects take time to react, but I’m finding everything is very timely.
A colleague used SQLAlchemy for a similar task and got abysmal performance. No wonder, it’s constantly querying the DB for single results.
Exactly!
We rewrote some Fortran code (known for fast perf) into Python and the net result was faster. Why? They used
bubble sortin a hot loop, whereas we used Python’s built-in sort (probably qsort or similar). So despite Python being “slower” on average, good architecture matters a lot more.And your Python code doesn’t have to be 100% Python, you can write performance-critical code in something else, like C++ or Rust. This is very common, and it’s why popular Python libraries like numpy and scipy are written in a more performant language with a Python wrapper.
I’ve worked on a library that’s Python because the users of said library are used to Python.
The original version of the project made heavy use of numpy, so the actual performance sensitive code was effectively C++ and fourtran, which is what numpy is under the hood.
We eventually replaced the performance sensitive part of the code with Rust (and still some fourtran because BLAS) which ended up being about 10x faster.
The outermost layer of code is still Python though.
Honestly most people use Python because it has fantastic libraries. They optimize it because the language is middling, but the libraries are gorgeous
ETA: This might double post because my Internet sucks right now, will fix when I have a chance
In C++ if I remember correctly…
Edit: I do https://codefinity.com/blog/Python-Libraries-Written-in-C-plus-plus
What do I care what language the library is written in as long as it works for what I need it do?
Exactly! Most of the important libraries in other languages have Python support. So I can use Python and not care if they wrote the library in C++, Rust, or something more exotic, provided it works in Python.
Python is more about gluing stuff together than writing a bunch of greenfield logic. If you run into performance problems and you’re using Python, the best solution is to start rewriting the slow parts in something faster, not to rewrite the whole thing in something different.
That said, most of my personal projects are in Rust, for a variety of reasons. But I still use a lot of Python as well, and it’s what I use 99% of the time at work.
My point is tha the libraries itself are not in Python and thus most likely not exclusive to it. This is not an attack on Python, I just find it funny a bit :)
This is my two cents as someone in the industry.
Because, while you don’t want to nitpick on each instruction cycle, sometimes the code runs millions of times and each microsecond adds up.
Keep in mind that people use this kind of things for work, serving real world customers who are doing their work.
Yes, the language itself is not optimal even by design, but its easy to work with, so they are making it worth a while. There’s no shortage of people who can work with it. It is easy to develop and maintain stuff with it, cutting development cost. Yes, we’re talking real businesses with real resource constraints.
Exactly. We picked it for the reasons you mentioned, and I still think it’s a good choice.
That said, some of our heavier logic is in a lower-level language. We had some Fortran code until recently (rewrote in Python and just ate the perf cost to lower barrier to other devs fixing stuff), and we’re introducing some C++ code in the next month or two. But the bulk of our code is in Python, because that’s what glues everything together, and the code is fast enough for our needs.
People seem to be unaware that python has bindings for lower-level languages like C. In fact, people have been heavily using resource intensive libraries implemented in C (e.g. numpy, scipy, pandas, uwsgi).
Also, Python interpreter performance has come a long way.
You may want to beneficiate from little performance boost even though you mostly don’t need it and still need python’s advantages. Being interested in performance isnt always looking for the very best performance there is out of any language, it can also be using little tips to go a tiny bit faster when you can.
Alternatively, why wait twice as long for your python code to execute as you have to?
I have the same question. I prefer other languages. I use G’MIC for image processing over Python and C++.