Reuven M. Lerner

@lernerpython.com

Helping you become more confident with Python and Pandas since 1995. • Courses: LernerPython.com • Newsletters: BetterDevelopersWeekly.com • BambooWeekly.com • Books: PythonWorkout.com • PandasWorkout.com • Videos: YouTube.com/reuvenlerner

Stacking loc to filter a #Python #Pandas dataframe? Order can matter: ( df .loc[ pd.col('total_amount') > 25 ] .loc[ pd.col('passenger_count') > 0 ] ) # 64.5 ms ( df .loc[ pd.col('passenger_count') > 0 ] .loc[ pd.col('total_amount') > 25 ] ) # 95.8 ms -- 48% slower

Bild

Filtering rows in a #Python #Pandas data frame? Use .loc: ( df .loc[ pd.col('passenger_count') > 1 ] ) pd.col refers to the previous line's returned data frame. So we can stack them: ( df .loc[ pd.col('passenger_count') > 1 ] .loc[ pd.col('total_amount') > 50 ] )

Bild

How much memory does a #Python list use? sys.getsizeof will tell you, sort of: It reports the memory used by the list, but not its elements: >>> import sys >>> x = [10, 20, 30, 40, 50] >>> sys.getsizeof(x) 104 >>> x[0] = 'abcdefghij' * 100_000_000 >>> sys.getsizeof(x) 104

Bild

Want stderr in your #Python program to go somewhere else? Use redirect_stderr in contextlib: import sys from contextlib import redirect_stderr with redirect_stderr(open('yikes.txt', 'w')): print('hi on screen') print('danger!', file=sys.stderr)

Bild

Writing a bunch to non-stdout in #Python? You might want to use redirect_stdout from contextlib: from contextlib import redirect_stdout with redirect_stdout(open('output.txt', 'w')): # in this block, stdout goes to output.txt print('hi 1')

Bild

Want to print to stderr, not stdout, in a #Python program? Use sys.stderr and the "file" kwarg: >>> import sys >>> print('Normal text') Normal text >>> print('Warning text', file=sys.stdout) Warning text stdout and stderr often look identical in a terminal -- but they aren't!

Bild

Normally, "print" in #Python writes to stdout. Write elsewhere with the "file" keyword argument: f1 = open('file1.txt', 'w') print('hi, stdout') # prints: hi, stdout print('hi, file1', file=f1) # nothing appears f1.close() print(open('file1.txt').read()) # we see: hi, file1

Bild

Want stdout from your #Python program to go elsewhere? Assign sys.stdout to a writeable file. (Don't forget to keep the original around!) old_stdout = sys.stdout sys.stdout = open('/tmp/output.txt', 'w') print('hello???') # goes to /tmp/output.txt Better: print's file kwarg.

Bild

After you import a #Python module, its repr (printed representation) shows its name and the loaded file. But some modules don't have files; they were "frozen" into Python. They have no __file__ attribute, and show "frozen" in their __loader__.

Bild

Once you import a #Python module, a second import won't reload it. That's usually good. But what if you want to? (Common in development and debugging.) from importlib import reload reload(mymod) This forces the reload.

Bild

You import a #Python module with the variable you'll define, not a filename. "import mymod" tells Python to find mymod.py in each dir in sys.path. The first directory is '' (the empty string) -- i.e., the directory where the program is running (i.e., not where it was defined).

Bild

Want #Python to import modules in non-default dirs? You could do this: import sys sys.path.insert(1, '/var/mymods') But: It doesn't scale across many programs. And what if the dir changes? Better: Set the PYTHONPATH environment variable — scalable and set in one place.

Bild

sys.path tells #Python where to look when importing a module. Think of it as a list of directories. But it also handles zipfiles. A zipfile in sys.path is treated as a directory. Files in the zipfile can be imported.

Bild

You say "import mymod". Where does #Python look? Check sys.path, a list of directory names (strings) where it searches. By default: - Current dir (empty string) - Standard library - site-packages The first match wins, which can lead to unpleasant surprises — be careful!

Bild

In #Python, import always defines a variable. But it only loads the module once. sys.modules, a dict, tracks already-loaded modules: - keys are strings, the module names - values are module objects If you say "import pandas as pd", sys.modules has a key 'pandas', not 'pd'.

Bild

"sys" is where #Python keeps vital info about its runtime environment. So why import it? Not to load it; sys is loaded when Python starts. Rather, the import just defines "sys" as a global variable. Importing "sys" takes almost no time, and provides useful info.

Bild

Old #Python: a traceback pointing vaguely at the wrong line. New Python: errors that underline the exact spot — and suggest fixes. Python 3.10 and 3.11 rewrote many error messages. They pinpoint the failing sub-expression and often add "did you mean...?"

Bild

Find the version of #Python: >>> sys.version '3.14.6 (main, Jun 10 2026, 10:03:53) [Clang 21.0.0 (clang-2100.0.123.102)]' To use it in a program, you want version_info: >>> sys.version_info sys.version_info(major=3, minor=14, micro=6, releaselevel='final', serial=0)

Bild

Want a #Python Enum, but don't care about the values? Use auto: from enum import Enum, auto class Days(Enum): SUN = auto() MON = auto() TUE = auto() You can now use Days.SUN and Days.MON. With auto, you care the Enum's values are different, not what they are.

Bild

Old #Python: threads and callbacks for network-heavy code. New Python: async def / await async/await (3.5) handle thousands of concurrent I/O operations in one thread. A different concurrency model, with many updates in recent versions of Python.

Bild

Want to iterate over a #Python Enum? __members__ returns a dict-like object, and it supports items(): from enum import Enum class Days(Enum): SUN = 1 MON = 2 TUE = 3 [etc] for k, v in Days.__members__.items(): print(f'{k}: {v}') SUN: Days.SUN MON: Days.MON TUE: Days.TUE [etc]

Bild