Skip to content

Instantly share code, notes, and snippets.

@moreati

moreati/popen_resourcewarning.py

Last active August 28, 2024 22:07
Show Gist options
  • Save moreati/b8d157ff82cb15234bece4033accc5e5 to your computer and use it in GitHub Desktop.
Save moreati/b8d157ff82cb15234bece4033accc5e5 to your computer and use it in GitHub Desktop.
Download ZIP
Demonstration of Python subprocess.Popen() ResourceWarning
Raw
popen_resourcewarning.py
import os
import subprocess
import sys
import warnings
# By default ResourceWarning output (and other) is hidden.
# Setting environment variable PYTHONWARNINGS=default will
# show them, similar to the below.
if not sys.warnoptions:
warnings.simplefilter('default')
proc1 = subprocess.Popen(['true'])
print(f'{proc1.pid=}')
print(f'{os.waitpid(proc1.pid, 0)=}')
# This will emit a ResourceWarning (on CPython atleast)
del proc1
print()
proc2 = subprocess.Popen(['true'])
print(f'{proc2.pid=}')
print(f'{proc2.wait()=}')
# This will not emit a ResourceWarning
del proc2
@moreati
Copy link
Author

moreati commented Aug 28, 2024 

$ python3 popen_resourcewarning.py
proc1.pid=27554
os.waitpid(proc1.pid, 0)=(27554, 0)
/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/subprocess.py:1127: ResourceWarning: subprocess 27554 is still running
  _warn("subprocess %s is still running" % self.pid,
ResourceWarning: Enable tracemalloc to get the object allocation traceback

proc2.pid=27555
proc2.wait()=0

@moreati
Copy link
Author

moreati commented Aug 28, 2024 •
edited
Loading

Although the ResourceWarning says "subprocess 27554 is still running", this is not strictly true. By calling os.waitpid() directly we have allowed the operating systems to clean up any zombie child process, but because we've bypassed the proc1 Popen object it is unaware.

Instead of using os.waitpid() or other variants we should use the Popen.wait() method, as shown with proc2.

The ResourceWarning text could possibly be worded more clearly.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment