Python: Killing subprocesses on Windows

We're on Python 2.6, and using the subprocess module to execute some applications (like we do on our build system), sometimes things hang (i.e. an open dialog box or something).

So far, I've just killed the process after timeout, which is fine if there are no childprocesses in there.

Now, we've shifted to the Nose module for running our unit-tests (and Freshen for the specification-by-example acceptance tests), and that always uses one or more subprocesses to execute tests in... ergo: problems.

The simplest way is just to run taskkill, which is available at least from Windows XP...

p = subprocess.POpen(....)
# wait for exit or timeout
if timeout:
  subprocess.call(['taskkill', '/F', '/T', '/PID', str(p.pid)])

If you don't want to run another program, there's the long way around, by using comtypes and ctypes to access the WMI and Win32 API functions. I wrote this before I found out about the above, but it was easy to port C++ samples to this, and I already had comtypes in our system. Maybe it's useful for someone...

def killsubprocesses(parent_pid):

    '''kill parent and all subprocess using COM/WMI and the win32api'''

    

    log = logging.getLogger('killprocesses')

    

    try:

        import comtypes.client

    except ImportError:

        log.debug("comtypes not present, not killing subprocesses")

        return

    

    logging.getLogger('comtypes').setLevel(logging.INFO)

    

    log.debug('Querying process tree...')

    # get pid and subprocess pids for all alive processes

    WMI = comtypes.client.CoGetObject('winmgmts:')

    processes = WMI.InstancesOf('Win32_Process')

    subprocess_pids = {} # parent pid -> list of child pids

    

    for process in processes:

        pid = process.Properties_('ProcessID').Value

        parent = process.Properties_('ParentProcessId').Value

        log.trace("process %i's parent is: %s" % (pid, parent))

        subprocess_pids.setdefault(parent, []).append(pid)

        subprocess_pids.setdefault(pid, [])

      

    # find which we need to kill

    log.debug('Determining subprocesses for pid %i...' % parent_pid)

    processes_to_kill = []

    parent_processes = [parent_pid]

    while parent_processes:

        current_pid = parent_processes.pop()

        subps = subprocess_pids[current_pid]

        log.debug("process %i children are: %s" % (current_pid, subps))

        parent_processes.extend(subps)

        processes_to_kill.extend(subps)

    # kill the subprocess tree

    if processes_to_kill:

        log.info('Process pid %i spawned %i subprocesses, terminating them...' % 

            (parent_pid, len(processes_to_kill)))

    else:

        log.debug('Process pid %i had no subprocesses.' % parent_pid)

    import ctypes

    kernel32 = ctypes.windll.kernel32

    for pid in processes_to_kill:

        hProcess = kernel32.OpenProcess(PROCESS_TERMINATE, FALSE, pid);

        if not hProcess:

            _log.warning('Unable to open process pid %i for termination' % pid)

        else:

            _log.debug('Terminating pid %i' % pid)                        

            kernel32.TerminateProcess(hProcess, 3)

            kernel32.CloseHandle(hProcess)

This code could of course be improved by not walking all processes, OTOH, I have between 80 and 200 running on my PC at all times, so it's not a lot of data. Especially, since I only call this function if I've already waited too long, it doesn't matter if it takes a few more seconds.

Also, I long for Haskell where this tree-walking mapping stuff could be way more concise. Maybe python's itertools package has something I could use to avoid coding the dirty stuff by hand.

A few more solutions is available at this StackOverflow question, depending on what kind third-party of libraries you have available.

Moral of the story:

It doesn't matter if you have to write a lot of code in order to find out that there's a one liner which solves your problem. Use the short version and be happy you don't have to maintain the 30+ lines version. :)