1 # subprocess - Subprocesses with accessible I/O streams
3 # For more information about this module, see PEP 324.
5 # This module should remain compatible with Python 2.2, see PEP 291.
7 # Copyright (c) 2003-2005 by Peter Astrand <astrand@lysator.liu.se>
9 # Licensed to PSF under a Contributor Agreement.
10 # See http://www.python.org/2.4/license for licensing details.
12 r"""subprocess - Subprocesses with accessible I/O streams
14 This module allows you to spawn processes, connect to their
15 input/output/error pipes, and obtain their return codes. This module
16 intends to replace several other, older modules and functions, like:
24 Information about how the subprocess module can be used to replace these
25 modules and functions can be found below.
29 Using the subprocess module
30 ===========================
31 This module defines one class called Popen:
33 class Popen(args, bufsize=0, executable=None,
34 stdin=None, stdout=None, stderr=None,
35 preexec_fn=None, close_fds=False, shell=False,
36 cwd=None, env=None, universal_newlines=False,
37 startupinfo=None, creationflags=0):
42 args should be a string, or a sequence of program arguments. The
43 program to execute is normally the first item in the args sequence or
44 string, but can be explicitly set by using the executable argument.
46 On UNIX, with shell=False (default): In this case, the Popen class
47 uses os.execvp() to execute the child program. args should normally
48 be a sequence. A string will be treated as a sequence with the string
49 as the only item (the program to execute).
51 On UNIX, with shell=True: If args is a string, it specifies the
52 command string to execute through the shell. If args is a sequence,
53 the first item specifies the command string, and any additional items
54 will be treated as additional shell arguments.
56 On Windows: the Popen class uses CreateProcess() to execute the child
57 program, which operates on strings. If args is a sequence, it will be
58 converted to a string using the list2cmdline method. Please note that
59 not all MS Windows applications interpret the command line the same
60 way: The list2cmdline is designed for applications using the same
61 rules as the MS C runtime.
63 bufsize, if given, has the same meaning as the corresponding argument
64 to the built-in open() function: 0 means unbuffered, 1 means line
65 buffered, any other positive value means use a buffer of
66 (approximately) that size. A negative bufsize means to use the system
67 default, which usually means fully buffered. The default value for
68 bufsize is 0 (unbuffered).
70 stdin, stdout and stderr specify the executed programs' standard
71 input, standard output and standard error file handles, respectively.
72 Valid values are PIPE, an existing file descriptor (a positive
73 integer), an existing file object, and None. PIPE indicates that a
74 new pipe to the child should be created. With None, no redirection
75 will occur; the child's file handles will be inherited from the
76 parent. Additionally, stderr can be STDOUT, which indicates that the
77 stderr data from the applications should be captured into the same
78 file handle as for stdout.
80 If preexec_fn is set to a callable object, this object will be called
81 in the child process just before the child is executed.
83 If close_fds is true, all file descriptors except 0, 1 and 2 will be
84 closed before the child process is executed.
86 if shell is true, the specified command will be executed through the
89 If cwd is not None, the current directory will be changed to cwd
90 before the child is executed.
92 If env is not None, it defines the environment variables for the new
95 If universal_newlines is true, the file objects stdout and stderr are
96 opened as a text files, but lines may be terminated by any of '\n',
97 the Unix end-of-line convention, '\r', the Macintosh convention or
98 '\r\n', the Windows convention. All of these external representations
99 are seen as '\n' by the Python program. Note: This feature is only
100 available if Python is built with universal newline support (the
101 default). Also, the newlines attribute of the file objects stdout,
102 stdin and stderr are not updated by the communicate() method.
104 The startupinfo and creationflags, if given, will be passed to the
105 underlying CreateProcess() function. They can specify things such as
106 appearance of the main window and priority for the new process.
110 This module also defines two shortcut functions:
112 call(*popenargs, **kwargs):
113 Run command with arguments. Wait for command to complete, then
114 return the returncode attribute.
116 The arguments are the same as for the Popen constructor. Example:
118 retcode = call(["ls", "-l"])
120 check_call(*popenargs, **kwargs):
121 Run command with arguments. Wait for command to complete. If the
122 exit code was zero then return, otherwise raise
123 CalledProcessError. The CalledProcessError object will have the
124 return code in the returncode attribute.
126 The arguments are the same as for the Popen constructor. Example:
128 check_call(["ls", "-l"])
132 Exceptions raised in the child process, before the new program has
133 started to execute, will be re-raised in the parent. Additionally,
134 the exception object will have one extra attribute called
135 'child_traceback', which is a string containing traceback information
136 from the childs point of view.
138 The most common exception raised is OSError. This occurs, for
139 example, when trying to execute a non-existent file. Applications
140 should prepare for OSErrors.
142 A ValueError will be raised if Popen is called with invalid arguments.
144 check_call() will raise CalledProcessError, if the called process
145 returns a non-zero return code.
150 Unlike some other popen functions, this implementation will never call
151 /bin/sh implicitly. This means that all characters, including shell
152 metacharacters, can safely be passed to child processes.
157 Instances of the Popen class have the following methods:
160 Check if child process has terminated. Returns returncode
164 Wait for child process to terminate. Returns returncode attribute.
166 communicate(input=None)
167 Interact with process: Send data to stdin. Read data from stdout
168 and stderr, until end-of-file is reached. Wait for process to
169 terminate. The optional stdin argument should be a string to be
170 sent to the child process, or None, if no data should be sent to
173 communicate() returns a tuple (stdout, stderr).
175 Note: The data read is buffered in memory, so do not use this
176 method if the data size is large or unlimited.
178 The following attributes are also available:
181 If the stdin argument is PIPE, this attribute is a file object
182 that provides input to the child process. Otherwise, it is None.
185 If the stdout argument is PIPE, this attribute is a file object
186 that provides output from the child process. Otherwise, it is
190 If the stderr argument is PIPE, this attribute is file object that
191 provides error output from the child process. Otherwise, it is
195 The process ID of the child process.
198 The child return code. A None value indicates that the process
199 hasn't terminated yet. A negative value -N indicates that the
200 child was terminated by signal N (UNIX only).
203 Replacing older functions with the subprocess module
204 ====================================================
205 In this section, "a ==> b" means that b can be used as a replacement
208 Note: All functions in this section fail (more or less) silently if
209 the executed program cannot be found; this module raises an OSError
212 In the following examples, we assume that the subprocess module is
213 imported with "from subprocess import *".
216 Replacing /bin/sh shell backquote
217 ---------------------------------
220 output = Popen(["mycmd", "myarg"], stdout=PIPE).communicate()[0]
223 Replacing shell pipe line
224 -------------------------
225 output=`dmesg | grep hda`
227 p1 = Popen(["dmesg"], stdout=PIPE)
228 p2 = Popen(["grep", "hda"], stdin=p1.stdout, stdout=PIPE)
229 output = p2.communicate()[0]
232 Replacing os.system()
233 ---------------------
234 sts = os.system("mycmd" + " myarg")
236 p = Popen("mycmd" + " myarg", shell=True)
237 pid, sts = os.waitpid(p.pid, 0)
241 * Calling the program through the shell is usually not required.
243 * It's easier to look at the returncode attribute than the
246 A more real-world example would look like this:
249 retcode = call("mycmd" + " myarg", shell=True)
251 print >>sys.stderr, "Child was terminated by signal", -retcode
253 print >>sys.stderr, "Child returned", retcode
255 print >>sys.stderr, "Execution failed:", e
262 pid = os.spawnlp(os.P_NOWAIT, "/bin/mycmd", "mycmd", "myarg")
264 pid = Popen(["/bin/mycmd", "myarg"]).pid
269 retcode = os.spawnlp(os.P_WAIT, "/bin/mycmd", "mycmd", "myarg")
271 retcode = call(["/bin/mycmd", "myarg"])
276 os.spawnvp(os.P_NOWAIT, path, args)
278 Popen([path] + args[1:])
283 os.spawnlpe(os.P_NOWAIT, "/bin/mycmd", "mycmd", "myarg", env)
285 Popen(["/bin/mycmd", "myarg"], env={"PATH": "/usr/bin"})
290 pipe = os.popen(cmd, mode='r', bufsize)
292 pipe = Popen(cmd, shell=True, bufsize=bufsize, stdout=PIPE).stdout
294 pipe = os.popen(cmd, mode='w', bufsize)
296 pipe = Popen(cmd, shell=True, bufsize=bufsize, stdin=PIPE).stdin
299 (child_stdin, child_stdout) = os.popen2(cmd, mode, bufsize)
301 p = Popen(cmd, shell=True, bufsize=bufsize,
302 stdin=PIPE, stdout=PIPE, close_fds=True)
303 (child_stdin, child_stdout) = (p.stdin, p.stdout)
308 child_stderr) = os.popen3(cmd, mode, bufsize)
310 p = Popen(cmd, shell=True, bufsize=bufsize,
311 stdin=PIPE, stdout=PIPE, stderr=PIPE, close_fds=True)
314 child_stderr) = (p.stdin, p.stdout, p.stderr)
317 (child_stdin, child_stdout_and_stderr) = os.popen4(cmd, mode, bufsize)
319 p = Popen(cmd, shell=True, bufsize=bufsize,
320 stdin=PIPE, stdout=PIPE, stderr=STDOUT, close_fds=True)
321 (child_stdin, child_stdout_and_stderr) = (p.stdin, p.stdout)
326 Note: If the cmd argument to popen2 functions is a string, the command
327 is executed through /bin/sh. If it is a list, the command is directly
330 (child_stdout, child_stdin) = popen2.popen2("somestring", bufsize, mode)
332 p = Popen(["somestring"], shell=True, bufsize=bufsize
333 stdin=PIPE, stdout=PIPE, close_fds=True)
334 (child_stdout, child_stdin) = (p.stdout, p.stdin)
337 (child_stdout, child_stdin) = popen2.popen2(["mycmd", "myarg"], bufsize, mode)
339 p = Popen(["mycmd", "myarg"], bufsize=bufsize,
340 stdin=PIPE, stdout=PIPE, close_fds=True)
341 (child_stdout, child_stdin) = (p.stdout, p.stdin)
343 The popen2.Popen3 and popen3.Popen4 basically works as subprocess.Popen,
346 * subprocess.Popen raises an exception if the execution fails
347 * the capturestderr argument is replaced with the stderr argument.
348 * stdin=PIPE and stdout=PIPE must be specified.
349 * popen2 closes all filedescriptors by default, but you have to specify
350 close_fds=True with subprocess.Popen.
356 mswindows = (sys.platform == "win32")
363 # Exception classes used by this module.
364 class CalledProcessError(Exception):
365 """This exception is raised when a process run by check_call() returns
366 a non-zero exit status. The exit status will be stored in the
367 returncode attribute."""
368 def __init__(self, returncode, cmd):
369 self.returncode = returncode
372 return "Command '%s' returned non-zero exit status %d" % (self.cmd, self.returncode)
379 # SCons: the threading module is only used by the communicate()
380 # method, which we don't actually use, so don't worry if we
384 if 0: # <-- change this to use pywin32 instead of the _subprocess driver
386 from win32api import GetStdHandle, STD_INPUT_HANDLE, \
387 STD_OUTPUT_HANDLE, STD_ERROR_HANDLE
388 from win32api import GetCurrentProcess, DuplicateHandle, \
389 GetModuleFileName, GetVersion
390 from win32con import DUPLICATE_SAME_ACCESS, SW_HIDE
391 from win32pipe import CreatePipe
392 from win32process import CreateProcess, STARTUPINFO, \
393 GetExitCodeProcess, STARTF_USESTDHANDLES, \
394 STARTF_USESHOWWINDOW, CREATE_NEW_CONSOLE
395 from win32event import WaitForSingleObject, INFINITE, WAIT_OBJECT_0
397 # SCons: don't die on Python versions that don't have _subprocess.
399 from _subprocess import *
418 except AttributeError:
423 except AttributeError:
426 __all__ = ["Popen", "PIPE", "STDOUT", "call", "check_call", "CalledProcessError"]
429 MAXFD = os.sysconf("SC_OPEN_MAX")
430 except KeyboardInterrupt:
431 raise # SCons: don't swallow keyboard interrupts
435 # True/False does not exist on 2.2.0
446 return type(obj) == type(1)
447 def is_int_or_long(obj):
448 return type(obj) in (type(1), type(1L))
451 return isinstance(obj, int)
452 def is_int_or_long(obj):
453 return isinstance(obj, (int, long))
457 except AttributeError:
459 types.StringTypes = (types.StringType, types.UnicodeType)
460 except AttributeError:
461 types.StringTypes = (types.StringType,)
463 return type(obj) in types.StringTypes
466 return isinstance(obj, types.StringTypes)
471 for inst in _active[:]:
472 if inst.poll(_deadstate=sys.maxint) >= 0:
476 # This can happen if two threads create a new Popen instance.
477 # It's harmless that it was already removed, so ignore.
484 def call(*popenargs, **kwargs):
485 """Run command with arguments. Wait for command to complete, then
486 return the returncode attribute.
488 The arguments are the same as for the Popen constructor. Example:
490 retcode = call(["ls", "-l"])
492 return apply(Popen, popenargs, kwargs).wait()
495 def check_call(*popenargs, **kwargs):
496 """Run command with arguments. Wait for command to complete. If
497 the exit code was zero then return, otherwise raise
498 CalledProcessError. The CalledProcessError object will have the
499 return code in the returncode attribute.
501 The arguments are the same as for the Popen constructor. Example:
503 check_call(["ls", "-l"])
505 retcode = apply(call, popenargs, kwargs)
506 cmd = kwargs.get("args")
510 raise CalledProcessError(retcode, cmd)
514 def list2cmdline(seq):
516 Translate a sequence of arguments into a command line
517 string, using the same rules as the MS C runtime:
519 1) Arguments are delimited by white space, which is either a
522 2) A string surrounded by double quotation marks is
523 interpreted as a single argument, regardless of white space
524 contained within. A quoted string can be embedded in an
527 3) A double quotation mark preceded by a backslash is
528 interpreted as a literal double quotation mark.
530 4) Backslashes are interpreted literally, unless they
531 immediately precede a double quotation mark.
533 5) If backslashes immediately precede a double quotation mark,
534 every pair of backslashes is interpreted as a literal
535 backslash. If the number of backslashes is odd, the last
536 backslash escapes the next double quotation mark as
541 # http://msdn.microsoft.com/library/en-us/vccelng/htm/progs_12.asp
547 # Add a space to separate this argument from the others
551 needquote = (" " in arg) or ("\t" in arg)
557 # Don't know if we need to double yet.
561 result.append('\\' * len(bs_buf)*2)
567 result.extend(bs_buf)
571 # Add remaining backspaces, if any.
573 result.extend(bs_buf)
576 result.extend(bs_buf)
579 return string.join(result, '')
589 def __init__(self, args, bufsize=0, executable=None,
590 stdin=None, stdout=None, stderr=None,
591 preexec_fn=None, close_fds=False, shell=False,
592 cwd=None, env=None, universal_newlines=False,
593 startupinfo=None, creationflags=0):
594 """Create new Popen instance."""
597 self._child_created = False
598 if not is_int_or_long(bufsize):
599 raise TypeError("bufsize must be an integer")
602 if preexec_fn is not None:
603 raise ValueError("preexec_fn is not supported on Windows "
606 raise ValueError("close_fds is not supported on Windows "
610 if startupinfo is not None:
611 raise ValueError("startupinfo is only supported on Windows "
613 if creationflags != 0:
614 raise ValueError("creationflags is only supported on Windows "
621 self.returncode = None
622 self.universal_newlines = universal_newlines
624 # Input and output objects. The general principle is like
629 # p2cwrite ---stdin---> p2cread
630 # c2pread <--stdout--- c2pwrite
631 # errread <--stderr--- errwrite
633 # On POSIX, the child objects are file descriptors. On
634 # Windows, these are Windows file handles. The parent objects
635 # are file descriptors on both platforms. The parent objects
636 # are None when not using PIPEs. The child objects are None
637 # when not redirecting.
641 errread, errwrite) = self._get_handles(stdin, stdout, stderr)
643 self._execute_child(args, executable, preexec_fn, close_fds,
644 cwd, env, universal_newlines,
645 startupinfo, creationflags, shell,
651 self.stdin = os.fdopen(p2cwrite, 'wb', bufsize)
653 if universal_newlines:
654 self.stdout = os.fdopen(c2pread, 'rU', bufsize)
656 self.stdout = os.fdopen(c2pread, 'rb', bufsize)
658 if universal_newlines:
659 self.stderr = os.fdopen(errread, 'rU', bufsize)
661 self.stderr = os.fdopen(errread, 'rb', bufsize)
664 def _translate_newlines(self, data):
665 data = data.replace("\r\n", "\n")
666 data = data.replace("\r", "\n")
671 if not self._child_created:
672 # We didn't get to successfully create a child process.
674 # In case the child hasn't been waited on, check if it's done.
675 self.poll(_deadstate=sys.maxint)
676 if self.returncode is None and _active is not None:
677 # Child is still running, keep us alive until we can wait on it.
681 def communicate(self, input=None):
682 """Interact with process: Send data to stdin. Read data from
683 stdout and stderr, until end-of-file is reached. Wait for
684 process to terminate. The optional input argument should be a
685 string to be sent to the child process, or None, if no data
686 should be sent to the child.
688 communicate() returns a tuple (stdout, stderr)."""
690 # Optimization: If we are only using one pipe, or no pipe at
691 # all, using select() or threads is unnecessary.
692 if [self.stdin, self.stdout, self.stderr].count(None) >= 2:
697 self.stdin.write(input)
700 stdout = self.stdout.read()
702 stderr = self.stderr.read()
704 return (stdout, stderr)
706 return self._communicate(input)
713 def _get_handles(self, stdin, stdout, stderr):
714 """Construct and return tupel with IO objects:
715 p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite
717 if stdin is None and stdout is None and stderr is None:
718 return (None, None, None, None, None, None)
720 p2cread, p2cwrite = None, None
721 c2pread, c2pwrite = None, None
722 errread, errwrite = None, None
725 p2cread = GetStdHandle(STD_INPUT_HANDLE)
727 p2cread, p2cwrite = CreatePipe(None, 0)
728 # Detach and turn into fd
729 p2cwrite = p2cwrite.Detach()
730 p2cwrite = msvcrt.open_osfhandle(p2cwrite, 0)
732 p2cread = msvcrt.get_osfhandle(stdin)
734 # Assuming file-like object
735 p2cread = msvcrt.get_osfhandle(stdin.fileno())
736 p2cread = self._make_inheritable(p2cread)
739 c2pwrite = GetStdHandle(STD_OUTPUT_HANDLE)
741 c2pread, c2pwrite = CreatePipe(None, 0)
742 # Detach and turn into fd
743 c2pread = c2pread.Detach()
744 c2pread = msvcrt.open_osfhandle(c2pread, 0)
746 c2pwrite = msvcrt.get_osfhandle(stdout)
748 # Assuming file-like object
749 c2pwrite = msvcrt.get_osfhandle(stdout.fileno())
750 c2pwrite = self._make_inheritable(c2pwrite)
753 errwrite = GetStdHandle(STD_ERROR_HANDLE)
755 errread, errwrite = CreatePipe(None, 0)
756 # Detach and turn into fd
757 errread = errread.Detach()
758 errread = msvcrt.open_osfhandle(errread, 0)
759 elif stderr == STDOUT:
762 errwrite = msvcrt.get_osfhandle(stderr)
764 # Assuming file-like object
765 errwrite = msvcrt.get_osfhandle(stderr.fileno())
766 errwrite = self._make_inheritable(errwrite)
768 return (p2cread, p2cwrite,
773 def _make_inheritable(self, handle):
774 """Return a duplicate of handle, which is inheritable"""
775 return DuplicateHandle(GetCurrentProcess(), handle,
776 GetCurrentProcess(), 0, 1,
777 DUPLICATE_SAME_ACCESS)
780 def _find_w9xpopen(self):
781 """Find and return absolut path to w9xpopen.exe"""
782 w9xpopen = os.path.join(os.path.dirname(GetModuleFileName(0)),
784 if not os.path.exists(w9xpopen):
785 # Eeek - file-not-found - possibly an embedding
786 # situation - see if we can locate it in sys.exec_prefix
787 w9xpopen = os.path.join(os.path.dirname(sys.exec_prefix),
789 if not os.path.exists(w9xpopen):
790 raise RuntimeError("Cannot locate w9xpopen.exe, which is "
791 "needed for Popen to work with your "
792 "shell or platform.")
796 def _execute_child(self, args, executable, preexec_fn, close_fds,
797 cwd, env, universal_newlines,
798 startupinfo, creationflags, shell,
802 """Execute program (MS Windows version)"""
804 if not isinstance(args, types.StringTypes):
805 args = list2cmdline(args)
807 # Process startup details
808 if startupinfo is None:
809 startupinfo = STARTUPINFO()
810 if None not in (p2cread, c2pwrite, errwrite):
811 startupinfo.dwFlags = startupinfo.dwFlags | STARTF_USESTDHANDLES
812 startupinfo.hStdInput = p2cread
813 startupinfo.hStdOutput = c2pwrite
814 startupinfo.hStdError = errwrite
817 startupinfo.dwFlags = startupinfo.dwFlags | STARTF_USESHOWWINDOW
818 startupinfo.wShowWindow = SW_HIDE
819 comspec = os.environ.get("COMSPEC", "cmd.exe")
820 args = comspec + " /c " + args
821 if (GetVersion() >= 0x80000000L or
822 os.path.basename(comspec).lower() == "command.com"):
823 # Win9x, or using command.com on NT. We need to
824 # use the w9xpopen intermediate program. For more
825 # information, see KB Q150956
826 # (http://web.archive.org/web/20011105084002/http://support.microsoft.com/support/kb/articles/Q150/9/56.asp)
827 w9xpopen = self._find_w9xpopen()
828 args = '"%s" %s' % (w9xpopen, args)
829 # Not passing CREATE_NEW_CONSOLE has been known to
830 # cause random failures on win9x. Specifically a
831 # dialog: "Your program accessed mem currently in
832 # use at xxx" and a hopeful warning about the
833 # stability of your system. Cost is Ctrl+C wont
835 creationflags = creationflags | CREATE_NEW_CONSOLE
839 hp, ht, pid, tid = CreateProcess(executable, args,
840 # no special security
842 # must inherit handles to pass std
849 except pywintypes.error, e:
850 # Translate pywintypes.error to WindowsError, which is
851 # a subclass of OSError. FIXME: We should really
852 # translate errno using _sys_errlist (or simliar), but
853 # how can this be done from Python?
854 raise apply(WindowsError, e.args)
856 # Retain the process handle, but close the thread handle
857 self._child_created = True
862 # Child is launched. Close the parent's copy of those pipe
863 # handles that only the child should have open. You need
864 # to make sure that no handles to the write end of the
865 # output pipe are maintained in this process or else the
866 # pipe will not close when the child process exits and the
867 # ReadFile will hang.
868 if p2cread is not None:
870 if c2pwrite is not None:
872 if errwrite is not None:
876 def poll(self, _deadstate=None):
877 """Check if child process has terminated. Returns returncode
879 if self.returncode is None:
880 if WaitForSingleObject(self._handle, 0) == WAIT_OBJECT_0:
881 self.returncode = GetExitCodeProcess(self._handle)
882 return self.returncode
886 """Wait for child process to terminate. Returns returncode
888 if self.returncode is None:
889 obj = WaitForSingleObject(self._handle, INFINITE)
890 self.returncode = GetExitCodeProcess(self._handle)
891 return self.returncode
894 def _readerthread(self, fh, buffer):
895 buffer.append(fh.read())
898 def _communicate(self, input):
899 stdout = None # Return
900 stderr = None # Return
904 stdout_thread = threading.Thread(target=self._readerthread,
905 args=(self.stdout, stdout))
906 stdout_thread.setDaemon(True)
907 stdout_thread.start()
910 stderr_thread = threading.Thread(target=self._readerthread,
911 args=(self.stderr, stderr))
912 stderr_thread.setDaemon(True)
913 stderr_thread.start()
916 if input is not None:
917 self.stdin.write(input)
925 # All data exchanged. Translate lists into strings.
926 if stdout is not None:
928 if stderr is not None:
931 # Translate newlines, if requested. We cannot let the file
932 # object do the translation: It is based on stdio, which is
933 # impossible to combine with select (unless forcing no
935 if self.universal_newlines and hasattr(file, 'newlines'):
937 stdout = self._translate_newlines(stdout)
939 stderr = self._translate_newlines(stderr)
942 return (stdout, stderr)
948 def _get_handles(self, stdin, stdout, stderr):
949 """Construct and return tupel with IO objects:
950 p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite
952 p2cread, p2cwrite = None, None
953 c2pread, c2pwrite = None, None
954 errread, errwrite = None, None
959 p2cread, p2cwrite = os.pipe()
963 # Assuming file-like object
964 p2cread = stdin.fileno()
969 c2pread, c2pwrite = os.pipe()
973 # Assuming file-like object
974 c2pwrite = stdout.fileno()
979 errread, errwrite = os.pipe()
980 elif stderr == STDOUT:
985 # Assuming file-like object
986 errwrite = stderr.fileno()
988 return (p2cread, p2cwrite,
993 def _set_cloexec_flag(self, fd):
995 cloexec_flag = fcntl.FD_CLOEXEC
996 except AttributeError:
999 old = fcntl.fcntl(fd, fcntl.F_GETFD)
1000 fcntl.fcntl(fd, fcntl.F_SETFD, old | cloexec_flag)
1003 def _close_fds(self, but):
1004 for i in xrange(3, MAXFD):
1009 except KeyboardInterrupt:
1010 raise # SCons: don't swallow keyboard interrupts
1015 def _execute_child(self, args, executable, preexec_fn, close_fds,
1016 cwd, env, universal_newlines,
1017 startupinfo, creationflags, shell,
1021 """Execute program (POSIX version)"""
1027 args = ["/bin/sh", "-c"] + args
1029 if executable is None:
1030 executable = args[0]
1032 # For transferring possible exec failure from child to parent
1033 # The first char specifies the exception type: 0 means
1034 # OSError, 1 means some other error.
1035 errpipe_read, errpipe_write = os.pipe()
1036 self._set_cloexec_flag(errpipe_write)
1038 self.pid = os.fork()
1039 self._child_created = True
1043 # Close parent's pipe ends
1050 os.close(errpipe_read)
1056 os.dup2(c2pwrite, 1)
1058 os.dup2(errwrite, 2)
1060 # Close pipe fds. Make sure we don't close the same
1061 # fd more than once, or standard fds.
1065 # Fall-back for earlier Python versions, so epydoc
1066 # can use this module directly to execute things.
1069 if c2pwrite and c2pwrite not in (p2cread,):
1071 if errwrite and errwrite not in (p2cread, c2pwrite):
1074 for fd in set((p2cread, c2pwrite, errwrite))-set((0,1,2)):
1077 # Close all other fds, if asked for
1079 self._close_fds(but=errpipe_write)
1088 os.execvp(executable, args)
1090 os.execvpe(executable, args, env)
1092 except KeyboardInterrupt:
1093 raise # SCons: don't swallow keyboard interrupts
1096 exc_type, exc_value, tb = sys.exc_info()
1097 # Save the traceback and attach it to the exception object
1098 exc_lines = traceback.format_exception(exc_type,
1101 exc_value.child_traceback = string.join(exc_lines, '')
1102 os.write(errpipe_write, pickle.dumps(exc_value))
1104 # This exitcode won't be reported to applications, so it
1105 # really doesn't matter what we return.
1109 os.close(errpipe_write)
1110 if p2cread and p2cwrite:
1112 if c2pwrite and c2pread:
1114 if errwrite and errread:
1117 # Wait for exec to fail or succeed; possibly raising exception
1118 data = os.read(errpipe_read, 1048576) # Exceptions limited to 1 MB
1119 os.close(errpipe_read)
1121 os.waitpid(self.pid, 0)
1122 child_exception = pickle.loads(data)
1123 raise child_exception
1126 def _handle_exitstatus(self, sts):
1127 if os.WIFSIGNALED(sts):
1128 self.returncode = -os.WTERMSIG(sts)
1129 elif os.WIFEXITED(sts):
1130 self.returncode = os.WEXITSTATUS(sts)
1132 # Should never happen
1133 raise RuntimeError("Unknown child exit status!")
1136 def poll(self, _deadstate=None):
1137 """Check if child process has terminated. Returns returncode
1139 if self.returncode is None:
1141 pid, sts = os.waitpid(self.pid, os.WNOHANG)
1143 self._handle_exitstatus(sts)
1145 if _deadstate is not None:
1146 self.returncode = _deadstate
1147 return self.returncode
1151 """Wait for child process to terminate. Returns returncode
1153 if self.returncode is None:
1154 pid, sts = os.waitpid(self.pid, 0)
1155 self._handle_exitstatus(sts)
1156 return self.returncode
1159 def _communicate(self, input):
1162 stdout = None # Return
1163 stderr = None # Return
1166 # Flush stdio buffer. This might block, if the user has
1167 # been writing to .stdin in an uncontrolled fashion.
1170 write_set.append(self.stdin)
1174 read_set.append(self.stdout)
1177 read_set.append(self.stderr)
1181 while read_set or write_set:
1182 rlist, wlist, xlist = select.select(read_set, write_set, [])
1184 if self.stdin in wlist:
1185 # When select has indicated that the file is writable,
1186 # we can write up to PIPE_BUF bytes without risk
1187 # blocking. POSIX defines PIPE_BUF >= 512
1188 bytes_written = os.write(self.stdin.fileno(), buffer(input, input_offset, 512))
1189 input_offset = input_offset + bytes_written
1190 if input_offset >= len(input):
1192 write_set.remove(self.stdin)
1194 if self.stdout in rlist:
1195 data = os.read(self.stdout.fileno(), 1024)
1198 read_set.remove(self.stdout)
1201 if self.stderr in rlist:
1202 data = os.read(self.stderr.fileno(), 1024)
1205 read_set.remove(self.stderr)
1208 # All data exchanged. Translate lists into strings.
1209 if stdout is not None:
1210 stdout = string.join(stdout, '')
1211 if stderr is not None:
1212 stderr = string.join(stderr, '')
1214 # Translate newlines, if requested. We cannot let the file
1215 # object do the translation: It is based on stdio, which is
1216 # impossible to combine with select (unless forcing no
1218 if self.universal_newlines and hasattr(file, 'newlines'):
1220 stdout = self._translate_newlines(stdout)
1222 stderr = self._translate_newlines(stderr)
1225 return (stdout, stderr)
1230 # Example 1: Simple redirection: Get process list
1232 plist = Popen(["ps"], stdout=PIPE).communicate()[0]
1233 print "Process list:"
1237 # Example 2: Change uid before executing child
1239 if os.getuid() == 0:
1240 p = Popen(["id"], preexec_fn=lambda: os.setuid(100))
1244 # Example 3: Connecting several subprocesses
1246 print "Looking for 'hda'..."
1247 p1 = Popen(["dmesg"], stdout=PIPE)
1248 p2 = Popen(["grep", "hda"], stdin=p1.stdout, stdout=PIPE)
1249 print repr(p2.communicate()[0])
1252 # Example 4: Catch execution error
1255 print "Trying a weird file..."
1257 print Popen(["/this/path/does/not/exist"]).communicate()
1259 if e.errno == errno.ENOENT:
1260 print "The file didn't exist. I thought so..."
1261 print "Child traceback:"
1262 print e.child_traceback
1264 print "Error", e.errno
1266 sys.stderr.write( "Gosh. No error.\n" )
1269 def _demo_windows():
1271 # Example 1: Connecting several subprocesses
1273 print "Looking for 'PROMPT' in set output..."
1274 p1 = Popen("set", stdout=PIPE, shell=True)
1275 p2 = Popen('find "PROMPT"', stdin=p1.stdout, stdout=PIPE)
1276 print repr(p2.communicate()[0])
1279 # Example 2: Simple execution of program
1281 print "Executing calc..."
1286 if __name__ == "__main__":