ÿØÿà JFIF    ÿÛ „ ( %!1!%*+...983,7(-.- PKF]~<2<2Open3.pmnu[package IPC::Open3; use strict; no strict 'refs'; # because users pass me bareword filehandles our ($VERSION, @ISA, @EXPORT); require Exporter; use Carp; use Symbol qw(gensym qualify); $VERSION = '1.20'; @ISA = qw(Exporter); @EXPORT = qw(open3); =head1 NAME IPC::Open3 - open a process for reading, writing, and error handling using open3() =head1 SYNOPSIS $pid = open3(\*CHLD_IN, \*CHLD_OUT, \*CHLD_ERR, 'some cmd and args', 'optarg', ...); my($wtr, $rdr, $err); use Symbol 'gensym'; $err = gensym; $pid = open3($wtr, $rdr, $err, 'some cmd and args', 'optarg', ...); waitpid( $pid, 0 ); my $child_exit_status = $? >> 8; =head1 DESCRIPTION Extremely similar to open2(), open3() spawns the given $cmd and connects CHLD_OUT for reading from the child, CHLD_IN for writing to the child, and CHLD_ERR for errors. If CHLD_ERR is false, or the same file descriptor as CHLD_OUT, then STDOUT and STDERR of the child are on the same filehandle (this means that an autovivified lexical cannot be used for the STDERR filehandle, see SYNOPSIS). The CHLD_IN will have autoflush turned on. If CHLD_IN begins with C<< <& >>, then CHLD_IN will be closed in the parent, and the child will read from it directly. If CHLD_OUT or CHLD_ERR begins with C<< >& >>, then the child will send output directly to that filehandle. In both cases, there will be a dup(2) instead of a pipe(2) made. If either reader or writer is the null string, this will be replaced by an autogenerated filehandle. If so, you must pass a valid lvalue in the parameter slot so it can be overwritten in the caller, or an exception will be raised. The filehandles may also be integers, in which case they are understood as file descriptors. open3() returns the process ID of the child process. It doesn't return on failure: it just raises an exception matching C. However, C failures in the child (such as no such file or permission denied), are just reported to CHLD_ERR under Windows and OS/2, as it is not possible to trap them. If the child process dies for any reason, the next write to CHLD_IN is likely to generate a SIGPIPE in the parent, which is fatal by default. So you may wish to handle this signal. Note if you specify C<-> as the command, in an analogous fashion to C the child process will just be the forked Perl process rather than an external command. This feature isn't yet supported on Win32 platforms. open3() does not wait for and reap the child process after it exits. Except for short programs where it's acceptable to let the operating system take care of this, you need to do this yourself. This is normally as simple as calling C when you're done with the process. Failing to do this can result in an accumulation of defunct or "zombie" processes. See L for more information. If you try to read from the child's stdout writer and their stderr writer, you'll have problems with blocking, which means you'll want to use select() or the IO::Select, which means you'd best use sysread() instead of readline() for normal stuff. This is very dangerous, as you may block forever. It assumes it's going to talk to something like B, both writing to it and reading from it. This is presumably safe because you "know" that commands like B will read a line at a time and output a line at a time. Programs like B that read their entire input stream first, however, are quite apt to cause deadlock. The big problem with this approach is that if you don't have control over source code being run in the child process, you can't control what it does with pipe buffering. Thus you can't just open a pipe to C and continually read and write a line from it. =head1 See Also =over 4 =item L Like Open3 but without STDERR capture. =item L This is a CPAN module that has better error handling and more facilities than Open3. =back =head1 WARNING The order of arguments differs from that of open2(). =cut # &open3: Marc Horowitz # derived mostly from &open2 by tom christiansen, # fixed for 5.001 by Ulrich Kunitz # ported to Win32 by Ron Schmidt, Merrill Lynch almost ended my career # fixed for autovivving FHs, tchrist again # allow fd numbers to be used, by Frank Tobin # allow '-' as command (c.f. open "-|"), by Adam Spiers # # usage: $pid = open3('wtr', 'rdr', 'err' 'some cmd and args', 'optarg', ...); # # spawn the given $cmd and connect rdr for # reading, wtr for writing, and err for errors. # if err is '', or the same as rdr, then stdout and # stderr of the child are on the same fh. returns pid # of child (or dies on failure). # if wtr begins with '<&', then wtr will be closed in the parent, and # the child will read from it directly. if rdr or err begins with # '>&', then the child will send output directly to that fd. In both # cases, there will be a dup() instead of a pipe() made. # WARNING: this is dangerous, as you may block forever # unless you are very careful. # # $wtr is left unbuffered. # # abort program if # rdr or wtr are null # a system call fails our $Me = 'open3 (bug)'; # you should never see this, it's always localized # Fatal.pm needs to be fixed WRT prototypes. sub xpipe { pipe $_[0], $_[1] or croak "$Me: pipe($_[0], $_[1]) failed: $!"; } # I tried using a * prototype character for the filehandle but it still # disallows a bareword while compiling under strict subs. sub xopen { open $_[0], $_[1], @_[2..$#_] and return; local $" = ', '; carp "$Me: open(@_) failed: $!"; } sub xclose { $_[0] =~ /\A=?(\d+)\z/ ? do { my $fh; open($fh, $_[1] . '&=' . $1) and close($fh); } : close $_[0] or croak "$Me: close($_[0]) failed: $!"; } sub xfileno { return $1 if $_[0] =~ /\A=?(\d+)\z/; # deal with fh just being an fd return fileno $_[0]; } use constant FORCE_DEBUG_SPAWN => 0; use constant DO_SPAWN => $^O eq 'os2' || $^O eq 'MSWin32' || FORCE_DEBUG_SPAWN; sub _open3 { local $Me = shift; # simulate autovivification of filehandles because # it's too ugly to use @_ throughout to make perl do it for us # tchrist 5-Mar-00 # Historically, open3(undef...) has silently worked, so keep # it working. splice @_, 0, 1, undef if \$_[0] == \undef; splice @_, 1, 1, undef if \$_[1] == \undef; unless (eval { $_[0] = gensym unless defined $_[0] && length $_[0]; $_[1] = gensym unless defined $_[1] && length $_[1]; 1; }) { # must strip crud for croak to add back, or looks ugly $@ =~ s/(?<=value attempted) at .*//s; croak "$Me: $@"; } my @handles = ({ mode => '<', handle => \*STDIN }, { mode => '>', handle => \*STDOUT }, { mode => '>', handle => \*STDERR }, ); foreach (@handles) { $_->{parent} = shift; $_->{open_as} = gensym; } if (@_ > 1 and $_[0] eq '-') { croak "Arguments don't make sense when the command is '-'" } $handles[2]{parent} ||= $handles[1]{parent}; $handles[2]{dup_of_out} = $handles[1]{parent} eq $handles[2]{parent}; my $package; foreach (@handles) { $_->{dup} = ($_->{parent} =~ s/^[<>]&//); if ($_->{parent} !~ /\A=?(\d+)\z/) { # force unqualified filehandles into caller's package $package //= caller 1; $_->{parent} = qualify $_->{parent}, $package; } next if $_->{dup} or $_->{dup_of_out}; if ($_->{mode} eq '<') { xpipe $_->{open_as}, $_->{parent}; } else { xpipe $_->{parent}, $_->{open_as}; } } my $kidpid; if (!DO_SPAWN) { # Used to communicate exec failures. xpipe my $stat_r, my $stat_w; $kidpid = fork; croak "$Me: fork failed: $!" unless defined $kidpid; if ($kidpid == 0) { # Kid eval { # A tie in the parent should not be allowed to cause problems. untie *STDIN; untie *STDOUT; untie *STDERR; close $stat_r; require Fcntl; my $flags = fcntl $stat_w, &Fcntl::F_GETFD, 0; croak "$Me: fcntl failed: $!" unless $flags; fcntl $stat_w, &Fcntl::F_SETFD, $flags|&Fcntl::FD_CLOEXEC or croak "$Me: fcntl failed: $!"; # If she wants to dup the kid's stderr onto her stdout I need to # save a copy of her stdout before I put something else there. if (!$handles[2]{dup_of_out} && $handles[2]{dup} && xfileno($handles[2]{parent}) == fileno \*STDOUT) { my $tmp = gensym; xopen($tmp, '>&', $handles[2]{parent}); $handles[2]{parent} = $tmp; } foreach (@handles) { if ($_->{dup_of_out}) { xopen \*STDERR, ">&STDOUT" if defined fileno STDERR && fileno STDERR != fileno STDOUT; } elsif ($_->{dup}) { xopen $_->{handle}, $_->{mode} . '&', $_->{parent} if fileno $_->{handle} != xfileno($_->{parent}); } else { xclose $_->{parent}, $_->{mode}; xopen $_->{handle}, $_->{mode} . '&=', fileno $_->{open_as}; } } return 1 if ($_[0] eq '-'); exec @_ or do { local($")=(" "); croak "$Me: exec of @_ failed: $!"; }; } and do { close $stat_w; return 0; }; my $bang = 0+$!; my $err = $@; utf8::encode $err if $] >= 5.008; print $stat_w pack('IIa*', $bang, length($err), $err); close $stat_w; eval { require POSIX; POSIX::_exit(255); }; exit 255; } else { # Parent close $stat_w; my $to_read = length(pack('I', 0)) * 2; my $bytes_read = read($stat_r, my $buf = '', $to_read); if ($bytes_read) { (my $bang, $to_read) = unpack('II', $buf); read($stat_r, my $err = '', $to_read); waitpid $kidpid, 0; # Reap child which should have exited if ($err) { utf8::decode $err if $] >= 5.008; } else { $err = "$Me: " . ($! = $bang); } $! = $bang; die($err); } } } else { # DO_SPAWN # All the bookkeeping of coincidence between handles is # handled in spawn_with_handles. my @close; foreach (@handles) { if ($_->{dup_of_out}) { $_->{open_as} = $handles[1]{open_as}; } elsif ($_->{dup}) { $_->{open_as} = $_->{parent} =~ /\A[0-9]+\z/ ? $_->{parent} : \*{$_->{parent}}; push @close, $_->{open_as}; } else { push @close, \*{$_->{parent}}, $_->{open_as}; } } require IO::Pipe; $kidpid = eval { spawn_with_handles(\@handles, \@close, @_); }; die "$Me: $@" if $@; } foreach (@handles) { next if $_->{dup} or $_->{dup_of_out}; xclose $_->{open_as}, $_->{mode}; } # If the write handle is a dup give it away entirely, close my copy # of it. xclose $handles[0]{parent}, $handles[0]{mode} if $handles[0]{dup}; select((select($handles[0]{parent}), $| = 1)[0]); # unbuffer pipe $kidpid; } sub open3 { if (@_ < 4) { local $" = ', '; croak "open3(@_): not enough arguments"; } return _open3 'open3', @_ } sub spawn_with_handles { my $fds = shift; # Fields: handle, mode, open_as my $close_in_child = shift; my ($fd, %saved, @errs); foreach $fd (@$fds) { $fd->{tmp_copy} = IO::Handle->new_from_fd($fd->{handle}, $fd->{mode}); $saved{fileno $fd->{handle}} = $fd->{tmp_copy} if $fd->{tmp_copy}; } foreach $fd (@$fds) { bless $fd->{handle}, 'IO::Handle' unless eval { $fd->{handle}->isa('IO::Handle') } ; # If some of handles to redirect-to coincide with handles to # redirect, we need to use saved variants: my $open_as = $fd->{open_as}; my $fileno = fileno($open_as); $fd->{handle}->fdopen(defined($fileno) ? $saved{$fileno} || $open_as : $open_as, $fd->{mode}); } unless ($^O eq 'MSWin32') { require Fcntl; # Stderr may be redirected below, so we save the err text: foreach $fd (@$close_in_child) { next unless fileno $fd; fcntl($fd, Fcntl::F_SETFD(), 1) or push @errs, "fcntl $fd: $!" unless $saved{fileno $fd}; # Do not close what we redirect! } } my $pid; unless (@errs) { if (FORCE_DEBUG_SPAWN) { pipe my $r, my $w or die "Pipe failed: $!"; $pid = fork; die "Fork failed: $!" unless defined $pid; if (!$pid) { { no warnings; exec @_ } print $w 0 + $!; close $w; require POSIX; POSIX::_exit(255); } close $w; my $bad = <$r>; if (defined $bad) { $! = $bad; undef $pid; } } else { $pid = eval { system 1, @_ }; # 1 == P_NOWAIT } if($@) { push @errs, "IO::Pipe: Can't spawn-NOWAIT: $@"; } elsif(!$pid || $pid < 0) { push @errs, "IO::Pipe: Can't spawn-NOWAIT: $!"; } } # Do this in reverse, so that STDERR is restored first: foreach $fd (reverse @$fds) { $fd->{handle}->fdopen($fd->{tmp_copy}, $fd->{mode}); } foreach (values %saved) { $_->close or croak "Can't close: $!"; } croak join "\n", @errs if @errs; return $pid; } 1; # so require is happy PKF]/~Open2.pmnu[package IPC::Open2; use strict; our ($VERSION, @ISA, @EXPORT); require 5.000; require Exporter; $VERSION = 1.04; @ISA = qw(Exporter); @EXPORT = qw(open2); =head1 NAME IPC::Open2 - open a process for both reading and writing using open2() =head1 SYNOPSIS use IPC::Open2; $pid = open2(\*CHLD_OUT, \*CHLD_IN, 'some cmd and args'); # or without using the shell $pid = open2(\*CHLD_OUT, \*CHLD_IN, 'some', 'cmd', 'and', 'args'); # or with handle autovivification my($chld_out, $chld_in); $pid = open2($chld_out, $chld_in, 'some cmd and args'); # or without using the shell $pid = open2($chld_out, $chld_in, 'some', 'cmd', 'and', 'args'); waitpid( $pid, 0 ); my $child_exit_status = $? >> 8; =head1 DESCRIPTION The open2() function runs the given $cmd and connects $chld_out for reading and $chld_in for writing. It's what you think should work when you try $pid = open(HANDLE, "|cmd args|"); The write filehandle will have autoflush turned on. If $chld_out is a string (that is, a bareword filehandle rather than a glob or a reference) and it begins with C<< >& >>, then the child will send output directly to that file handle. If $chld_in is a string that begins with C<< <& >>, then $chld_in will be closed in the parent, and the child will read from it directly. In both cases, there will be a dup(2) instead of a pipe(2) made. If either reader or writer is the null string, this will be replaced by an autogenerated filehandle. If so, you must pass a valid lvalue in the parameter slot so it can be overwritten in the caller, or an exception will be raised. open2() returns the process ID of the child process. It doesn't return on failure: it just raises an exception matching C. However, C failures in the child are not detected. You'll have to trap SIGPIPE yourself. open2() does not wait for and reap the child process after it exits. Except for short programs where it's acceptable to let the operating system take care of this, you need to do this yourself. This is normally as simple as calling C when you're done with the process. Failing to do this can result in an accumulation of defunct or "zombie" processes. See L for more information. This whole affair is quite dangerous, as you may block forever. It assumes it's going to talk to something like B, both writing to it and reading from it. This is presumably safe because you "know" that commands like B will read a line at a time and output a line at a time. Programs like B that read their entire input stream first, however, are quite apt to cause deadlock. The big problem with this approach is that if you don't have control over source code being run in the child process, you can't control what it does with pipe buffering. Thus you can't just open a pipe to C and continually read and write a line from it. The IO::Pty and Expect modules from CPAN can help with this, as they provide a real tty (well, a pseudo-tty, actually), which gets you back to line buffering in the invoked command again. =head1 WARNING The order of arguments differs from that of open3(). =head1 SEE ALSO See L for an alternative that handles STDERR as well. This function is really just a wrapper around open3(). =cut # &open2: tom christiansen, # # usage: $pid = open2('rdr', 'wtr', 'some cmd and args'); # or $pid = open2('rdr', 'wtr', 'some', 'cmd', 'and', 'args'); # # spawn the given $cmd and connect $rdr for # reading and $wtr for writing. return pid # of child, or 0 on failure. # # WARNING: this is dangerous, as you may block forever # unless you are very careful. # # $wtr is left unbuffered. # # abort program if # rdr or wtr are null # a system call fails require IPC::Open3; sub open2 { local $Carp::CarpLevel = $Carp::CarpLevel + 1; return IPC::Open3::_open3('open2', $_[1], $_[0], '>&STDERR', @_[2 .. $#_]); } 1 PKS]L.2.2Driver/Files.pmnu[package Test2::IPC::Driver::Files; use strict; use warnings; our $VERSION = '1.302135'; BEGIN { require Test2::IPC::Driver; our @ISA = qw(Test2::IPC::Driver) } use Test2::Util::HashBase qw{tempdir event_ids read_ids timeouts tid pid globals}; use Scalar::Util qw/blessed/; use File::Temp(); use Storable(); use File::Spec(); use POSIX(); use Test2::Util qw/try get_tid pkg_to_file IS_WIN32 ipc_separator do_rename do_unlink try_sig_mask/; use Test2::API qw/test2_ipc_set_pending/; sub use_shm { 1 } sub shm_size() { 64 } sub is_viable { 1 } sub init { my $self = shift; my $tmpdir = File::Temp::tempdir( $ENV{T2_TEMPDIR_TEMPLATE} || "test2" . ipc_separator . $$ . ipc_separator . "XXXXXX", CLEANUP => 0, TMPDIR => 1, ); $self->abort_trace("Could not get a temp dir") unless $tmpdir; $self->{+TEMPDIR} = File::Spec->canonpath($tmpdir); print STDERR "\nIPC Temp Dir: $tmpdir\n\n" if $ENV{T2_KEEP_TEMPDIR}; $self->{+EVENT_IDS} = {}; $self->{+READ_IDS} = {}; $self->{+TIMEOUTS} = {}; $self->{+TID} = get_tid(); $self->{+PID} = $$; $self->{+GLOBALS} = {}; return $self; } sub hub_file { my $self = shift; my ($hid) = @_; my $tdir = $self->{+TEMPDIR}; return File::Spec->catfile($tdir, "HUB" . ipc_separator . $hid); } sub event_file { my $self = shift; my ($hid, $e) = @_; my $tempdir = $self->{+TEMPDIR}; my $type = blessed($e) or $self->abort("'$e' is not a blessed object!"); $self->abort("'$e' is not an event object!") unless $type->isa('Test2::Event'); my $tid = get_tid(); my $eid = $self->{+EVENT_IDS}->{$hid}->{$$}->{$tid} += 1; my @type = split '::', $type; my $name = join(ipc_separator, $hid, $$, $tid, $eid, @type); return File::Spec->catfile($tempdir, $name); } sub add_hub { my $self = shift; my ($hid) = @_; my $hfile = $self->hub_file($hid); $self->abort_trace("File for hub '$hid' already exists") if -e $hfile; open(my $fh, '>', $hfile) or $self->abort_trace("Could not create hub file '$hid': $!"); print $fh "$$\n" . get_tid() . "\n"; close($fh); } sub drop_hub { my $self = shift; my ($hid) = @_; my $tdir = $self->{+TEMPDIR}; my $hfile = $self->hub_file($hid); $self->abort_trace("File for hub '$hid' does not exist") unless -e $hfile; open(my $fh, '<', $hfile) or $self->abort_trace("Could not open hub file '$hid': $!"); my ($pid, $tid) = <$fh>; close($fh); $self->abort_trace("A hub file can only be closed by the process that started it\nExpected $pid, got $$") unless $pid == $$; $self->abort_trace("A hub file can only be closed by the thread that started it\nExpected $tid, got " . get_tid()) unless get_tid() == $tid; if ($ENV{T2_KEEP_TEMPDIR}) { my ($ok, $err) = do_rename($hfile, File::Spec->canonpath("$hfile.complete")); $self->abort_trace("Could not rename file '$hfile' -> '$hfile.complete': $err") unless $ok } else { my ($ok, $err) = do_unlink($hfile); $self->abort_trace("Could not remove file for hub '$hid': $err") unless $ok } opendir(my $dh, $tdir) or $self->abort_trace("Could not open temp dir!"); for my $file (readdir($dh)) { next if $file =~ m{\.complete$}; next unless $file =~ m{^$hid}; $self->abort_trace("Not all files from hub '$hid' have been collected!"); } closedir($dh); } sub send { my $self = shift; my ($hid, $e, $global) = @_; my $tempdir = $self->{+TEMPDIR}; my $hfile = $self->hub_file($hid); my $dest = $global ? 'GLOBAL' : $hid; $self->abort(<<" EOT") unless $global || -f $hfile; hub '$hid' is not available, failed to send event! There was an attempt to send an event to a hub in a parent process or thread, but that hub appears to be gone. This can happen if you fork, or start a new thread from inside subtest, and the parent finishes the subtest before the child returns. This can also happen if the parent process is done testing before the child finishes. Test2 normally waits automatically in the root process, but will not do so if Test::Builder is loaded for legacy reasons. EOT my $file = $self->event_file($dest, $e); my $ready = File::Spec->canonpath("$file.ready"); if ($global) { my $name = $ready; $name =~ s{^.*(GLOBAL)}{GLOBAL}; $self->{+GLOBALS}->{$hid}->{$name}++; } # Write and rename the file. my ($ren_ok, $ren_err); my ($ok, $err) = try_sig_mask { Storable::store($e, $file); ($ren_ok, $ren_err) = do_rename("$file", $ready); }; if ($ok) { $self->abort("Could not rename file '$file' -> '$ready': $ren_err") unless $ren_ok; test2_ipc_set_pending(substr($file, -(shm_size))); } else { my $src_file = __FILE__; $err =~ s{ at \Q$src_file\E.*$}{}; chomp($err); my $tid = get_tid(); my $trace = $e->trace->debug; my $type = blessed($e); $self->abort(<<" EOT"); ******************************************************************************* There was an error writing an event: Destination: $dest Origin PID: $$ Origin TID: $tid Event Type: $type Event Trace: $trace File Name: $file Ready Name: $ready Error: $err ******************************************************************************* EOT } return 1; } sub driver_abort { my $self = shift; my ($msg) = @_; local ($@, $!, $?, $^E); eval { my $abort = File::Spec->catfile($self->{+TEMPDIR}, "ABORT"); open(my $fh, '>>', $abort) or die "Could not open abort file: $!"; print $fh $msg, "\n"; close($fh) or die "Could not close abort file: $!"; 1; } or warn $@; } sub cull { my $self = shift; my ($hid) = @_; my $tempdir = $self->{+TEMPDIR}; opendir(my $dh, $tempdir) or $self->abort("could not open IPC temp dir ($tempdir)!"); my $read = $self->{+READ_IDS}; my $timeouts = $self->{+TIMEOUTS}; my @out; for my $info (sort cmp_events map { $self->should_read_event($hid, $_) } readdir($dh)) { unless ($info->{global}) { my $next = $self->{+READ_IDS}->{$info->{hid}}->{$info->{pid}}->{$info->{tid}} ||= 1; $timeouts->{$info->{file}} ||= time; if ($next != $info->{eid}) { # Wait up to N seconds for missing events next unless 5 < time - $timeouts->{$info->{file}}; $self->abort("Missing event HID: $info->{hid}, PID: $info->{pid}, TID: $info->{tid}, EID: $info->{eid}."); } $self->{+READ_IDS}->{$info->{hid}}->{$info->{pid}}->{$info->{tid}} = $info->{eid} + 1; } my $full = $info->{full_path}; my $obj = $self->read_event_file($full); push @out => $obj; # Do not remove global events next if $info->{global}; if ($ENV{T2_KEEP_TEMPDIR}) { my $complete = File::Spec->canonpath("$full.complete"); my ($ok, $err) = do_rename($full, $complete); $self->abort("Could not rename IPC file '$full', '$complete': $err") unless $ok; } else { my ($ok, $err) = do_unlink("$full"); $self->abort("Could not unlink IPC file '$full': $err") unless $ok; } } closedir($dh); return @out; } sub parse_event_filename { my $self = shift; my ($file) = @_; # The || is to force 0 in false my $complete = substr($file, -9, 9) eq '.complete' || 0 and substr($file, -9, 9, ""); my $ready = substr($file, -6, 6) eq '.ready' || 0 and substr($file, -6, 6, ""); my @parts = split ipc_separator, $file; my ($global, $hid) = $parts[0] eq 'GLOBAL' ? (1, shift @parts) : (0, join ipc_separator, splice(@parts, 0, 4)); my ($pid, $tid, $eid) = splice(@parts, 0, 3); my $type = join '::' => @parts; return { file => $file, ready => $ready, complete => $complete, global => $global, type => $type, hid => $hid, pid => $pid, tid => $tid, eid => $eid, }; } sub should_read_event { my $self = shift; my ($hid, $file) = @_; return if substr($file, 0, 1) eq '.'; return if substr($file, 0, 3) eq 'HUB'; CORE::exit(255) if $file eq 'ABORT'; my $parsed = $self->parse_event_filename($file); return if $parsed->{complete}; return unless $parsed->{ready}; return unless $parsed->{global} || $parsed->{hid} eq $hid; return if $parsed->{global} && $self->{+GLOBALS}->{$hid}->{$file}++; # Untaint the path. my $full = File::Spec->catfile($self->{+TEMPDIR}, $file); ($full) = ($full =~ m/^(.*)$/gs) if ${^TAINT}; $parsed->{full_path} = $full; return $parsed; } sub cmp_events { # Globals first return -1 if $a->{global} && !$b->{global}; return 1 if $b->{global} && !$a->{global}; return $a->{pid} <=> $b->{pid} || $a->{tid} <=> $b->{tid} || $a->{eid} <=> $b->{eid}; } sub read_event_file { my $self = shift; my ($file) = @_; my $obj = Storable::retrieve($file); $self->abort("Got an unblessed object: '$obj'") unless blessed($obj); unless ($obj->isa('Test2::Event')) { my $pkg = blessed($obj); my $mod_file = pkg_to_file($pkg); my ($ok, $err) = try { require $mod_file }; $self->abort("Event has unknown type ($pkg), tried to load '$mod_file' but failed: $err") unless $ok; $self->abort("'$obj' is not a 'Test2::Event' object") unless $obj->isa('Test2::Event'); } return $obj; } sub waiting { my $self = shift; require Test2::Event::Waiting; $self->send( GLOBAL => Test2::Event::Waiting->new( trace => Test2::EventFacet::Trace->new(frame => [caller()]), ), 'GLOBAL' ); return; } sub DESTROY { my $self = shift; return unless defined $self->pid; return unless defined $self->tid; return unless $$ == $self->pid; return unless get_tid() == $self->tid; my $tempdir = $self->{+TEMPDIR}; my $aborted = 0; my $abort_file = File::Spec->catfile($self->{+TEMPDIR}, "ABORT"); if (-e $abort_file) { $aborted = 1; my ($ok, $err) = do_unlink($abort_file); warn $err unless $ok; } opendir(my $dh, $tempdir) or $self->abort("Could not open temp dir! ($tempdir)"); while(my $file = readdir($dh)) { next if $file =~ m/^\.+$/; next if $file =~ m/\.complete$/; my $full = File::Spec->catfile($tempdir, $file); my $sep = ipc_separator; if ($aborted || $file =~ m/^(GLOBAL|HUB$sep)/) { $full =~ m/^(.*)$/; $full = $1; # Untaint it next if $ENV{T2_KEEP_TEMPDIR}; my ($ok, $err) = do_unlink($full); $self->abort("Could not unlink IPC file '$full': $err") unless $ok; next; } $self->abort("Leftover files in the directory ($full)!\n"); } closedir($dh); if ($ENV{T2_KEEP_TEMPDIR}) { print STDERR "# Not removing temp dir: $tempdir\n"; return; } my $abort = File::Spec->catfile($self->{+TEMPDIR}, "ABORT"); unlink($abort) if -e $abort; rmdir($tempdir) or warn "Could not remove IPC temp dir ($tempdir)"; } 1; __END__ =pod =encoding UTF-8 =head1 NAME Test2::IPC::Driver::Files - Temp dir + Files concurrency model. =head1 DESCRIPTION This is the default, and fallback concurrency model for L. This sends events between processes and threads using serialized files in a temporary directory. This is not particularly fast, but it works everywhere. =head1 SYNOPSIS use Test2::IPC::Driver::Files; # IPC is now enabled =head1 ENVIRONMENT VARIABLES =over 4 =item T2_KEEP_TEMPDIR=0 When true, the tempdir used by the IPC driver will not be deleted when the test is done. =item T2_TEMPDIR_TEMPLATE='test2-XXXXXX' This can be used to set the template for the IPC temp dir. The template should follow template specifications from L. =back =head1 SEE ALSO See L for methods. =head1 SOURCE The source code repository for Test2 can be found at F. =head1 MAINTAINERS =over 4 =item Chad Granum Eexodist@cpan.orgE =back =head1 AUTHORS =over 4 =item Chad Granum Eexodist@cpan.orgE =back =head1 COPYRIGHT Copyright 2018 Chad Granum Eexodist@cpan.orgE. This program is free software; you can redistribute it and/or modify it under the same terms as Perl itself. See F =cut PKS]i\}} Driver.pmnu[package Test2::IPC::Driver; use strict; use warnings; our $VERSION = '1.302135'; use Carp qw/confess/; use Test2::Util::HashBase qw{no_fatal no_bail}; use Test2::API qw/test2_ipc_add_driver/; my %ADDED; sub import { my $class = shift; return if $class eq __PACKAGE__; return if $ADDED{$class}++; test2_ipc_add_driver($class); } sub use_shm { 0 } for my $meth (qw/send cull add_hub drop_hub waiting is_viable/) { no strict 'refs'; *$meth = sub { my $thing = shift; confess "'$thing' did not define the required method '$meth'." }; } # Print the error and call exit. We are not using 'die' cause this is a # catastrophic error that should never be caught. If we get here it # means some serious shit has happened in a child process, the only way # to inform the parent may be to exit false. sub abort { my $self = shift; chomp(my ($msg) = @_); $self->driver_abort($msg) if $self->can('driver_abort'); print STDERR "IPC Fatal Error: $msg\n"; print STDOUT "Bail out! IPC Fatal Error: $msg\n" unless $self->no_bail; CORE::exit(255) unless $self->no_fatal; } sub abort_trace { my $self = shift; my ($msg) = @_; # Older versions of Carp do not export longmess() function, so it needs to be called with package name $self->abort(Carp::longmess($msg)); } 1; __END__ =pod =encoding UTF-8 =head1 NAME Test2::IPC::Driver - Base class for Test2 IPC drivers. =head1 SYNOPSIS package Test2::IPC::Driver::MyDriver; use base 'Test2::IPC::Driver'; ... =head1 METHODS =over 4 =item $self->abort($msg) If an IPC encounters a fatal error it should use this. This will print the message to STDERR with C<'IPC Fatal Error: '> prefixed to it, then it will forcefully exit 255. IPC errors may occur in threads or processes other than the main one, this method provides the best chance of the harness noticing the error. =item $self->abort_trace($msg) This is the same as C<< $ipc->abort($msg) >> except that it uses C to add a stack trace to the message. =item $false = $self->use_shm The base class always returns false for this method. You may override it if you wish to use the SHM made available in L/L. =back =head1 LOADING DRIVERS Test2::IPC::Driver has an C method. All drivers inherit this import method. This import method registers the driver. In most cases you just need to load the desired IPC driver to make it work. You should load this driver as early as possible. A warning will be issued if you load it too late for it to be effective. use Test2::IPC::Driver::MyDriver; ... =head1 WRITING DRIVERS package Test2::IPC::Driver::MyDriver; use strict; use warnings; use base 'Test2::IPC::Driver'; sub is_viable { return 0 if $^O eq 'win32'; # Will not work on windows. return 1; } sub add_hub { my $self = shift; my ($hid) = @_; ... # Make it possible to contact the hub } sub drop_hub { my $self = shift; my ($hid) = @_; ... # Nothing should try to reach the hub anymore. } sub send { my $self = shift; my ($hid, $e, $global) = @_; ... # Send the event to the proper hub. # If you are using the SHM you should notify other procs/threads that # there is a pending event. Test2::API::test2_ipc_set_pending($uniq_val); } sub cull { my $self = shift; my ($hid) = @_; my @events = ...; # Here is where you get the events for the hub return @events; } sub waiting { my $self = shift; ... # Notify all listening procs and threads that the main ... # process/thread is waiting for them to finish. } 1; =head2 METHODS SUBCLASSES MUST IMPLEMENT =over 4 =item $ipc->is_viable This should return true if the driver works in the current environment. This should return false if it does not. This is a CLASS method. =item $ipc->add_hub($hid) This is used to alert the driver that a new hub is expecting events. The driver should keep track of the process and thread ids, the hub should only be dropped by the proc+thread that started it. sub add_hub { my $self = shift; my ($hid) = @_; ... # Make it possible to contact the hub } =item $ipc->drop_hub($hid) This is used to alert the driver that a hub is no longer accepting events. The driver should keep track of the process and thread ids, the hub should only be dropped by the proc+thread that started it (This is the drivers responsibility to enforce). sub drop_hub { my $self = shift; my ($hid) = @_; ... # Nothing should try to reach the hub anymore. } =item $ipc->send($hid, $event); =item $ipc->send($hid, $event, $global); Used to send events from the current process/thread to the specified hub in its process+thread. sub send { my $self = shift; my ($hid, $e) = @_; ... # Send the event to the proper hub. # If you are using the SHM you should notify other procs/threads that # there is a pending event. Test2::API::test2_ipc_set_pending($uniq_val); } If C<$global> is true then the driver should send the event to all hubs in all processes and threads. =item @events = $ipc->cull($hid) Used to collect events that have been sent to the specified hub. sub cull { my $self = shift; my ($hid) = @_; my @events = ...; # Here is where you get the events for the hub return @events; } =item $ipc->waiting() This is called in the parent process when it is complete and waiting for all child processes and threads to complete. sub waiting { my $self = shift; ... # Notify all listening procs and threads that the main ... # process/thread is waiting for them to finish. } =back =head2 METHODS SUBCLASSES MAY IMPLEMENT OR OVERRIDE =over 4 =item $ipc->driver_abort($msg) This is a hook called by C<< Test2::IPC::Driver->abort() >>. This is your chance to cleanup when an abort happens. You cannot prevent the abort, but you can gracefully except it. =item $bool = $ipc->use_shm() True if you want to make use of the L/L SHM. =item $bites = $ipc->shm_size() Use this to customize the size of the SHM space. There are no guarantees about what the size will be if you do not implement this. =back =head1 SOURCE The source code repository for Test2 can be found at F. =head1 MAINTAINERS =over 4 =item Chad Granum Eexodist@cpan.orgE =back =head1 AUTHORS =over 4 =item Chad Granum Eexodist@cpan.orgE =back =head1 COPYRIGHT Copyright 2018 Chad Granum Eexodist@cpan.orgE. This program is free software; you can redistribute it and/or modify it under the same terms as Perl itself. See F =cut PKu]Cmd.pmnu[package IPC::Cmd; use strict; BEGIN { use constant IS_VMS => $^O eq 'VMS' ? 1 : 0; use constant IS_WIN32 => $^O eq 'MSWin32' ? 1 : 0; use constant IS_HPUX => $^O eq 'hpux' ? 1 : 0; use constant IS_WIN98 => (IS_WIN32 and !Win32::IsWinNT()) ? 1 : 0; use constant ALARM_CLASS => __PACKAGE__ . '::TimeOut'; use constant SPECIAL_CHARS => qw[< > | &]; use constant QUOTE => do { IS_WIN32 ? q["] : q['] }; use Exporter (); use vars qw[ @ISA $VERSION @EXPORT_OK $VERBOSE $DEBUG $USE_IPC_RUN $USE_IPC_OPEN3 $CAN_USE_RUN_FORKED $WARN $INSTANCES $ALLOW_NULL_ARGS $HAVE_MONOTONIC ]; $VERSION = '1.02'; $VERBOSE = 0; $DEBUG = 0; $WARN = 1; $USE_IPC_RUN = IS_WIN32 && !IS_WIN98; $USE_IPC_OPEN3 = not IS_VMS; $ALLOW_NULL_ARGS = 0; $CAN_USE_RUN_FORKED = 0; eval { require POSIX; POSIX->import(); require IPC::Open3; IPC::Open3->import(); require IO::Select; IO::Select->import(); require IO::Handle; IO::Handle->import(); require FileHandle; FileHandle->import(); require Socket; require Time::HiRes; Time::HiRes->import(); require Win32 if IS_WIN32; }; $CAN_USE_RUN_FORKED = $@ || !IS_VMS && !IS_WIN32; eval { my $wait_start_time = Time::HiRes::clock_gettime(&Time::HiRes::CLOCK_MONOTONIC); }; if ($@) { $HAVE_MONOTONIC = 0; } else { $HAVE_MONOTONIC = 1; } @ISA = qw[Exporter]; @EXPORT_OK = qw[can_run run run_forked QUOTE]; } require Carp; use File::Spec; use Params::Check qw[check]; use Text::ParseWords (); # import ONLY if needed! use Module::Load::Conditional qw[can_load]; use Locale::Maketext::Simple Style => 'gettext'; local $Module::Load::Conditional::FORCE_SAFE_INC = 1; =pod =head1 NAME IPC::Cmd - finding and running system commands made easy =head1 SYNOPSIS use IPC::Cmd qw[can_run run run_forked]; my $full_path = can_run('wget') or warn 'wget is not installed!'; ### commands can be arrayrefs or strings ### my $cmd = "$full_path -b theregister.co.uk"; my $cmd = [$full_path, '-b', 'theregister.co.uk']; ### in scalar context ### my $buffer; if( scalar run( command => $cmd, verbose => 0, buffer => \$buffer, timeout => 20 ) ) { print "fetched webpage successfully: $buffer\n"; } ### in list context ### my( $success, $error_message, $full_buf, $stdout_buf, $stderr_buf ) = run( command => $cmd, verbose => 0 ); if( $success ) { print "this is what the command printed:\n"; print join "", @$full_buf; } ### run_forked example ### my $result = run_forked("$full_path -q -O - theregister.co.uk", {'timeout' => 20}); if ($result->{'exit_code'} eq 0 && !$result->{'timeout'}) { print "this is what wget returned:\n"; print $result->{'stdout'}; } ### check for features print "IPC::Open3 available: " . IPC::Cmd->can_use_ipc_open3; print "IPC::Run available: " . IPC::Cmd->can_use_ipc_run; print "Can capture buffer: " . IPC::Cmd->can_capture_buffer; ### don't have IPC::Cmd be verbose, ie don't print to stdout or ### stderr when running commands -- default is '0' $IPC::Cmd::VERBOSE = 0; =head1 DESCRIPTION IPC::Cmd allows you to run commands platform independently, interactively if desired, but have them still work. The C function can tell you if a certain binary is installed and if so where, whereas the C function can actually execute any of the commands you give it and give you a clear return value, as well as adhere to your verbosity settings. =head1 CLASS METHODS =head2 $ipc_run_version = IPC::Cmd->can_use_ipc_run( [VERBOSE] ) Utility function that tells you if C is available. If the C flag is passed, it will print diagnostic messages if L can not be found or loaded. =cut sub can_use_ipc_run { my $self = shift; my $verbose = shift || 0; ### IPC::Run doesn't run on win98 return if IS_WIN98; ### if we don't have ipc::run, we obviously can't use it. return unless can_load( modules => { 'IPC::Run' => '0.55' }, verbose => ($WARN && $verbose), ); ### otherwise, we're good to go return $IPC::Run::VERSION; } =head2 $ipc_open3_version = IPC::Cmd->can_use_ipc_open3( [VERBOSE] ) Utility function that tells you if C is available. If the verbose flag is passed, it will print diagnostic messages if C can not be found or loaded. =cut sub can_use_ipc_open3 { my $self = shift; my $verbose = shift || 0; ### IPC::Open3 is not working on VMS because of a lack of fork. return if IS_VMS; ### IPC::Open3 works on every non-VMS platform, but it can't ### capture buffers on win32 :( return unless can_load( modules => { map {$_ => '0.0'} qw|IPC::Open3 IO::Select Symbol| }, verbose => ($WARN && $verbose), ); return $IPC::Open3::VERSION; } =head2 $bool = IPC::Cmd->can_capture_buffer Utility function that tells you if C is capable of capturing buffers in it's current configuration. =cut sub can_capture_buffer { my $self = shift; return 1 if $USE_IPC_RUN && $self->can_use_ipc_run; return 1 if $USE_IPC_OPEN3 && $self->can_use_ipc_open3; return; } =head2 $bool = IPC::Cmd->can_use_run_forked Utility function that tells you if C is capable of providing C on the current platform. =head1 FUNCTIONS =head2 $path = can_run( PROGRAM ); C takes only one argument: the name of a binary you wish to locate. C works much like the unix binary C or the bash command C, which scans through your path, looking for the requested binary. Unlike C and C, this function is platform independent and will also work on, for example, Win32. If called in a scalar context it will return the full path to the binary you asked for if it was found, or C if it was not. If called in a list context and the global variable C<$INSTANCES> is a true value, it will return a list of the full paths to instances of the binary where found in C, or an empty list if it was not found. =cut sub can_run { my $command = shift; # a lot of VMS executables have a symbol defined # check those first if ( $^O eq 'VMS' ) { require VMS::DCLsym; my $syms = VMS::DCLsym->new; return $command if scalar $syms->getsym( uc $command ); } require File::Spec; require ExtUtils::MM::Utils; my @possibles; if( File::Spec->file_name_is_absolute($command) ) { return MM->maybe_command($command); } else { for my $dir ( File::Spec->path, ( IS_WIN32 ? File::Spec->curdir : () ) ) { next if ! $dir || ! -d $dir; my $abs = File::Spec->catfile( IS_WIN32 ? Win32::GetShortPathName( $dir ) : $dir, $command); push @possibles, $abs if $abs = MM->maybe_command($abs); } } return @possibles if wantarray and $INSTANCES; return shift @possibles; } =head2 $ok | ($ok, $err, $full_buf, $stdout_buff, $stderr_buff) = run( command => COMMAND, [verbose => BOOL, buffer => \$SCALAR, timeout => DIGIT] ); C takes 4 arguments: =over 4 =item command This is the command to execute. It may be either a string or an array reference. This is a required argument. See L<"Caveats"> for remarks on how commands are parsed and their limitations. =item verbose This controls whether all output of a command should also be printed to STDOUT/STDERR or should only be trapped in buffers (NOTE: buffers require L to be installed, or your system able to work with L). It will default to the global setting of C<$IPC::Cmd::VERBOSE>, which by default is 0. =item buffer This will hold all the output of a command. It needs to be a reference to a scalar. Note that this will hold both the STDOUT and STDERR messages, and you have no way of telling which is which. If you require this distinction, run the C command in list context and inspect the individual buffers. Of course, this requires that the underlying call supports buffers. See the note on buffers above. =item timeout Sets the maximum time the command is allowed to run before aborting, using the built-in C call. If the timeout is triggered, the C in the return value will be set to an object of the C class. See the L<"error message"> section below for details. Defaults to C<0>, meaning no timeout is set. =back C will return a simple C or C when called in scalar context. In list context, you will be returned a list of the following items: =over 4 =item success A simple boolean indicating if the command executed without errors or not. =item error message If the first element of the return value (C) was 0, then some error occurred. This second element is the error message the command you requested exited with, if available. This is generally a pretty printed value of C<$?> or C<$@>. See C for details on what they can contain. If the error was a timeout, the C will be prefixed with the string C, the timeout class. =item full_buffer This is an array reference containing all the output the command generated. Note that buffers are only available if you have L installed, or if your system is able to work with L -- see below). Otherwise, this element will be C. =item out_buffer This is an array reference containing all the output sent to STDOUT the command generated. The notes from L<"full_buffer"> apply. =item error_buffer This is an arrayreference containing all the output sent to STDERR the command generated. The notes from L<"full_buffer"> apply. =back See the L<"HOW IT WORKS"> section below to see how C decides what modules or function calls to use when issuing a command. =cut { my @acc = qw[ok error _fds]; ### autogenerate accessors ### for my $key ( @acc ) { no strict 'refs'; *{__PACKAGE__."::$key"} = sub { $_[0]->{$key} = $_[1] if @_ > 1; return $_[0]->{$key}; } } } sub can_use_run_forked { return $CAN_USE_RUN_FORKED eq "1"; } sub get_monotonic_time { if ($HAVE_MONOTONIC) { return Time::HiRes::clock_gettime(&Time::HiRes::CLOCK_MONOTONIC); } else { return time(); } } sub adjust_monotonic_start_time { my ($ref_vars, $now, $previous) = @_; # workaround only for those systems which don't have # Time::HiRes::CLOCK_MONOTONIC (Mac OSX in particular) return if $HAVE_MONOTONIC; # don't have previous monotonic value (only happens once # in the beginning of the program execution) return unless $previous; my $time_diff = $now - $previous; # adjust previously saved time with the skew value which is # either negative when clock moved back or more than 5 seconds -- # assuming that event loop does happen more often than once # per five seconds, which might not be always true (!) but # hopefully that's ok, because it's just a workaround if ($time_diff > 5 || $time_diff < 0) { foreach my $ref_var (@{$ref_vars}) { if (defined($$ref_var)) { $$ref_var = $$ref_var + $time_diff; } } } } sub uninstall_signals { return unless defined($IPC::Cmd::{'__old_signals'}); foreach my $sig_name (keys %{$IPC::Cmd::{'__old_signals'}}) { $SIG{$sig_name} = $IPC::Cmd::{'__old_signals'}->{$sig_name}; } } # incompatible with POSIX::SigAction # sub install_layered_signal { my ($s, $handler_code) = @_; my %available_signals = map {$_ => 1} keys %SIG; Carp::confess("install_layered_signal got nonexistent signal name [$s]") unless defined($available_signals{$s}); Carp::confess("install_layered_signal expects coderef") if !ref($handler_code) || ref($handler_code) ne 'CODE'; $IPC::Cmd::{'__old_signals'} = {} unless defined($IPC::Cmd::{'__old_signals'}); $IPC::Cmd::{'__old_signals'}->{$s} = $SIG{$s}; my $previous_handler = $SIG{$s}; my $sig_handler = sub { my ($called_sig_name, @sig_param) = @_; # $s is a closure referring to real signal name # for which this handler is being installed. # it is used to distinguish between # real signal handlers and aliased signal handlers my $signal_name = $s; # $called_sig_name is a signal name which # was passed to this signal handler; # it doesn't equal $signal_name in case # some signal handlers in %SIG point # to other signal handler (CHLD and CLD, # ABRT and IOT) # # initial signal handler for aliased signal # calls some other signal handler which # should not execute the same handler_code again if ($called_sig_name eq $signal_name) { $handler_code->($signal_name); } # run original signal handler if any (including aliased) # if (ref($previous_handler)) { $previous_handler->($called_sig_name, @sig_param); } }; $SIG{$s} = $sig_handler; } # give process a chance sending TERM, # waiting for a while (2 seconds) # and killing it with KILL sub kill_gently { my ($pid, $opts) = @_; require POSIX; $opts = {} unless $opts; $opts->{'wait_time'} = 2 unless defined($opts->{'wait_time'}); $opts->{'first_kill_type'} = 'just_process' unless $opts->{'first_kill_type'}; $opts->{'final_kill_type'} = 'just_process' unless $opts->{'final_kill_type'}; if ($opts->{'first_kill_type'} eq 'just_process') { kill(15, $pid); } elsif ($opts->{'first_kill_type'} eq 'process_group') { kill(-15, $pid); } my $do_wait = 1; my $child_finished = 0; my $wait_start_time = get_monotonic_time(); my $now; my $previous_monotonic_value; while ($do_wait) { $previous_monotonic_value = $now; $now = get_monotonic_time(); adjust_monotonic_start_time([\$wait_start_time], $now, $previous_monotonic_value); if ($now > $wait_start_time + $opts->{'wait_time'}) { $do_wait = 0; next; } my $waitpid = waitpid($pid, POSIX::WNOHANG); if ($waitpid eq -1) { $child_finished = 1; $do_wait = 0; next; } Time::HiRes::usleep(250000); # quarter of a second } if (!$child_finished) { if ($opts->{'final_kill_type'} eq 'just_process') { kill(9, $pid); } elsif ($opts->{'final_kill_type'} eq 'process_group') { kill(-9, $pid); } } } sub open3_run { my ($cmd, $opts) = @_; $opts = {} unless $opts; my $child_in = FileHandle->new; my $child_out = FileHandle->new; my $child_err = FileHandle->new; $child_out->autoflush(1); $child_err->autoflush(1); my $pid = open3($child_in, $child_out, $child_err, $cmd); Time::HiRes::usleep(1) if IS_HPUX; # will consider myself orphan if my ppid changes # from this one: my $original_ppid = $opts->{'original_ppid'}; # push my child's pid to our parent # so in case i am killed parent # could stop my child (search for # child_child_pid in parent code) if ($opts->{'parent_info'}) { my $ps = $opts->{'parent_info'}; print $ps "spawned $pid\n"; } if ($child_in && $child_out->opened && $opts->{'child_stdin'}) { # If the child process dies for any reason, # the next write to CHLD_IN is likely to generate # a SIGPIPE in the parent, which is fatal by default. # So you may wish to handle this signal. # # from http://perldoc.perl.org/IPC/Open3.html, # absolutely needed to catch piped commands errors. # local $SIG{'PIPE'} = sub { 1; }; print $child_in $opts->{'child_stdin'}; } close($child_in); my $child_output = { 'out' => $child_out->fileno, 'err' => $child_err->fileno, $child_out->fileno => { 'parent_socket' => $opts->{'parent_stdout'}, 'scalar_buffer' => "", 'child_handle' => $child_out, 'block_size' => ($child_out->stat)[11] || 1024, }, $child_err->fileno => { 'parent_socket' => $opts->{'parent_stderr'}, 'scalar_buffer' => "", 'child_handle' => $child_err, 'block_size' => ($child_err->stat)[11] || 1024, }, }; my $select = IO::Select->new(); $select->add($child_out, $child_err); # pass any signal to the child # effectively creating process # strongly attached to the child: # it will terminate only after child # has terminated (except for SIGKILL, # which is specially handled) SIGNAL: foreach my $s (keys %SIG) { next SIGNAL if $s eq '__WARN__' or $s eq '__DIE__'; # Skip and don't clobber __DIE__ & __WARN__ my $sig_handler; $sig_handler = sub { kill("$s", $pid); $SIG{$s} = $sig_handler; }; $SIG{$s} = $sig_handler; } my $child_finished = 0; my $real_exit; my $exit_value; while(!$child_finished) { # parent was killed otherwise we would have got # the same signal as parent and process it same way if (getppid() != $original_ppid) { # end my process group with all the children # (i am the process group leader, so my pid # equals to the process group id) # # same thing which is done # with $opts->{'clean_up_children'} # in run_forked # kill(-9, $$); POSIX::_exit 1; } my $waitpid = waitpid($pid, POSIX::WNOHANG); # child finished, catch it's exit status if ($waitpid ne 0 && $waitpid ne -1) { $real_exit = $?; $exit_value = $? >> 8; } if ($waitpid eq -1) { $child_finished = 1; } my $ready_fds = []; push @{$ready_fds}, $select->can_read(1/100); READY_FDS: while (scalar(@{$ready_fds})) { my $fd = shift @{$ready_fds}; $ready_fds = [grep {$_ ne $fd} @{$ready_fds}]; my $str = $child_output->{$fd->fileno}; Carp::confess("child stream not found: $fd") unless $str; my $data; my $count = $fd->sysread($data, $str->{'block_size'}); if ($count) { if ($str->{'parent_socket'}) { my $ph = $str->{'parent_socket'}; print $ph $data; } else { $str->{'scalar_buffer'} .= $data; } } elsif ($count eq 0) { $select->remove($fd); $fd->close(); } else { Carp::confess("error during sysread: " . $!); } push @{$ready_fds}, $select->can_read(1/100) if $child_finished; } Time::HiRes::usleep(1); } # since we've successfully reaped the child, # let our parent know about this. # if ($opts->{'parent_info'}) { my $ps = $opts->{'parent_info'}; # child was killed, inform parent if ($real_exit & 127) { print $ps "$pid killed with " . ($real_exit & 127) . "\n"; } print $ps "reaped $pid\n"; } if ($opts->{'parent_stdout'} || $opts->{'parent_stderr'}) { return $exit_value; } else { return { 'stdout' => $child_output->{$child_output->{'out'}}->{'scalar_buffer'}, 'stderr' => $child_output->{$child_output->{'err'}}->{'scalar_buffer'}, 'exit_code' => $exit_value, }; } } =head2 $hashref = run_forked( COMMAND, { child_stdin => SCALAR, timeout => DIGIT, stdout_handler => CODEREF, stderr_handler => CODEREF} ); C is used to execute some program or a coderef, optionally feed it with some input, get its return code and output (both stdout and stderr into separate buffers). In addition, it allows to terminate the program if it takes too long to finish. The important and distinguishing feature of run_forked is execution timeout which at first seems to be quite a simple task but if you think that the program which you're spawning might spawn some children itself (which in their turn could do the same and so on) it turns out to be not a simple issue. C is designed to survive and successfully terminate almost any long running task, even a fork bomb in case your system has the resources to survive during given timeout. This is achieved by creating separate watchdog process which spawns the specified program in a separate process session and supervises it: optionally feeds it with input, stores its exit code, stdout and stderr, terminates it in case it runs longer than specified. Invocation requires the command to be executed or a coderef and optionally a hashref of options: =over =item C Specify in seconds how long to run the command before it is killed with SIG_KILL (9), which effectively terminates it and all of its children (direct or indirect). =item C Specify some text that will be passed into the C of the executed program. =item C Coderef of a subroutine to call when a portion of data is received on STDOUT from the executing program. =item C Coderef of a subroutine to call when a portion of data is received on STDERR from the executing program. =item C Coderef of a subroutine to call inside of the main waiting loop (while C waits for the external to finish or fail). It is useful to stop running external process before it ends by itself, e.g. my $r = run_forked("some external command", { 'wait_loop_callback' => sub { if (condition) { kill(1, $$); } }, 'terminate_on_signal' => 'HUP', }); Combined with C and C allows terminating external command based on its output. Could also be used as a timer without engaging with L (signals). Remember that this code could be called every millisecond (depending on the output which external command generates), so try to make it as lightweight as possible. =item C Discards the buffering of the standard output and standard errors for return by run_forked(). With this option you have to use the std*_handlers to read what the command outputs. Useful for commands that send a lot of output. =item C Enable this option if you wish all spawned processes to be killed if the initially spawned process (the parent) is killed or dies without waiting for child processes. =back C will return a HASHREF with the following keys: =over =item C The exit code of the executed program. =item C The number of seconds the program ran for before being terminated, or 0 if no timeout occurred. =item C Holds the standard output of the executed command (or empty string if there was no STDOUT output or if C was used; it's always defined!) =item C Holds the standard error of the executed command (or empty string if there was no STDERR output or if C was used; it's always defined!) =item C Holds the standard output and error of the executed command merged into one stream (or empty string if there was no output at all or if C was used; it's always defined!) =item C Holds some explanation in the case of an error. =back =cut sub run_forked { ### container to store things in my $self = bless {}, __PACKAGE__; if (!can_use_run_forked()) { Carp::carp("run_forked is not available: $CAN_USE_RUN_FORKED"); return; } require POSIX; my ($cmd, $opts) = @_; if (ref($cmd) eq 'ARRAY') { $cmd = join(" ", @{$cmd}); } if (!$cmd) { Carp::carp("run_forked expects command to run"); return; } $opts = {} unless $opts; $opts->{'timeout'} = 0 unless $opts->{'timeout'}; $opts->{'terminate_wait_time'} = 2 unless defined($opts->{'terminate_wait_time'}); # turned on by default $opts->{'clean_up_children'} = 1 unless defined($opts->{'clean_up_children'}); # sockets to pass child stdout to parent my $child_stdout_socket; my $parent_stdout_socket; # sockets to pass child stderr to parent my $child_stderr_socket; my $parent_stderr_socket; # sockets for child -> parent internal communication my $child_info_socket; my $parent_info_socket; socketpair($child_stdout_socket, $parent_stdout_socket, &Socket::AF_UNIX, &Socket::SOCK_STREAM, &Socket::PF_UNSPEC) || Carp::confess ("socketpair: $!"); socketpair($child_stderr_socket, $parent_stderr_socket, &Socket::AF_UNIX, &Socket::SOCK_STREAM, &Socket::PF_UNSPEC) || Carp::confess ("socketpair: $!"); socketpair($child_info_socket, $parent_info_socket, &Socket::AF_UNIX, &Socket::SOCK_STREAM, &Socket::PF_UNSPEC) || Carp::confess ("socketpair: $!"); $child_stdout_socket->autoflush(1); $parent_stdout_socket->autoflush(1); $child_stderr_socket->autoflush(1); $parent_stderr_socket->autoflush(1); $child_info_socket->autoflush(1); $parent_info_socket->autoflush(1); my $start_time = get_monotonic_time(); my $pid; my $ppid = $$; if ($pid = fork) { # we are a parent close($parent_stdout_socket); close($parent_stderr_socket); close($parent_info_socket); my $flags; # prepare sockets to read from child $flags = fcntl($child_stdout_socket, POSIX::F_GETFL, 0) || Carp::confess "can't fnctl F_GETFL: $!"; $flags |= POSIX::O_NONBLOCK; fcntl($child_stdout_socket, POSIX::F_SETFL, $flags) || Carp::confess "can't fnctl F_SETFL: $!"; $flags = fcntl($child_stderr_socket, POSIX::F_GETFL, 0) || Carp::confess "can't fnctl F_GETFL: $!"; $flags |= POSIX::O_NONBLOCK; fcntl($child_stderr_socket, POSIX::F_SETFL, $flags) || Carp::confess "can't fnctl F_SETFL: $!"; $flags = fcntl($child_info_socket, POSIX::F_GETFL, 0) || Carp::confess "can't fnctl F_GETFL: $!"; $flags |= POSIX::O_NONBLOCK; fcntl($child_info_socket, POSIX::F_SETFL, $flags) || Carp::confess "can't fnctl F_SETFL: $!"; # print "child $pid started\n"; my $child_output = { $child_stdout_socket->fileno => { 'scalar_buffer' => "", 'child_handle' => $child_stdout_socket, 'block_size' => ($child_stdout_socket->stat)[11] || 1024, 'protocol' => 'stdout', }, $child_stderr_socket->fileno => { 'scalar_buffer' => "", 'child_handle' => $child_stderr_socket, 'block_size' => ($child_stderr_socket->stat)[11] || 1024, 'protocol' => 'stderr', }, $child_info_socket->fileno => { 'scalar_buffer' => "", 'child_handle' => $child_info_socket, 'block_size' => ($child_info_socket->stat)[11] || 1024, 'protocol' => 'info', }, }; my $select = IO::Select->new(); $select->add($child_stdout_socket, $child_stderr_socket, $child_info_socket); my $child_timedout = 0; my $child_finished = 0; my $child_stdout = ''; my $child_stderr = ''; my $child_merged = ''; my $child_exit_code = 0; my $child_killed_by_signal = 0; my $parent_died = 0; my $last_parent_check = 0; my $got_sig_child = 0; my $got_sig_quit = 0; my $orig_sig_child = $SIG{'CHLD'}; $SIG{'CHLD'} = sub { $got_sig_child = get_monotonic_time(); }; if ($opts->{'terminate_on_signal'}) { install_layered_signal($opts->{'terminate_on_signal'}, sub { $got_sig_quit = time(); }); } my $child_child_pid; my $now; my $previous_monotonic_value; while (!$child_finished) { $previous_monotonic_value = $now; $now = get_monotonic_time(); adjust_monotonic_start_time([\$start_time, \$last_parent_check, \$got_sig_child], $now, $previous_monotonic_value); if ($opts->{'terminate_on_parent_sudden_death'}) { # check for parent once each five seconds if ($now > $last_parent_check + 5) { if (getppid() eq "1") { kill_gently ($pid, { 'first_kill_type' => 'process_group', 'final_kill_type' => 'process_group', 'wait_time' => $opts->{'terminate_wait_time'} }); $parent_died = 1; } $last_parent_check = $now; } } # user specified timeout if ($opts->{'timeout'}) { if ($now > $start_time + $opts->{'timeout'}) { kill_gently ($pid, { 'first_kill_type' => 'process_group', 'final_kill_type' => 'process_group', 'wait_time' => $opts->{'terminate_wait_time'} }); $child_timedout = 1; } } # give OS 10 seconds for correct return of waitpid, # kill process after that and finish wait loop; # shouldn't ever happen -- remove this code? if ($got_sig_child) { if ($now > $got_sig_child + 10) { print STDERR "waitpid did not return -1 for 10 seconds after SIG_CHLD, killing [$pid]\n"; kill (-9, $pid); $child_finished = 1; } } if ($got_sig_quit) { kill_gently ($pid, { 'first_kill_type' => 'process_group', 'final_kill_type' => 'process_group', 'wait_time' => $opts->{'terminate_wait_time'} }); $child_finished = 1; } my $waitpid = waitpid($pid, POSIX::WNOHANG); # child finished, catch it's exit status if ($waitpid ne 0 && $waitpid ne -1) { $child_exit_code = $? >> 8; } if ($waitpid eq -1) { $child_finished = 1; } my $ready_fds = []; push @{$ready_fds}, $select->can_read(1/100); READY_FDS: while (scalar(@{$ready_fds})) { my $fd = shift @{$ready_fds}; $ready_fds = [grep {$_ ne $fd} @{$ready_fds}]; my $str = $child_output->{$fd->fileno}; Carp::confess("child stream not found: $fd") unless $str; my $data = ""; my $count = $fd->sysread($data, $str->{'block_size'}); if ($count) { # extract all the available lines and store the rest in temporary buffer if ($data =~ /(.+\n)([^\n]*)/so) { $data = $str->{'scalar_buffer'} . $1; $str->{'scalar_buffer'} = $2 || ""; } else { $str->{'scalar_buffer'} .= $data; $data = ""; } } elsif ($count eq 0) { $select->remove($fd); $fd->close(); if ($str->{'scalar_buffer'}) { $data = $str->{'scalar_buffer'} . "\n"; } } else { Carp::confess("error during sysread on [$fd]: " . $!); } # $data contains only full lines (or last line if it was unfinished read # or now new-line in the output of the child); dat is processed # according to the "protocol" of socket if ($str->{'protocol'} eq 'info') { if ($data =~ /^spawned ([0-9]+?)\n(.*?)/so) { $child_child_pid = $1; $data = $2; } if ($data =~ /^reaped ([0-9]+?)\n(.*?)/so) { $child_child_pid = undef; $data = $2; } if ($data =~ /^[\d]+ killed with ([0-9]+?)\n(.*?)/so) { $child_killed_by_signal = $1; $data = $2; } # we don't expect any other data in info socket, so it's # some strange violation of protocol, better know about this if ($data) { Carp::confess("info protocol violation: [$data]"); } } if ($str->{'protocol'} eq 'stdout') { if (!$opts->{'discard_output'}) { $child_stdout .= $data; $child_merged .= $data; } if ($opts->{'stdout_handler'} && ref($opts->{'stdout_handler'}) eq 'CODE') { $opts->{'stdout_handler'}->($data); } } if ($str->{'protocol'} eq 'stderr') { if (!$opts->{'discard_output'}) { $child_stderr .= $data; $child_merged .= $data; } if ($opts->{'stderr_handler'} && ref($opts->{'stderr_handler'}) eq 'CODE') { $opts->{'stderr_handler'}->($data); } } # process may finish (waitpid returns -1) before # we've read all of its output because of buffering; # so try to read all the way it is possible to read # in such case - this shouldn't be too much (unless # the buffer size is HUGE -- should introduce # another counter in such case, maybe later) # push @{$ready_fds}, $select->can_read(1/100) if $child_finished; } if ($opts->{'wait_loop_callback'} && ref($opts->{'wait_loop_callback'}) eq 'CODE') { $opts->{'wait_loop_callback'}->(); } Time::HiRes::usleep(1); } # $child_pid_pid is not defined in two cases: # * when our child was killed before # it had chance to tell us the pid # of the child it spawned. we can do # nothing in this case :( # * our child successfully reaped its child, # we have nothing left to do in this case # # defined $child_pid_pid means child's child # has not died but nobody is waiting for it, # killing it brutally. # if ($child_child_pid) { kill_gently($child_child_pid); } # in case there are forks in child which # do not forward or process signals (TERM) correctly # kill whole child process group, effectively trying # not to return with some children or their parts still running # # to be more accurate -- we need to be sure # that this is process group created by our child # (and not some other process group with the same pgid, # created just after death of our child) -- fortunately # this might happen only when process group ids # are reused quickly (there are lots of processes # spawning new process groups for example) # if ($opts->{'clean_up_children'}) { kill(-9, $pid); } # print "child $pid finished\n"; close($child_stdout_socket); close($child_stderr_socket); close($child_info_socket); my $o = { 'stdout' => $child_stdout, 'stderr' => $child_stderr, 'merged' => $child_merged, 'timeout' => $child_timedout ? $opts->{'timeout'} : 0, 'exit_code' => $child_exit_code, 'parent_died' => $parent_died, 'killed_by_signal' => $child_killed_by_signal, 'child_pgid' => $pid, 'cmd' => $cmd, }; my $err_msg = ''; if ($o->{'exit_code'}) { $err_msg .= "exited with code [$o->{'exit_code'}]\n"; } if ($o->{'timeout'}) { $err_msg .= "ran more than [$o->{'timeout'}] seconds\n"; } if ($o->{'parent_died'}) { $err_msg .= "parent died\n"; } if ($o->{'stdout'} && !$opts->{'non_empty_stdout_ok'}) { $err_msg .= "stdout:\n" . $o->{'stdout'} . "\n"; } if ($o->{'stderr'}) { $err_msg .= "stderr:\n" . $o->{'stderr'} . "\n"; } if ($o->{'killed_by_signal'}) { $err_msg .= "killed by signal [" . $o->{'killed_by_signal'} . "]\n"; } $o->{'err_msg'} = $err_msg; if ($orig_sig_child) { $SIG{'CHLD'} = $orig_sig_child; } else { delete($SIG{'CHLD'}); } uninstall_signals(); return $o; } else { Carp::confess("cannot fork: $!") unless defined($pid); # create new process session for open3 call, # so we hopefully can kill all the subprocesses # which might be spawned in it (except for those # which do setsid theirselves -- can't do anything # with those) POSIX::setsid() || Carp::confess("Error running setsid: " . $!); if ($opts->{'child_BEGIN'} && ref($opts->{'child_BEGIN'}) eq 'CODE') { $opts->{'child_BEGIN'}->(); } close($child_stdout_socket); close($child_stderr_socket); close($child_info_socket); my $child_exit_code; # allow both external programs # and internal perl calls if (!ref($cmd)) { $child_exit_code = open3_run($cmd, { 'parent_info' => $parent_info_socket, 'parent_stdout' => $parent_stdout_socket, 'parent_stderr' => $parent_stderr_socket, 'child_stdin' => $opts->{'child_stdin'}, 'original_ppid' => $ppid, }); } elsif (ref($cmd) eq 'CODE') { # reopen STDOUT and STDERR for child code: # https://rt.cpan.org/Ticket/Display.html?id=85912 open STDOUT, '>&', $parent_stdout_socket || Carp::confess("Unable to reopen STDOUT: $!\n"); open STDERR, '>&', $parent_stderr_socket || Carp::confess("Unable to reopen STDERR: $!\n"); $child_exit_code = $cmd->({ 'opts' => $opts, 'parent_info' => $parent_info_socket, 'parent_stdout' => $parent_stdout_socket, 'parent_stderr' => $parent_stderr_socket, 'child_stdin' => $opts->{'child_stdin'}, }); } else { print $parent_stderr_socket "Invalid command reference: " . ref($cmd) . "\n"; $child_exit_code = 1; } close($parent_stdout_socket); close($parent_stderr_socket); close($parent_info_socket); if ($opts->{'child_END'} && ref($opts->{'child_END'}) eq 'CODE') { $opts->{'child_END'}->(); } $| = 1; POSIX::_exit $child_exit_code; } } sub run { ### container to store things in my $self = bless {}, __PACKAGE__; my %hash = @_; ### if the user didn't provide a buffer, we'll store it here. my $def_buf = ''; my($verbose,$cmd,$buffer,$timeout); my $tmpl = { verbose => { default => $VERBOSE, store => \$verbose }, buffer => { default => \$def_buf, store => \$buffer }, command => { required => 1, store => \$cmd, allow => sub { !ref($_[0]) or ref($_[0]) eq 'ARRAY' }, }, timeout => { default => 0, store => \$timeout }, }; unless( check( $tmpl, \%hash, $VERBOSE ) ) { Carp::carp( loc( "Could not validate input: %1", Params::Check->last_error ) ); return; }; $cmd = _quote_args_vms( $cmd ) if IS_VMS; ### strip any empty elements from $cmd if present if ( $ALLOW_NULL_ARGS ) { $cmd = [ grep { defined } @$cmd ] if ref $cmd; } else { $cmd = [ grep { defined && length } @$cmd ] if ref $cmd; } my $pp_cmd = (ref $cmd ? "@$cmd" : $cmd); print loc("Running [%1]...\n", $pp_cmd ) if $verbose; ### did the user pass us a buffer to fill or not? if so, set this ### flag so we know what is expected of us ### XXX this is now being ignored. in the future, we could add diagnostic ### messages based on this logic #my $user_provided_buffer = $buffer == \$def_buf ? 0 : 1; ### buffers that are to be captured my( @buffer, @buff_err, @buff_out ); ### capture STDOUT my $_out_handler = sub { my $buf = shift; return unless defined $buf; print STDOUT $buf if $verbose; push @buffer, $buf; push @buff_out, $buf; }; ### capture STDERR my $_err_handler = sub { my $buf = shift; return unless defined $buf; print STDERR $buf if $verbose; push @buffer, $buf; push @buff_err, $buf; }; ### flag to indicate we have a buffer captured my $have_buffer = $self->can_capture_buffer ? 1 : 0; ### flag indicating if the subcall went ok my $ok; ### don't look at previous errors: local $?; local $@; local $!; ### we might be having a timeout set eval { local $SIG{ALRM} = sub { die bless sub { ALARM_CLASS . qq[: Command '$pp_cmd' aborted by alarm after $timeout seconds] }, ALARM_CLASS } if $timeout; alarm $timeout || 0; ### IPC::Run is first choice if $USE_IPC_RUN is set. if( !IS_WIN32 and $USE_IPC_RUN and $self->can_use_ipc_run( 1 ) ) { ### ipc::run handlers needs the command as a string or an array ref $self->_debug( "# Using IPC::Run. Have buffer: $have_buffer" ) if $DEBUG; $ok = $self->_ipc_run( $cmd, $_out_handler, $_err_handler ); ### since IPC::Open3 works on all platforms, and just fails on ### win32 for capturing buffers, do that ideally } elsif ( $USE_IPC_OPEN3 and $self->can_use_ipc_open3( 1 ) ) { $self->_debug("# Using IPC::Open3. Have buffer: $have_buffer") if $DEBUG; ### in case there are pipes in there; ### IPC::Open3 will call exec and exec will do the right thing my $method = IS_WIN32 ? '_open3_run_win32' : '_open3_run'; $ok = $self->$method( $cmd, $_out_handler, $_err_handler, $verbose ); ### if we are allowed to run verbose, just dispatch the system command } else { $self->_debug( "# Using system(). Have buffer: $have_buffer" ) if $DEBUG; $ok = $self->_system_run( $cmd, $verbose ); } alarm 0; }; ### restore STDIN after duping, or STDIN will be closed for ### this current perl process! $self->__reopen_fds( @{ $self->_fds} ) if $self->_fds; my $err; unless( $ok ) { ### alarm happened if ( $@ and ref $@ and $@->isa( ALARM_CLASS ) ) { $err = $@->(); # the error code is an expired alarm ### another error happened, set by the dispatchub } else { $err = $self->error; } } ### fill the buffer; $$buffer = join '', @buffer if @buffer; ### return a list of flags and buffers (if available) in list ### context, or just a simple 'ok' in scalar return wantarray ? $have_buffer ? ($ok, $err, \@buffer, \@buff_out, \@buff_err) : ($ok, $err ) : $ok } sub _open3_run_win32 { my $self = shift; my $cmd = shift; my $outhand = shift; my $errhand = shift; require Socket; my $pipe = sub { socketpair($_[0], $_[1], &Socket::AF_UNIX, &Socket::SOCK_STREAM, &Socket::PF_UNSPEC) or return undef; shutdown($_[0], 1); # No more writing for reader shutdown($_[1], 0); # No more reading for writer return 1; }; my $open3 = sub { local (*TO_CHLD_R, *TO_CHLD_W); local (*FR_CHLD_R, *FR_CHLD_W); local (*FR_CHLD_ERR_R, *FR_CHLD_ERR_W); $pipe->(*TO_CHLD_R, *TO_CHLD_W ) or die $^E; $pipe->(*FR_CHLD_R, *FR_CHLD_W ) or die $^E; $pipe->(*FR_CHLD_ERR_R, *FR_CHLD_ERR_W) or die $^E; my $pid = IPC::Open3::open3('>&TO_CHLD_R', '<&FR_CHLD_W', '<&FR_CHLD_ERR_W', @_); return ( $pid, *TO_CHLD_W, *FR_CHLD_R, *FR_CHLD_ERR_R ); }; $cmd = [ grep { defined && length } @$cmd ] if ref $cmd; $cmd = $self->__fix_cmd_whitespace_and_special_chars( $cmd ); my ($pid, $to_chld, $fr_chld, $fr_chld_err) = $open3->( ( ref $cmd ? @$cmd : $cmd ) ); my $in_sel = IO::Select->new(); my $out_sel = IO::Select->new(); my %objs; $objs{ fileno( $fr_chld ) } = $outhand; $objs{ fileno( $fr_chld_err ) } = $errhand; $in_sel->add( $fr_chld ); $in_sel->add( $fr_chld_err ); close($to_chld); while ($in_sel->count() + $out_sel->count()) { my ($ins, $outs) = IO::Select::select($in_sel, $out_sel, undef); for my $fh (@$ins) { my $obj = $objs{ fileno($fh) }; my $buf; my $bytes_read = sysread($fh, $buf, 64*1024 ); #, length($buf)); if (!$bytes_read) { $in_sel->remove($fh); } else { $obj->( "$buf" ); } } for my $fh (@$outs) { } } waitpid($pid, 0); ### some error occurred if( $? ) { $self->error( $self->_pp_child_error( $cmd, $? ) ); $self->ok( 0 ); return; } else { return $self->ok( 1 ); } } sub _open3_run { my $self = shift; my $cmd = shift; my $_out_handler = shift; my $_err_handler = shift; my $verbose = shift || 0; ### Following code are adapted from Friar 'abstracts' in the ### Perl Monastery (http://www.perlmonks.org/index.pl?node_id=151886). ### XXX that code didn't work. ### we now use the following code, thanks to theorbtwo ### define them beforehand, so we always have defined FH's ### to read from. use Symbol; my $kidout = Symbol::gensym(); my $kiderror = Symbol::gensym(); ### Dup the filehandle so we can pass 'our' STDIN to the ### child process. This stops us from having to pump input ### from ourselves to the childprocess. However, we will need ### to revive the FH afterwards, as IPC::Open3 closes it. ### We'll do the same for STDOUT and STDERR. It works without ### duping them on non-unix derivatives, but not on win32. my @fds_to_dup = ( IS_WIN32 && !$verbose ? qw[STDIN STDOUT STDERR] : qw[STDIN] ); $self->_fds( \@fds_to_dup ); $self->__dup_fds( @fds_to_dup ); ### pipes have to come in a quoted string, and that clashes with ### whitespace. This sub fixes up such commands so they run properly $cmd = $self->__fix_cmd_whitespace_and_special_chars( $cmd ); ### don't stringify @$cmd, so spaces in filenames/paths are ### treated properly my $pid = eval { IPC::Open3::open3( '<&STDIN', (IS_WIN32 ? '>&STDOUT' : $kidout), (IS_WIN32 ? '>&STDERR' : $kiderror), ( ref $cmd ? @$cmd : $cmd ), ); }; ### open3 error occurred if( $@ and $@ =~ /^open3:/ ) { $self->ok( 0 ); $self->error( $@ ); return; }; ### use OUR stdin, not $kidin. Somehow, ### we never get the input.. so jump through ### some hoops to do it :( my $selector = IO::Select->new( (IS_WIN32 ? \*STDERR : $kiderror), \*STDIN, (IS_WIN32 ? \*STDOUT : $kidout) ); STDOUT->autoflush(1); STDERR->autoflush(1); STDIN->autoflush(1); $kidout->autoflush(1) if UNIVERSAL::can($kidout, 'autoflush'); $kiderror->autoflush(1) if UNIVERSAL::can($kiderror, 'autoflush'); ### add an explicit break statement ### code courtesy of theorbtwo from #london.pm my $stdout_done = 0; my $stderr_done = 0; OUTER: while ( my @ready = $selector->can_read ) { for my $h ( @ready ) { my $buf; ### $len is the amount of bytes read my $len = sysread( $h, $buf, 4096 ); # try to read 4096 bytes ### see perldoc -f sysread: it returns undef on error, ### so bail out. if( not defined $len ) { warn(loc("Error reading from process: %1", $!)); last OUTER; } ### check for $len. it may be 0, at which point we're ### done reading, so don't try to process it. ### if we would print anyway, we'd provide bogus information $_out_handler->( "$buf" ) if $len && $h == $kidout; $_err_handler->( "$buf" ) if $len && $h == $kiderror; ### Wait till child process is done printing to both ### stdout and stderr. $stdout_done = 1 if $h == $kidout and $len == 0; $stderr_done = 1 if $h == $kiderror and $len == 0; last OUTER if ($stdout_done && $stderr_done); } } waitpid $pid, 0; # wait for it to die ### restore STDIN after duping, or STDIN will be closed for ### this current perl process! ### done in the parent call now # $self->__reopen_fds( @fds_to_dup ); ### some error occurred if( $? ) { $self->error( $self->_pp_child_error( $cmd, $? ) ); $self->ok( 0 ); return; } else { return $self->ok( 1 ); } } ### Text::ParseWords::shellwords() uses unix semantics. that will break ### on win32 { my $parse_sub = IS_WIN32 ? __PACKAGE__->can('_split_like_shell_win32') : Text::ParseWords->can('shellwords'); sub _ipc_run { my $self = shift; my $cmd = shift; my $_out_handler = shift; my $_err_handler = shift; STDOUT->autoflush(1); STDERR->autoflush(1); ### a command like: # [ # '/usr/bin/gzip', # '-cdf', # '/Users/kane/sources/p4/other/archive-extract/t/src/x.tgz', # '|', # '/usr/bin/tar', # '-tf -' # ] ### needs to become: # [ # ['/usr/bin/gzip', '-cdf', # '/Users/kane/sources/p4/other/archive-extract/t/src/x.tgz'] # '|', # ['/usr/bin/tar', '-tf -'] # ] my @command; my $special_chars; my $re = do { my $x = join '', SPECIAL_CHARS; qr/([$x])/ }; if( ref $cmd ) { my $aref = []; for my $item (@$cmd) { if( $item =~ $re ) { push @command, $aref, $item; $aref = []; $special_chars .= $1; } else { push @$aref, $item; } } push @command, $aref; } else { @command = map { if( $_ =~ $re ) { $special_chars .= $1; $_; } else { # [ split /\s+/ ] [ map { m/[ ]/ ? qq{'$_'} : $_ } $parse_sub->($_) ] } } split( /\s*$re\s*/, $cmd ); } ### if there's a pipe in the command, *STDIN needs to ### be inserted *BEFORE* the pipe, to work on win32 ### this also works on *nix, so we should do it when possible ### this should *also* work on multiple pipes in the command ### if there's no pipe in the command, append STDIN to the back ### of the command instead. ### XXX seems IPC::Run works it out for itself if you just ### don't pass STDIN at all. # if( $special_chars and $special_chars =~ /\|/ ) { # ### only add STDIN the first time.. # my $i; # @command = map { ($_ eq '|' && not $i++) # ? ( \*STDIN, $_ ) # : $_ # } @command; # } else { # push @command, \*STDIN; # } # \*STDIN is already included in the @command, see a few lines up my $ok = eval { IPC::Run::run( @command, fileno(STDOUT).'>', $_out_handler, fileno(STDERR).'>', $_err_handler ) }; ### all is well if( $ok ) { return $self->ok( $ok ); ### some error occurred } else { $self->ok( 0 ); ### if the eval fails due to an exception, deal with it ### unless it's an alarm if( $@ and not UNIVERSAL::isa( $@, ALARM_CLASS ) ) { $self->error( $@ ); ### if it *is* an alarm, propagate } elsif( $@ ) { die $@; ### some error in the sub command } else { $self->error( $self->_pp_child_error( $cmd, $? ) ); } return; } } } sub _system_run { my $self = shift; my $cmd = shift; my $verbose = shift || 0; ### pipes have to come in a quoted string, and that clashes with ### whitespace. This sub fixes up such commands so they run properly $cmd = $self->__fix_cmd_whitespace_and_special_chars( $cmd ); my @fds_to_dup = $verbose ? () : qw[STDOUT STDERR]; $self->_fds( \@fds_to_dup ); $self->__dup_fds( @fds_to_dup ); ### system returns 'true' on failure -- the exit code of the cmd $self->ok( 1 ); system( ref $cmd ? @$cmd : $cmd ) == 0 or do { $self->error( $self->_pp_child_error( $cmd, $? ) ); $self->ok( 0 ); }; ### done in the parent call now #$self->__reopen_fds( @fds_to_dup ); return unless $self->ok; return $self->ok; } { my %sc_lookup = map { $_ => $_ } SPECIAL_CHARS; sub __fix_cmd_whitespace_and_special_chars { my $self = shift; my $cmd = shift; ### command has a special char in it if( ref $cmd and grep { $sc_lookup{$_} } @$cmd ) { ### since we have special chars, we have to quote white space ### this *may* conflict with the parsing :( my $fixed; my @cmd = map { / / ? do { $fixed++; QUOTE.$_.QUOTE } : $_ } @$cmd; $self->_debug( "# Quoted $fixed arguments containing whitespace" ) if $DEBUG && $fixed; ### stringify it, so the special char isn't escaped as argument ### to the program $cmd = join ' ', @cmd; } return $cmd; } } ### Command-line arguments (but not the command itself) must be quoted ### to ensure case preservation. Borrowed from Module::Build with adaptations. ### Patch for this supplied by Craig Berry, see RT #46288: [PATCH] Add argument ### quoting for run() on VMS sub _quote_args_vms { ### Returns a command string with proper quoting so that the subprocess ### sees this same list of args, or if we get a single arg that is an ### array reference, quote the elements of it (except for the first) ### and return the reference. my @args = @_; my $got_arrayref = (scalar(@args) == 1 && UNIVERSAL::isa($args[0], 'ARRAY')) ? 1 : 0; @args = split(/\s+/, $args[0]) unless $got_arrayref || scalar(@args) > 1; my $cmd = $got_arrayref ? shift @{$args[0]} : shift @args; ### Do not quote qualifiers that begin with '/' or previously quoted args. map { if (/^[^\/\"]/) { $_ =~ s/\"/""/g; # escape C<"> by doubling $_ = q(").$_.q("); } } ($got_arrayref ? @{$args[0]} : @args ); $got_arrayref ? unshift(@{$args[0]}, $cmd) : unshift(@args, $cmd); return $got_arrayref ? $args[0] : join(' ', @args); } ### XXX this is cribbed STRAIGHT from M::B 0.30 here: ### http://search.cpan.org/src/KWILLIAMS/Module-Build-0.30/lib/Module/Build/Platform/Windows.pm:split_like_shell ### XXX this *should* be integrated into text::parsewords sub _split_like_shell_win32 { # As it turns out, Windows command-parsing is very different from # Unix command-parsing. Double-quotes mean different things, # backslashes don't necessarily mean escapes, and so on. So we # can't use Text::ParseWords::shellwords() to break a command string # into words. The algorithm below was bashed out by Randy and Ken # (mostly Randy), and there are a lot of regression tests, so we # should feel free to adjust if desired. local $_ = shift; my @argv; return @argv unless defined() && length(); my $arg = ''; my( $i, $quote_mode ) = ( 0, 0 ); while ( $i < length() ) { my $ch = substr( $_, $i , 1 ); my $next_ch = substr( $_, $i+1, 1 ); if ( $ch eq '\\' && $next_ch eq '"' ) { $arg .= '"'; $i++; } elsif ( $ch eq '\\' && $next_ch eq '\\' ) { $arg .= '\\'; $i++; } elsif ( $ch eq '"' && $next_ch eq '"' && $quote_mode ) { $quote_mode = !$quote_mode; $arg .= '"'; $i++; } elsif ( $ch eq '"' && $next_ch eq '"' && !$quote_mode && ( $i + 2 == length() || substr( $_, $i + 2, 1 ) eq ' ' ) ) { # for cases like: a"" => [ 'a' ] push( @argv, $arg ); $arg = ''; $i += 2; } elsif ( $ch eq '"' ) { $quote_mode = !$quote_mode; } elsif ( $ch eq ' ' && !$quote_mode ) { push( @argv, $arg ) if defined( $arg ) && length( $arg ); $arg = ''; ++$i while substr( $_, $i + 1, 1 ) eq ' '; } else { $arg .= $ch; } $i++; } push( @argv, $arg ) if defined( $arg ) && length( $arg ); return @argv; } { use File::Spec; use Symbol; my %Map = ( STDOUT => [qw|>&|, \*STDOUT, Symbol::gensym() ], STDERR => [qw|>&|, \*STDERR, Symbol::gensym() ], STDIN => [qw|<&|, \*STDIN, Symbol::gensym() ], ); ### dups FDs and stores them in a cache sub __dup_fds { my $self = shift; my @fds = @_; __PACKAGE__->_debug( "# Closing the following fds: @fds" ) if $DEBUG; for my $name ( @fds ) { my($redir, $fh, $glob) = @{$Map{$name}} or ( Carp::carp(loc("No such FD: '%1'", $name)), next ); ### MUST use the 2-arg version of open for dup'ing for ### 5.6.x compatibility. 5.8.x can use 3-arg open ### see perldoc5.6.2 -f open for details open $glob, $redir . fileno($fh) or ( Carp::carp(loc("Could not dup '$name': %1", $!)), return ); ### we should re-open this filehandle right now, not ### just dup it ### Use 2-arg version of open, as 5.5.x doesn't support ### 3-arg version =/ if( $redir eq '>&' ) { open( $fh, '>' . File::Spec->devnull ) or ( Carp::carp(loc("Could not reopen '$name': %1", $!)), return ); } } return 1; } ### reopens FDs from the cache sub __reopen_fds { my $self = shift; my @fds = @_; __PACKAGE__->_debug( "# Reopening the following fds: @fds" ) if $DEBUG; for my $name ( @fds ) { my($redir, $fh, $glob) = @{$Map{$name}} or ( Carp::carp(loc("No such FD: '%1'", $name)), next ); ### MUST use the 2-arg version of open for dup'ing for ### 5.6.x compatibility. 5.8.x can use 3-arg open ### see perldoc5.6.2 -f open for details open( $fh, $redir . fileno($glob) ) or ( Carp::carp(loc("Could not restore '$name': %1", $!)), return ); ### close this FD, we're not using it anymore close $glob; } return 1; } } sub _debug { my $self = shift; my $msg = shift or return; my $level = shift || 0; local $Carp::CarpLevel += $level; Carp::carp($msg); return 1; } sub _pp_child_error { my $self = shift; my $cmd = shift or return; my $ce = shift or return; my $pp_cmd = ref $cmd ? "@$cmd" : $cmd; my $str; if( $ce == -1 ) { ### Include $! in the error message, so that the user can ### see 'No such file or directory' versus 'Permission denied' ### versus 'Cannot fork' or whatever the cause was. $str = "Failed to execute '$pp_cmd': $!"; } elsif ( $ce & 127 ) { ### some signal $str = loc( "'%1' died with signal %2, %3 coredump", $pp_cmd, ($ce & 127), ($ce & 128) ? 'with' : 'without'); } else { ### Otherwise, the command run but gave error status. $str = "'$pp_cmd' exited with value " . ($ce >> 8); } $self->_debug( "# Child error '$ce' translated to: $str" ) if $DEBUG; return $str; } 1; __END__ =head2 $q = QUOTE Returns the character used for quoting strings on this platform. This is usually a C<'> (single quote) on most systems, but some systems use different quotes. For example, C uses C<"> (double quote). You can use it as follows: use IPC::Cmd qw[run QUOTE]; my $cmd = q[echo ] . QUOTE . q[foo bar] . QUOTE; This makes sure that C is treated as a string, rather than two separate arguments to the C function. =head1 HOW IT WORKS C will try to execute your command using the following logic: =over 4 =item * If you have C installed, and the variable C<$IPC::Cmd::USE_IPC_RUN> is set to true (See the L<"Global Variables"> section) use that to execute the command. You will have the full output available in buffers, interactive commands are sure to work and you are guaranteed to have your verbosity settings honored cleanly. =item * Otherwise, if the variable C<$IPC::Cmd::USE_IPC_OPEN3> is set to true (See the L<"Global Variables"> section), try to execute the command using L. Buffers will be available on all platforms, interactive commands will still execute cleanly, and also your verbosity settings will be adhered to nicely; =item * Otherwise, if you have the C argument set to true, we fall back to a simple C call. We cannot capture any buffers, but interactive commands will still work. =item * Otherwise we will try and temporarily redirect STDERR and STDOUT, do a C call with your command and then re-open STDERR and STDOUT. This is the method of last resort and will still allow you to execute your commands cleanly. However, no buffers will be available. =back =head1 Global Variables The behaviour of IPC::Cmd can be altered by changing the following global variables: =head2 $IPC::Cmd::VERBOSE This controls whether IPC::Cmd will print any output from the commands to the screen or not. The default is 0. =head2 $IPC::Cmd::USE_IPC_RUN This variable controls whether IPC::Cmd will try to use L when available and suitable. =head2 $IPC::Cmd::USE_IPC_OPEN3 This variable controls whether IPC::Cmd will try to use L when available and suitable. Defaults to true. =head2 $IPC::Cmd::WARN This variable controls whether run-time warnings should be issued, like the failure to load an C module you explicitly requested. Defaults to true. Turn this off at your own risk. =head2 $IPC::Cmd::INSTANCES This variable controls whether C will return all instances of the binary it finds in the C when called in a list context. Defaults to false, set to true to enable the described behaviour. =head2 $IPC::Cmd::ALLOW_NULL_ARGS This variable controls whether C will remove any empty/null arguments it finds in command arguments. Defaults to false, so it will remove null arguments. Set to true to allow them. =head1 Caveats =over 4 =item Whitespace and IPC::Open3 / system() When using C or C, if you provide a string as the C argument, it is assumed to be appropriately escaped. You can use the C constant to use as a portable quote character (see above). However, if you provide an array reference, special rules apply: If your command contains B (< > | &), it will be internally stringified before executing the command, to avoid that these special characters are escaped and passed as arguments instead of retaining their special meaning. However, if the command contained arguments that contained whitespace, stringifying the command would lose the significance of the whitespace. Therefore, C will quote any arguments containing whitespace in your command if the command is passed as an arrayref and contains special characters. =item Whitespace and IPC::Run When using C, if you provide a string as the C argument, the string will be split on whitespace to determine the individual elements of your command. Although this will usually just Do What You Mean, it may break if you have files or commands with whitespace in them. If you do not wish this to happen, you should provide an array reference, where all parts of your command are already separated out. Note however, if there are extra or spurious whitespaces in these parts, the parser or underlying code may not interpret it correctly, and cause an error. Example: The following code gzip -cdf foo.tar.gz | tar -xf - should either be passed as "gzip -cdf foo.tar.gz | tar -xf -" or as ['gzip', '-cdf', 'foo.tar.gz', '|', 'tar', '-xf', '-'] But take care not to pass it as, for example ['gzip -cdf foo.tar.gz', '|', 'tar -xf -'] Since this will lead to issues as described above. =item IO Redirect Currently it is too complicated to parse your command for IO redirections. For capturing STDOUT or STDERR there is a work around however, since you can just inspect your buffers for the contents. =item Interleaving STDOUT/STDERR Neither IPC::Run nor IPC::Open3 can interleave STDOUT and STDERR. For short bursts of output from a program, e.g. this sample, for ( 1..4 ) { $_ % 2 ? print STDOUT $_ : print STDERR $_; } IPC::[Run|Open3] will first read all of STDOUT, then all of STDERR, meaning the output looks like '13' on STDOUT and '24' on STDERR, instead of 1 2 3 4 This has been recorded in L as bug #37532: Unable to interleave STDOUT and STDERR. =back =head1 See Also L, L =head1 ACKNOWLEDGEMENTS Thanks to James Mastros and Martijn van der Streek for their help in getting L to behave nicely. Thanks to Petya Kohts for the C code. =head1 BUG REPORTS Please report bugs or other issues to Ebug-ipc-cmd@rt.cpan.orgE. =head1 AUTHOR Original author: Jos Boumans Ekane@cpan.orgE. Current maintainer: Chris Williams Ebingos@cpan.orgE. =head1 COPYRIGHT This library is free software; you may redistribute and/or modify it under the same terms as Perl itself. =cut PKu]7cNNSystem/Simple.pmnu[package IPC::System::Simple; # ABSTRACT: Run commands simply, with detailed diagnostics use 5.006; use strict; use warnings; use re 'taint'; use Carp; use List::Util qw(first); use Scalar::Util qw(tainted); use Config; use constant WINDOWS => ($^O eq 'MSWin32'); use constant VMS => ($^O eq 'VMS'); BEGIN { # It would be lovely to use the 'if' module here, but it didn't # enter core until 5.6.2, and we want to keep 5.6.0 compatibility. if (WINDOWS) { ## no critic (ProhibitStringyEval) eval q{ use Win32::Process qw(INFINITE NORMAL_PRIORITY_CLASS); use File::Spec; use Win32; # This uses the same rules as the core win32.c/get_shell() call. use constant WINDOWS_SHELL => eval { Win32::IsWinNT() } ? [ qw(cmd.exe /x/d/c) ] : [ qw(command.com /c) ]; # These are used when invoking _win32_capture use constant NO_SHELL => 0; use constant USE_SHELL => 1; }; ## use critic # Die nosily if any of the above broke. die $@ if $@; } } # Note that we don't use WIFSTOPPED because perl never uses # the WUNTRACED flag, and hence will never return early from # system() if the child processes is suspended with a SIGSTOP. use POSIX qw(WIFEXITED WEXITSTATUS WIFSIGNALED WTERMSIG); use constant FAIL_START => q{"%s" failed to start: "%s"}; use constant FAIL_PLUMBING => q{Error in IPC::System::Simple plumbing: "%s" - "%s"}; use constant FAIL_CMD_BLANK => q{Entirely blank command passed: "%s"}; use constant FAIL_INTERNAL => q{Internal error in IPC::System::Simple: "%s"}; use constant FAIL_TAINT => q{%s called with tainted argument "%s"}; use constant FAIL_TAINT_ENV => q{%s called with tainted environment $ENV{%s}}; use constant FAIL_SIGNAL => q{"%s" died to signal "%s" (%d)%s}; use constant FAIL_BADEXIT => q{"%s" unexpectedly returned exit value %d}; use constant FAIL_UNDEF => q{%s called with undefined command}; use constant FAIL_POSIX => q{IPC::System::Simple does not understand the POSIX error '%s'. Please check http://search.cpan.org/perldoc?IPC::System::Simple to see if there is an updated version. If not please report this as a bug to http://rt.cpan.org/Public/Bug/Report.html?Queue=IPC-System-Simple}; # On Perl's older than 5.8.x we can't assume that there'll be a # $^{TAINT} for us to check, so we assume that our args may always # be tainted. use constant ASSUME_TAINTED => ($] < 5.008); use constant EXIT_ANY_CONST => -1; # Used internally use constant EXIT_ANY => [ EXIT_ANY_CONST ]; # Exported use constant UNDEFINED_POSIX_RE => qr{not (?:defined|a valid) POSIX macro|not implemented on this architecture}; require Exporter; our @ISA = qw(Exporter); our @EXPORT_OK = qw( capture capturex run runx system systemx $EXITVAL EXIT_ANY ); our $VERSION = '1.25'; # VERSION : From dzil our $EXITVAL = -1; my @Signal_from_number = split(' ', $Config{sig_name}); # Environment variables we don't want to see tainted. my @Check_tainted_env = qw(PATH IFS CDPATH ENV BASH_ENV); if (WINDOWS) { push(@Check_tainted_env, 'PERL5SHELL'); } if (VMS) { push(@Check_tainted_env, 'DCL$PATH'); } # Not all systems implement the WIFEXITED calls, but POSIX # will always export them (even if they're just stubs that # die with an error). Test for the presence of a working # WIFEXITED and friends, or define our own. eval { WIFEXITED(0); }; if ($@ =~ UNDEFINED_POSIX_RE) { no warnings 'redefine'; ## no critic *WIFEXITED = sub { not $_[0] & 0xff }; *WEXITSTATUS = sub { $_[0] >> 8 }; *WIFSIGNALED = sub { $_[0] & 127 }; *WTERMSIG = sub { $_[0] & 127 }; } elsif ($@) { croak sprintf FAIL_POSIX, $@; } # None of the POSIX modules I've found define WCOREDUMP, although # many systems define it. Check the POSIX module in the hope that # it may actually be there. # TODO: Ideally, $NATIVE_WCOREDUMP should be a constant. my $NATIVE_WCOREDUMP; eval { POSIX::WCOREDUMP(1); }; if ($@ =~ UNDEFINED_POSIX_RE) { *WCOREDUMP = sub { $_[0] & 128 }; $NATIVE_WCOREDUMP = 0; } elsif ($@) { croak sprintf FAIL_POSIX, $@; } else { # POSIX actually has it defined! Huzzah! *WCOREDUMP = \&POSIX::WCOREDUMP; $NATIVE_WCOREDUMP = 1; } sub _native_wcoredump { return $NATIVE_WCOREDUMP; } # system simply calls run *system = \&run; *systemx = \&runx; # run is our way of running a process with system() semantics sub run { _check_taint(@_); my ($valid_returns, $command, @args) = _process_args(@_); # If we have arguments, we really want to call systemx, # so we do so. if (@args) { return systemx($valid_returns, $command, @args); } # Without arguments, we're calling system, and checking # the results. # We're throwing our own exception on command not found, so # we don't need a warning from Perl. { # silence 'Statement unlikely to be reached' warning no warnings 'exec'; ## no critic CORE::system($command,@args); } return _process_child_error($?,$command,$valid_returns); } # runx is just like system/run, but *never* invokes the shell. sub runx { _check_taint(@_); my ($valid_returns, $command, @args) = _process_args(@_); if (WINDOWS) { our $EXITVAL = -1; my $pid = _spawn_or_die($command, "$command @args"); $pid->Wait(INFINITE); # Wait for process exit. $pid->GetExitCode($EXITVAL); return _check_exit($command,$EXITVAL,$valid_returns); } # If system() fails, we throw our own exception. We don't # need to have perl complain about it too. no warnings; ## no critic CORE::system { $command } $command, @args; return _process_child_error($?, $command, $valid_returns); } # capture is our way of running a process with backticks/qx semantics sub capture { _check_taint(@_); my ($valid_returns, $command, @args) = _process_args(@_); if (@args) { return capturex($valid_returns, $command, @args); } if (WINDOWS) { # USE_SHELL really means "You may use the shell if you need it." return _win32_capture(USE_SHELL, $valid_returns, $command, @args); } our $EXITVAL = -1; my $wantarray = wantarray(); # We'll produce our own warnings on failure to execute. no warnings 'exec'; ## no critic if ($wantarray) { my @results = qx($command); _process_child_error($?,$command,$valid_returns); return @results; } my $results = qx($command); _process_child_error($?,$command,$valid_returns); return $results; } # _win32_capture implements the capture and capurex commands on Win32. # We need to wrap the whole internals of this sub into # an if (WINDOWS) block to avoid it being compiled on non-Win32 systems. sub _win32_capture { if (not WINDOWS) { croak sprintf(FAIL_INTERNAL, "_win32_capture called when not under Win32"); } else { my ($use_shell, $valid_returns, $command, @args) = @_; my $wantarray = wantarray(); # Perl doesn't support multi-arg open under # Windows. Perl also doesn't provide very good # feedback when normal backtails fail, either; # it returns exit status from the shell # (which is indistinguishable from the command # running and producing the same exit status). # As such, we essentially have to write our own # backticks. # We start by dup'ing STDOUT. open(my $saved_stdout, '>&', \*STDOUT) ## no critic or croak sprintf(FAIL_PLUMBING, "Can't dup STDOUT", $!); # We now open up a pipe that will allow us to # communicate with the new process. pipe(my ($read_fh, $write_fh)) or croak sprintf(FAIL_PLUMBING, "Can't create pipe", $!); # Allow CRLF sequences to become "\n", since # this is what Perl backticks do. binmode($read_fh, ':crlf'); # Now we re-open our STDOUT to $write_fh... open(STDOUT, '>&', $write_fh) ## no critic or croak sprintf(FAIL_PLUMBING, "Can't redirect STDOUT", $!); # If we have args, or we're told not to use the shell, then # we treat $command as our shell. Otherwise we grub around # in our command to look for a command to run. # # Note that we don't actually *use* the shell (although in # a future version we might). Being told not to use the shell # (capturex) means we treat our command as really being a command, # and not a command line. my $exe = @args ? $command : (! $use_shell) ? $command : $command =~ m{^"([^"]+)"}x ? $1 : $command =~ m{(\S+) }x ? $1 : croak sprintf(FAIL_CMD_BLANK, $command); # And now we spawn our new process with inherited # filehandles. my $err; my $pid = eval { _spawn_or_die($exe, "$command @args"); } or do { $err = $@; }; # Regardless of whether our command ran, we must restore STDOUT. # RT #48319 open(STDOUT, '>&', $saved_stdout) ## no critic or croak sprintf(FAIL_PLUMBING,"Can't restore STDOUT", $!); # And now, if there was an actual error , propagate it. die $err if defined $err; # If there's an error from _spawn_or_die # Clean-up the filehandles we no longer need... close($write_fh) or croak sprintf(FAIL_PLUMBING,q{Can't close write end of pipe}, $!); close($saved_stdout) or croak sprintf(FAIL_PLUMBING,q{Can't close saved STDOUT}, $!); # Read the data from our child... my (@results, $result); if ($wantarray) { @results = <$read_fh>; } else { $result = join("",<$read_fh>); } # Tidy up our windows process and we're done! $pid->Wait(INFINITE); # Wait for process exit. $pid->GetExitCode($EXITVAL); _check_exit($command,$EXITVAL,$valid_returns); return $wantarray ? @results : $result; } } # capturex() is just like backticks/qx, but never invokes the shell. sub capturex { _check_taint(@_); my ($valid_returns, $command, @args) = _process_args(@_); our $EXITVAL = -1; my $wantarray = wantarray(); if (WINDOWS) { return _win32_capture(NO_SHELL, $valid_returns, $command, @args); } # We can't use a multi-arg piped open here, since 5.6.x # doesn't like them. Instead we emulate what 5.8.x does, # which is to create a pipe(), set the close-on-exec flag # on the child, and the fork/exec. If the exec fails, the # child writes to the pipe. If the exec succeeds, then # the pipe closes without data. pipe(my ($read_fh, $write_fh)) or croak sprintf(FAIL_PLUMBING, "Can't create pipe", $!); # This next line also does an implicit fork. my $pid = open(my $pipe, '-|'); ## no critic if (not defined $pid) { croak sprintf(FAIL_START, $command, $!); } elsif (not $pid) { # Child process, execs command. close($read_fh); # TODO: 'no warnings exec' doesn't get rid # of the 'unlikely to be reached' warnings. # This is a bug in perl / perldiag / perllexwarn / warnings. no warnings; ## no critic CORE::exec { $command } $command, @args; # Oh no, exec fails! Send the reason why to # the parent. print {$write_fh} int($!); exit(-1); } { # In parent process. close($write_fh); # Parent process, check for child error. my $error = <$read_fh>; # Tidy up our pipes. close($read_fh); # Check for error. if ($error) { # Setting $! to our child error number gives # us nice looking strings when printed. local $! = $error; croak sprintf(FAIL_START, $command, $!); } } # Parent process, we don't care about our pid, but we # do go and read our pipe. if ($wantarray) { my @results = <$pipe>; close($pipe); _process_child_error($?,$command,$valid_returns); return @results; } # NB: We don't check the return status on close(), since # on failure it sets $?, which we then inspect for more # useful information. my $results = join("",<$pipe>); close($pipe); _process_child_error($?,$command,$valid_returns); return $results; } # Tries really hard to spawn a process under Windows. Returns # the pid on success, or undef on error. sub _spawn_or_die { # We need to wrap practically the entire sub in an # if block to ensure it doesn't get compiled under non-Win32 # systems. Compiling on these systems would not only be a # waste of time, but also results in complaints about # the NORMAL_PRIORITY_CLASS constant. if (not WINDOWS) { croak sprintf(FAIL_INTERNAL, "_spawn_or_die called when not under Win32"); } else { my ($orig_exe, $cmdline) = @_; my $pid; my $exe = $orig_exe; # If our command doesn't have an extension, add one. $exe .= $Config{_exe} if ($exe !~ m{\.}); Win32::Process::Create( $pid, $exe, $cmdline, 1, NORMAL_PRIORITY_CLASS, "." ) and return $pid; my @path = split(/;/,$ENV{PATH}); foreach my $dir (@path) { my $fullpath = File::Spec->catfile($dir,$exe); # We're using -x here on the assumption that stat() # is faster than spawn, so trying to spawn a process # for each path element will be unacceptably # inefficient. if (-x $fullpath) { Win32::Process::Create( $pid, $fullpath, $cmdline, 1, NORMAL_PRIORITY_CLASS, "." ) and return $pid; } } croak sprintf(FAIL_START, $orig_exe, $^E); } } # Complain on tainted arguments or environment. # ASSUME_TAINTED is true for 5.6.x, since it's missing ${^TAINT} sub _check_taint { return if not (ASSUME_TAINTED or ${^TAINT}); my $caller = (caller(1))[3]; foreach my $var (@_) { if (tainted $var) { croak sprintf(FAIL_TAINT, $caller, $var); } } foreach my $var (@Check_tainted_env) { if (tainted $ENV{$var} ) { croak sprintf(FAIL_TAINT_ENV, $caller, $var); } } return; } # This subroutine performs the difficult task of interpreting # $?. It's not intended to be called directly, as it will # croak on errors, and its implementation and interface may # change in the future. sub _process_child_error { my ($child_error, $command, $valid_returns) = @_; $EXITVAL = -1; my $coredump = WCOREDUMP($child_error); # There's a bug in perl 5.10.0 where if the system # does not provide a native WCOREDUMP, then $? will # never contain coredump information. This code # checks to see if we have the bug, and works around # it if needed. if ($] >= 5.010 and not $NATIVE_WCOREDUMP) { $coredump ||= WCOREDUMP( ${^CHILD_ERROR_NATIVE} ); } if ($child_error == -1) { croak sprintf(FAIL_START, $command, $!); } elsif ( WIFEXITED( $child_error ) ) { $EXITVAL = WEXITSTATUS( $child_error ); return _check_exit($command,$EXITVAL,$valid_returns); } elsif ( WIFSIGNALED( $child_error ) ) { my $signal_no = WTERMSIG( $child_error ); my $signal_name = $Signal_from_number[$signal_no] || "UNKNOWN"; croak sprintf FAIL_SIGNAL, $command, $signal_name, $signal_no, ($coredump ? " and dumped core" : ""); } croak sprintf(FAIL_INTERNAL, qq{'$command' ran without exit value or signal}); } # A simple subroutine for checking exit values. Results in better # assurance of consistent error messages, and better forward support # for new features in I::S::S. sub _check_exit { my ($command, $exitval, $valid_returns) = @_; # If we have a single-value list consisting of the EXIT_ANY # value, then we're happy with whatever exit value we're given. if (@$valid_returns == 1 and $valid_returns->[0] == EXIT_ANY_CONST) { return $exitval; } if (not defined first { $_ == $exitval } @$valid_returns) { croak sprintf FAIL_BADEXIT, $command, $exitval; } return $exitval; } # This subroutine simply determines a list of valid returns, the command # name, and any arguments that we need to pass to it. sub _process_args { my $valid_returns = [ 0 ]; my $caller = (caller(1))[3]; if (not @_) { croak "$caller called with no arguments"; } if (ref $_[0] eq "ARRAY") { $valid_returns = shift(@_); } if (not @_) { croak "$caller called with no command"; } my $command = shift(@_); if (not defined $command) { croak sprintf( FAIL_UNDEF, $caller ); } return ($valid_returns,$command,@_); } 1; __END__ =head1 NAME IPC::System::Simple - Run commands simply, with detailed diagnostics =head1 SYNOPSIS use IPC::System::Simple qw(system systemx capture capturex); system("some_command"); # Command succeeds or dies! system("some_command",@args); # Succeeds or dies, avoids shell if @args systemx("some_command",@args); # Succeeds or dies, NEVER uses the shell # Capture the output of a command (just like backticks). Dies on error. my $output = capture("some_command"); # Just like backticks in list context. Dies on error. my @output = capture("some_command"); # As above, but avoids the shell if @args is non-empty my $output = capture("some_command", @args); # As above, but NEVER invokes the shell. my $output = capturex("some_command", @args); my @output = capturex("some_command", @args); =head1 DESCRIPTION Calling Perl's in-built C function is easy, determining if it was successful is I. Let's face it, C<$?> isn't the nicest variable in the world to play with, and even if you I check it, producing a well-formatted error string takes a lot of work. C takes the hard work out of calling external commands. In fact, if you want to be really lazy, you can just write: use IPC::System::Simple qw(system); and all of your C commands will either succeed (run to completion and return a zero exit value), or die with rich diagnostic messages. The C module also provides a simple replacement to Perl's backticks operator. Simply write: use IPC::System::Simple qw(capture); and then use the L command just like you'd use backticks. If there's an error, it will die with a detailed description of what went wrong. Better still, you can even use C to run the equivalent of backticks, but without the shell: use IPC::System::Simple qw(capturex); my $result = capturex($command, @args); If you want more power than the basic interface, including the ability to specify which exit values are acceptable, trap errors, or process diagnostics, then read on! =head1 ADVANCED SYNOPSIS use IPC::System::Simple qw( capture capturex system systemx run runx $EXITVAL EXIT_ANY ); # Run a command, throwing exception on failure run("some_command"); runx("some_command",@args); # Run a command, avoiding the shell # Do the same thing, but with the drop-in system replacement. system("some_command"); systemx("some_command", @args); # Run a command which must return 0..5, avoid the shell, and get the # exit value (we could also look at $EXITVAL) my $exit_value = runx([0..5], "some_command", @args); # The same, but any exit value will do. my $exit_value = runx(EXIT_ANY, "some_command", @args); # Capture output into $result and throw exception on failure my $result = capture("some_command"); # Check exit value from captured command print "some_command exited with status $EXITVAL\n"; # Captures into @lines, splitting on $/ my @lines = capture("some_command"); # Run a command which must return 0..5, capture the output into # @lines, and avoid the shell. my @lines = capturex([0..5], "some_command", @args); =head1 ADVANCED USAGE =head2 run() and system() C provides a subroutine called C, that executes a command using the same semantics is Perl's built-in C: use IPC::System::Simple qw(run); run("cat *.txt"); # Execute command via the shell run("cat","/etc/motd"); # Execute command without shell The primary difference between Perl's in-built system and the C command is that C will throw an exception on failure, and allows a list of acceptable exit values to be set. See L for further information. In fact, you can even have C replace the default C function for your package so it has the same behaviour: use IPC::System::Simple qw(system); system("cat *.txt"); # system now suceeds or dies! C and C are aliases to each other. See also L for variants of C and C that never invoke the shell, even with a single argument. =head2 capture() A second subroutine, named C executes a command with the same semantics as Perl's built-in backticks (and C): use IPC::System::Simple qw(capture); # Capture text while invoking the shell. my $file = capture("cat /etc/motd"); my @lines = capture("cat /etc/passwd"); However unlike regular backticks, which always use the shell, C will bypass the shell when called with multiple arguments: # Capture text while avoiding the shell. my $file = capture("cat", "/etc/motd"); my @lines = capture("cat", "/etc/passwd"); See also L for a variant of C that never invokes the shell, even with a single argument. =head2 runx(), systemx() and capturex() The C, C and C commands are identical to the multi-argument forms of C, C and C respectively, but I invoke the shell, even when called with a single argument. These forms are particularly useful when a command's argument list I be empty, for example: systemx($cmd, @args); The use of C here guarantees that the shell will I be invoked, even if C<@args> is empty. =head2 Exception handling In the case where the command returns an unexpected status, both C and C will throw an exception, which if not caught will terminate your program with an error. Capturing the exception is easy: eval { run("cat *.txt"); }; if ($@) { print "Something went wrong - $@\n"; } See the diagnostics section below for more details. =head3 Exception cases C considers the following to be unexpected, and worthy of exception: =over 4 =item * Failing to start entirely (eg, command not found, permission denied). =item * Returning an exit value other than zero (but see below). =item * Being killed by a signal. =item * Being passed tainted data (in taint mode). =back =head2 Exit values Traditionally, system commands return a zero status for success and a non-zero status for failure. C will default to throwing an exception if a non-zero exit value is returned. You may specify a range of values which are considered acceptable exit values by passing an I as the first argument. The special constant C can be used to allow I exit value to be returned. use IPC::System::Simple qw(run system capture EXIT_ANY); run( [0..5], "cat *.txt"); # Exit values 0-5 are OK system( [0..5], "cat *.txt"); # This works the same way my @lines = capture( EXIT_ANY, "cat *.txt"); # Any exit is fine. The C and replacement C subroutines returns the exit value of the process: my $exit_value = run( [0..5], "cat *.txt"); # OR: my $exit_value = system( [0..5] "cat *.txt"); print "Program exited with value $exit_value\n"; =head3 $EXITVAL The exit value of any command executed by C can always be retrieved from the C<$IPC::System::Simple::EXITVAL> variable: This is particularly useful when inspecting results from C, which returns the captured text from the command. use IPC::System::Simple qw(capture $EXITVAL EXIT_ANY); my @enemies_defeated = capture(EXIT_ANY, "defeat_evil", "/dev/mordor"); print "Program exited with value $EXITVAL\n"; C<$EXITVAL> will be set to C<-1> if the command did not exit normally (eg, being terminated by a signal) or did not start. In this situation an exception will also be thrown. =head2 WINDOWS-SPECIFIC NOTES As of C v0.06, the C subroutine I will make available the full 32-bit exit value on Win32 systems. This is different from the previous versions of C and from Perl's in-build C function, which can only handle 8-bit return values. The C subroutine always returns the 32-bit exit value under Windows. The C subroutine also never uses the shell, even when passed a single argument. Versions of C before v0.09 would not search the C environment variable when the multi-argument form of C was called. Versions from v0.09 onwards correctly search the path provided the command is provided including the extension (eg, C rather than just C, or C rather than just C). If no extension is provided, C<.exe> is assumed. Signals are not supported on Windows systems. Sending a signal to a Windows process will usually cause it to exit with the signal number used. =head1 DIAGNOSTICS =over 4 =item "%s" failed to start: "%s" The command specified did not even start. It may not exist, or you may not have permission to use it. The reason it could not start (as determined from C<$!>) will be provided. =item "%s" unexpectedly returned exit value %d The command ran successfully, but returned an exit value we did not expect. The value returned is reported. =item "%s" died to signal "%s" (%d) %s The command was killed by a signal. The name of the signal will be reported, or C if it cannot be determined. The signal number is always reported. If we detected that the process dumped core, then the string C is appended. =item IPC::System::Simple::%s called with no arguments You attempted to call C or C but did not provide any arguments at all. At the very lease you need to supply a command to run. =item IPC::System::Simple::%s called with no command You called C or C with a list of acceptable exit values, but no actual command. =item IPC::System::Simple::%s called with tainted argument "%s" You called C or C with tainted (untrusted) arguments, which is almost certainly a bad idea. To untaint your arguments you'll need to pass your data through a regular expression and use the resulting match variables. See L for more information. =item IPC::System::Simple::%s called with tainted environment $ENV{%s} You called C or C but part of your environment was tainted (untrusted). You should either delete the named environment variable before calling C, or set it to an untainted value (usually one set inside your program). See L for more information. =item Error in IPC::System::Simple plumbing: "%s" - "%s" Implementing the C command involves dark and terrible magicks involving pipes, and one of them has sprung a leak. This could be due to a lack of file descriptors, although there are other possibilities. If you are able to reproduce this error, you are encouraged to submit a bug report according to the L section below. =item Internal error in IPC::System::Simple: "%s" You've found a bug in C. Please check to see if an updated version of C is available. If not, please file a bug report according to the L section below. =item IPC::System::Simple::%s called with undefined command You've passed the undefined value as a command to be executed. While this is a very Zen-like action, it's not supported by Perl's current implementation. =back =head1 DEPENDENCIES This module depends upon L when used on Win32 system. C is bundled as a core module in ActivePerl 5.6 and above. There are no non-core dependencies on non-Win32 systems. =head1 COMPARISON TO OTHER APIs Perl provides a range of in-built functions for handling external commands, and CPAN provides even more. The C differentiates itself from other options by providing: =over 4 =item Extremely detailed diagnostics The diagnostics produced by C are designed to provide as much information as possible. Rather than requiring the developer to inspect C<$?>, C does the hard work for you. If an odd exit status is provided, you're informed of what it is. If a signal kills your process, you are informed of both its name and number. If tainted data or environment prevents your command from running, you are informed of exactly which datais =item Exceptions on failure C takes an aggressive approach to error handling. Rather than allow commands to fail silently, exceptions are thrown when unexpected results are seen. This allows for easy development using a try/catch style, and avoids the possibility of accidently continuing after a failed command. =item Easy access to exit status The C, C and C commands all set C<$EXITVAL>, making it easy to determine the exit status of a command. Additionally, the C and C interfaces return the exit status. =item Consistent interfaces When called with multiple arguments, the C, C and C interfaces I invoke the shell. This differs from the in-built Perl C command which may invoke the shell under Windows when called with multiple arguments. It differs from the in-built Perl backticks operator which always invokes the shell. =back =head1 BUGS When C is exported, the exotic form C is not supported. Attemping to use the exotic form is a syntax error. This affects the calling package I. Use C if you need it, or consider using the L module to replace C with lexical scope. Core dumps are only checked for when a process dies due to a signal. It is not believed there are any systems where processes can dump core without dying to a signal. C status is not checked, as perl never spawns processes with the C option. Signals are not supported under Win32 systems, since they don't work at all like Unix signals. Win32 singals cause commands to exit with a given exit value, which this modules I capture. Only 8-bit values are returned when C or C is called with a single value under Win32. Multi-argument calls to C and C, as well as the C and C always return the 32-bit Windows return values. =head2 Reporting bugs Before reporting a bug, please check to ensure you are using the most recent version of C. Your problem may have already been fixed in a new release. You can find the C bug-tracker at L . Please check to see if your bug has already been reported; if in doubt, report yours anyway. Submitting a patch and/or failing test case will greatly expedite the fixing of bugs. =head1 FEEDBACK If you find this module useful, please consider rating it on the CPAN Ratings service at L . The module author loves to hear how C has made your life better (or worse). Feedback can be sent to Epjf@perltraining.com.auE. =head1 SEE ALSO L uses C to provide succeed-or-die replacements to C (and other built-ins) with lexical scope. L, L, L, L, L, L, L =head1 AUTHOR Paul Fenwick Epjf@cpan.orgE =head1 COPYRIGHT AND LICENSE Copyright (C) 2006-2008 by Paul Fenwick This library is free software; you can redistribute it and/or modify it under the same terms as Perl itself, either Perl version 5.6.0 or, at your option, any later version of Perl 5 you may have available. =for Pod::Coverage WCOREDUMP =cut PKF]~<2<2Open3.pmnu[PKF]/~t2Open2.pmnu[PKS]L.2.2PBDriver/Files.pmnu[PKS]i\}} tDriver.pmnu[PKu]sCmd.pmnu[PKu]7cNNSystem/Simple.pmnu[PK7