有時長時間運行ssh命令會停止打印到標準輸出

Question

我一直在使用Perl :: Net :: SSH來自動運行遠程盒子上的一些腳本。但是，這些腳本中的一些需要很長時間才能完成（一兩個小時），有時我會停止從中獲取數據，而不會實際上丟失連接。

下面是我使用的代碼：

sub run_regression_tests { 
    for(my $i = 0; $i < @servers; $i++){ 
     my $inner = $users[$i]; 
     foreach(@$inner){ 
      my $user = $_; 
      my $server = $servers[$i]; 

      my $outFile; 
      open($outFile, ">" . $outputDir . $user . "@" . $server . ".log.txt"); 
      print $outFile "Opening connection to $user at $server on " . localtime() . "\n\n"; 
      close($outFile); 

      my $pid = $pm->start and next; 

       print "Connecting to $user@" . "$server...\n"; 

       my $hasWentToDownloadYet = 0; 
       my $ssh = Net::SSH::Perl->new($server, %sshParams); 
       $ssh->login($user, $password);    

       $ssh->register_handler("stdout", sub { 
        my($channel, $buffer) = @_;    
        my $outFile; 
        open($outFile, ">>", $outputDir . $user . "@" . $server . ".log.txt");     
        print $outFile $buffer->bytes;    
        close($outFile);     

        my @lines = split("\n", $buffer->bytes); 
        foreach(@lines){ 
         if($_ =~ m/REGRESSION TEST IS COMPLETE/){ 
          $ssh->_disconnect(); 

          if(!$hasWentToDownloadYet){ 
           $hasWentToDownloadYet = 1; 
           print "Caught exit signal.\n"; 
           print("Regression tests for ${user}\@${server} finised.\n"); 
           download_regression_results($user, $server); 
           $pm->finish; 
          } 
         } 
        } 

       }); 
       $ssh->register_handler("stderr", sub { 
        my($channel, $buffer) = @_;    
        my $outFile; 
        open($outFile, ">>", $outputDir . $user . "@" . $server . ".log.txt"); 

        print $outFile $buffer->bytes;    

        close($outFile);     
       }); 
       if($debug){ 
        $ssh->cmd('tail -fn 40 /GDS/gds/gdstest/t-gds-master/bin/comp.reg'); 
       }else{ 
        my ($stdout, $stderr, $exit) = $ssh->cmd('. ./.profile && cleanall && my.comp.reg'); 
        if(!$exit){ 
         print "SSH connection failed for ${user}\@${server} finised.\n"; 
        } 
       } 
       #$ssh->cmd('. ./.profile'); 

       if(!$hasWentToDownloadYet){ 
        $hasWentToDownloadYet = 1; 
        print("Regression tests for ${user}\@${server} finised.\n"); 
        download_regression_results($user, $server); 
       } 

      $pm->finish;   
     } 
    } 
    sleep(1); 
    print "\n\n\nAll tests started. Tests typically take 1 hour to complete.\n"; 
    print "If they take significantly less time, there could be an error.\n"; 
    print "\n\nNo output will be printed until all commands have executed and finished.\n"; 
    print "If you wish to watch the progress tail -f one of the logs this script produces.\n Example:\n\t" . 'tail -f ./gds1@tdgds10.log.txt' . "\n"; 
    $pm->wait_all_children; 
    print "\n\nAll Tests are Finished. \n"; 
} 

這裏是我的％sshParams：

my %sshParams = (
    protocol => '2', 
    port => '22', 
    options => [ 
     "TCPKeepAlive yes", 
     "ConenctTimeout 10", 
     "BatchMode yes" 
    ] 
); 

有時長時間運行命令的隨機一個剛剛停止印刷/燒製的標準輸出或標準錯誤事件並不會退出。 SSH連接不會死（據我所知），因爲$ssh->cmd仍然阻塞。

任何想法如何糾正這種行爲？

Answer 1

在您％sshParams哈希，您可能需要添加「TCPKEEPALIVE是」你的選擇：

$sshParams{'options'} = ["BatchMode yes", "TCPKeepAlive yes"]; 

這些選項可能會或可能不適合你，但TCPKEEPALIVE是我會建議設置用於任何長時間運行的SSH連接。如果您的路徑中有任何一種有狀態的防火牆，它可能會丟失狀態，如果它長時間沒有通過連接傳遞流量。

Answer 2

它可能由於您查看REGRESSION TEST IS COMPLETE標記的輸出的方式而失敗。它可能分成兩個不同的SSH數據包，所以你的回調將永遠不會發現它。

更好，使用時，它是這一個班輪做過結束遠程命令：

perl -pe 'BEGIN {$p = open STDIN, "my.comp.reg |" or die $!}; kill TERM => -$p if /REGRESSION TEST IS COMPLETE/}' 

否則，將關閉遠程連接，但不停止遠程進程，將留活着。

除此之外，你應該嘗試使用Net::OpenSSH或Net::OpenSSH::Parallel，而不是淨:: SSH :: Perl中的：

use Net::OpenSSH::Parallel; 

my $pssh = Net::OpenSSH::Parallel->new; 

for my $i (0..$#server) { 
    my $server = $server[$i]; 
    for my $user (@{$users[$ix]}) { 
     $pssh->add_host("$user\@$server", password => $password); 
    } 
} 

if ($debug) { 
    $pssh->all(cmd => { stdout_file => "$outputDir%USER%\@%HOST%.log.txt", 
         stderr_to_stdout => 1 }, 
       'fail -fn 40 /GDS/gds/gdstest/t-gds-master/bin/comp.reg'); 
} 
else { 
    $pssh->all(cmd => { stdout_file => "$outputDir%USER%\@%HOST%.log.txt", 
         stderr_to_stdout => 1 }, 
       '. ./.profile && cleanall && my.comp.reg'); 
} 

$pssh->all(scp_get => $remote_regression_results_path, "regression_results/%USER%\@%HOST%/"); 

$pssh->run;