Perl中的多線程

Question

請幫我做我的腳本多線程。我已閱讀了threads::shared模塊的文件，但它並沒有幫助瞭解如何做到這一點

use threads; 
use threads::shared; 
use LWP::UserAgent; 
use HTTP::Cookies; 

my $NUM_WORKERS = 2; 

sub worker { 
    my ($i) = @_; 
    my ($web, $ck) = browser(); 
    ($username, $password) = split ':', $acc; 
    my $url = 'http://www.site.ru/?tkn'. int(rand(10000)); 
    my $response = $web->post($url, Content => 
        [//////]); 
    while(1) 
    { 
     my $url = 'http://www.site.ru/dk?st.page='.$i.'&st.name=%D0%B0'; 
     my $response = $web->get($url); 
     @list = ($response->content =~ /card_wrp"><div class="photoWrapper"><div><a href="\/(.*?)\?/g); 
     @popl = ($response->content =~/<\/div><div class="info">(.*?)<\/div>/g); 

     for ($j = 0; $j <= scalar @list - 1; $j++) 
     { 
      $popl[$j] =~ s/&nbsp;//g; 
      open F, ">>gr.txt"; 
      print F $list[$j].':'.$popl[$j]. "\n"; 
      close F; 
     } 
     print "[+] Page $i \n"; 

    } 
} 

my $i :shared = 1; 
my $last = 79265; 
my @workers; 
for (1..$NUM_WORKERS) { 
    push @workers, async { 
     while (1) { 
     my $i; 
     { 
      lock $I; 
      return if $i == $last; 
      $i = ++$I; 
     } 
     worker($i); 
     } 
    }; 
} 

$_->join() for @workers; 
sub browser 
{ 
my $web = new LWP::UserAgent; 
my $ck = new HTTP::Cookies; 
    $web->cookie_jar($ck); 
    $web->agent('Opera/9.80 (Windows 7; U; en) Presto/2.9.168 Version/11.50'); 
    $web->requests_redirectable(0); 

    $web->timeout(5); 
return $web, $ck; 
} 
sub loadf { 
    open (F, "<".$_[0]) or erroropen($_[0]); 
    chomp(my @data = <F>); 
    close F; 
    return @data; 
} 

我不明白我需要什麼共享變量。非常感謝所有成員誰幫我

Answer 1

如果沒有線程，工作環會是什麼樣子像

for my $i (1..79265) { 
    worker($i); 
} 

的問題是，該變量不可共享和for保持一個不能共享的內部狀態，所以我們需要將它重寫成沒有這些問題的東西。

選項1：

my @a = 1..79265; 
while (@a) { 
    worker(shift(@a)); 
} 

選項2：

my $i = 0; 
while (++$i <= 79265) { 
    worker($i); 
} 

所有這些並行任一版本所需要的是確保@a/$i沒有時間之間改變你的檢查和當你使用它。這是通過添加一個鎖來完成的。

選項1：

my @a :shared = 1..79265; 
while (1) { 
    my $i; 
    { lock @a; return if !@a; $i = shift(@a); } 
    worker($i); 
} 

選項2：

my $I :shared = 1; 
while (1) { 
    my $i; 
    { lock $I; $i = $I; return if ++$I > 79265; } 
    worker($i); 
} 

選項1是如下（雖然也可以原樣使用）線程::隊列解決方案的基礎。選項2在下面按原樣使用。

---

我通常使用Thread::Queue或Thread::Queue::Any：

use threads; 
use Thread::Queue qw(); 

my $NUM_WORKERS = 5; 

sub worker { 
    my ($i) = @_; 
    ... put your download code here ... 
} 

my $q = Thread::Queue->new(); 
my @workers; 
for (1..$NUM_WORKERS) { 
    push @workers, async { 
     while (defined(my $i = $q->dequeue())) { 
     worker($i); 
     } 
    }; 
} 

$q->enqueue($_) for 1..79265; 
$q->enqueue(undef) for @workers; 
$_->join() for @workers; 

但是，我們可以很容易做到，而不在這裏：

use threads; 
use threads::shared; 

my $NUM_WORKERS = 5; 

sub worker { 
    my ($i) = @_; 
    ... put your download code here ... 
} 

my $I :shared = 1; 
my $last = 79265; 
my @workers; 
for (1..$NUM_WORKERS) { 
    push @workers, async { 
     while (1) { 
     my $i; 
     { 
      lock $I; 
      $i = $I; 
      return if ++$I > $last; 
     } 
     worker($i); 
     } 
    }; 
} 

$_->join() for @workers;