2016-11-22 97 views
1

我有一個HTTP線程代理,即從客戶的每個請求我產生一個線程統計:現在我想收集一些統計像每秒(PPS)比特每秒(BPS)和包。使用libpcap的收集上的連接

我喜歡我的代碼做的只有一兩件事,所以如果一個線程處理的連接,將不計算BPS和PPS爲每個數據包過,我要離開這個給另一個線程。

我爲來自客戶端的每個HTTP請求創建一個線程,並且如果代理成功連接到請求的遠程服務器,proxt將實際的HTTP請求發送到服務器,並且在路由數據之前,連接線程創建一個日誌線程:將計算bps和pps,直到連接打開。連接線程提供了關於要過濾哪些數據包(本地IP地址,本地端口,遠程IP地址,遠程端口)的日誌線程信息,以便每個日誌記錄線程僅過濾來自父連接線程的數據包。

我有問題,計算爲每個數據包BPS和PPS。

下面是記錄線程包我的循環拍攝的僞代碼:

// pcap variables 
pcap_t *handle; 
struct pcap_pkthdr *header; 
const u_char *pkt_data; 
// timevals used to calculate delay from last filtered packet 
struct timeval oldTimevalUpload; 
struct timeval oldTimevalDownload;  
memset(&oldTimevalUpload, 0, sizeof(oldTimevalUpload)); 
memset(&oldTimevalDownload, 0, sizeof(oldTimevalDownload)); 

// stopLogging is a boolean flag declared in connection "parent" thread: 
// it is set to false when connection thread has done sending and 
// receiving data and connection is going to be closed 
while (((res = pcap_next_ex(handle, &header, &pkt_data)) >= 0) && (!stopLogging)) { 
    // check res 
    if (packet is upload) { 
     struct timeval difference; 
     timeval_subtract(&difference, &(header->ts), &oldTimevalUpload); 
     long long delay = (difference.tv_sec * 1000000) + difference.tv_usec; 
     long long acceptedPackets = ((long long)(pkt_data)) * 1000000; 
     long long acceptedBits = ((long long)(pkt_data+8)) * 8 * 1000000; 
     long long pps = acceptedPackets/delay; 
     long long bps = acceptedBits/delay; 
     debugRed(host << ", UPLOAD DIVIDE ACCEPTED PKTS " << acceptedPackets << 
      " AND ACCEPTED BITS " << acceptedBits << " PER DELAY " << delay << 
      " IS PPS " << pps << " AND BPS " << bps); 
     oldTimevalUpload.tv_sec = header->ts.tv_sec; 
     oldTimevalUpload.tv_usec = header->ts.tv_usec; 
    } else if (packet is download) { 
     // basically the same as above 
    } 
} 
debug("Quit logging connection " << localIPaddr << ":" << localPort << " and " 
    << remoteIPaddr << ":" << remotePort); 
pcap_close(handle); 

下面是一個示例輸出:

www.netflix.com UPLOAD, DIVIDE ACCEPTED PKTS 99239440000000 AND ACCEPTED BITS 793915584000000 PER DELAY 1479811349890053 IS PPS 0 AND BPS 0 
www.netflix.com DOWNLOAD, DIVIDE ACCEPTED PKTS 99239440000000 AND ACCEPTED BITS 793915584000000 PER DELAY 1479811350032141 IS PPS 0 AND BPS 0 
www.netflix.com DOWNLOAD, DIVIDE ACCEPTED PKTS 99239440000000 AND ACCEPTED BITS 793915584000000 PER DELAY 4344 IS PPS 22845174953 AND BPS 182761414364 
www.netflix.com UPLOAD, DIVIDE ACCEPTED PKTS 99239440000000 AND ACCEPTED BITS 793915584000000 PER DELAY 146464 IS PPS 677568822 AND BPS 5420551015 
www.netflix.com DOWNLOAD, DIVIDE ACCEPTED PKTS 99239440000000 AND ACCEPTED BITS 793915584000000 PER DELAY 2815 IS PPS 35253797513 AND BPS 282030402841 
www.netflix.com UPLOAD, DIVIDE ACCEPTED PKTS 99239440000000 AND ACCEPTED BITS 793915584000000 PER DELAY 2808 IS PPS 35341680911 AND BPS 282733470085 
www.netflix.com DOWNLOAD, DIVIDE ACCEPTED PKTS 99239440000000 AND ACCEPTED BITS 793915584000000 PER DELAY 1120 IS PPS 88606642857 AND BPS 708853200000 
www.netflix.com UPLOAD, DIVIDE ACCEPTED PKTS 99239440000000 AND ACCEPTED BITS 793915584000000 PER DELAY 1134 IS PPS 87512733686 AND BPS 700101925925 
www.netflix.com UPLOAD, DIVIDE ACCEPTED PKTS 99239440000000 AND ACCEPTED BITS 793915584000000 PER DELAY 39658 IS PPS 2502381360 AND BPS 20019052498 
www.netflix.com DOWNLOAD, DIVIDE ACCEPTED PKTS 99239440000000 AND ACCEPTED BITS 793915584000000 PER DELAY 176317 IS PPS 562846690 AND BPS 4502773890 
www.netflix.com UPLOAD, DIVIDE ACCEPTED PKTS 99239440000000 AND ACCEPTED BITS 793915584000000 PER DELAY 136687 IS PPS 726034224 AND BPS 5808274261 

我從來不知道我的家庭網絡能夠承受超過500 MBps的,所以一定是錯的。

This page顯示瞭如何計算bps和pps,並且解釋了在acceptedBits中的移位,但是我要反正報告它。他說的話基本上

Starting from the beginning of <code>pkt_data</code>, you can see the first <code>long integer</code> bytes represent accepted bytes

,我所做的一切:在這裏你可以看到的功能pcap_next_ex第二和第三個參數!爲什麼我會變得如此龐大而奇怪的bps和pps?

在Ubuntu 14.04的工作;不知道如何檢查libpcap版本,但locate libpcap給出了這樣的:

/home/dexter/Desktop/wireshark-1.99.9/wiretap/libpcap.c 
/home/dexter/Desktop/wireshark-1.99.9/wiretap/libpcap.h 
/usr/lib/x86_64-linux-gnu/libpcap.a 
/usr/lib/x86_64-linux-gnu/libpcap.so 
/usr/lib/x86_64-linux-gnu/libpcap.so.0.8 
/usr/lib/x86_64-linux-gnu/libpcap.so.1.5.3 
/usr/share/doc/libpcap-dev 
/usr/share/doc/libpcap0.8 
/usr/share/doc/libpcap0.8-dev 
/usr/share/doc/libpcap-dev/changelog.Debian.gz 
/usr/share/doc/libpcap-dev/copyright 
/usr/share/doc/libpcap0.8/CREDITS.gz 
/usr/share/doc/libpcap0.8/README.Debian 
/usr/share/doc/libpcap0.8/README.gz 
/usr/share/doc/libpcap0.8/changelog.Debian.gz 
/usr/share/doc/libpcap0.8/copyright 
/usr/share/doc/libpcap0.8-dev/changelog.Debian.gz 
/usr/share/doc/libpcap0.8-dev/copyright 
/var/lib/dpkg/info/libpcap-dev.[list,md5sums] 
/var/lib/dpkg/info/libpcap0.8-dev.[list,md5sums,preinst] 
/var/lib/dpkg/info/libpcap0.8:amd64.[list,md5sums,postinst,postrm,shlibs,symbols] 

回答

1

在您的代碼:

long long acceptedPackets = ((long long)(pkt_data)) * 1000000; 
long long acceptedBits = ((long long)(pkt_data+8)) * 8 * 1000000; 

雖然pkt_data是一個指針。

你正在做的是基本上獲取分組數據的地址,將其轉換爲long long,加上8(對於第二行),將其乘以一個常數,並認爲這是你的值,這在語義上是不正確的。您應該取消引用該指針,並考慮您的數據類型(將pkt_data轉換爲指向long long的指針)。

在代碼:

long long acceptedPackets = (*(long long*)(pkt_data)) * 1000000; 
long long acceptedBits = (*(long long*)(pkt_data+8)) * 8 * 1000000; 
// this also works: 
//long long acceptedPackets = *(long long*)pkt_data * 1000000; 
//long long acceptedBits = *((long long*)pkt_data + 1) * 8 * 1000000; 

舉一個例子,看http://ideone.com/JqmRre

編輯

this guide

最後一個參數是最有趣的人全部,最多 混亂的平均新手pcap程序員。本發明的另一個指針 到u_char,並指向數據塊的第一個字節含有 整個分組

這意味着pkt_data是數據包的內容本身。除非數據包的前16個字節包含所需的信息(這是不真實的,因爲它包含原始數據包,因此它具有ETH,IP和TCP/UDP標頭),否則不能使用該數據。爲了獲得PPS指標,你必須在你的循環中實現一個簡單的計數器(因爲你每幀都要打印該指標,簡單的long long pps = (long long)(1.0/delay);應該就足夠了 - 請注意,除法處於浮點數。 。,你應該使用幀頭的信息,以便long long bps = (long long)(header->caplen * 8.0/delay);應該做

作爲一個側面說明,時間指標,因爲你正在使用C++ 11,請嘗試使用chrono它比timeval中更清晰,更安全:

添加一個#include <chrono>

您的最終代碼應該如下所示:

// pcap variables 
pcap_t *handle; 
struct pcap_pkthdr *header; 
const u_char *pkt_data; 
// Use of high_resolution_clock 
std::chrono::high_resolution_clock::time_point oldTimeUpload = std::chrono::high_resolution_clock::now(); 
std::chrono::high_resolution_clock::time_point oldTimeDownload = std::chrono::high_resolution_clock::now(); 

// stopLogging is a boolean flag declared in connection "parent" thread: 
// it is set to false when connection thread has done sending and 
// receiving data and connection is going to be closed 
while (((res = pcap_next_ex(handle, &header, &pkt_data)) >= 0) && (!stopLogging)) { 
    // check res 
    if (packet is upload) { 
     std::chrono::high_resolution_clock::time_point now = std::chrono::high_resolution_clock::now(); 
     long long delay = std::chrono::duration_cast<std::chrono::nanoseconds>(now - oldTimeUpload).count(); 
     long long pps = (long long)(1000000000.0/delay); 
     long long bps = (long long)(header->caplen * 8 * 1000000000.0/delay); 
     debugRed(host << ", UPLOAD DIVIDE PER DELAY " << delay << 
      " IS PPS " << pps << " AND BPS " << bps); 
     oldTimevalUpload.tv_sec = header->ts.tv_sec; 
     oldTimevalUpload.tv_usec = header->ts.tv_usec; 
    } else if (packet is download) { 
     // basically the same as above 
    } 
} 
debug("Quit logging connection " << localIPaddr << ":" << localPort << " and " 
    << remoteIPaddr << ":" << remotePort); 
pcap_close(handle); 
+0

我也跟着你的建議:我現在有負數XD 'www.youtube.com下載DIVIDE被接受PKTS -9153173178195537408及接受BITS -2444556543073261568 PER DELAY 1479847752267586 IS PPS -6185與BPS -1651 www.youtube.com DOWNLOAD DIVIDE ACCEPTED PKTS -9153173178195537408 AND ACCEPTED BITS -2444556543073261568 PER DELAY 2745 IS PPS -3334489318104020與BPS -890548831720678 www.youtube.com UPLOAD DIVIDE ACCEPTED PKTS -1866508824056485888 AND接受位-2458024441489261568 PER DELAY 1479847752270362是PPS -1261和BPS -1660'和s o on ... – elmazzun

+0

請檢查我的編輯 –

0

如何在另一個進程而不是另一個線程上執行pps和bps計算?我可以推薦HttpAnalyzer實用程序,它可以直接從網絡接口捕獲HTTP數據包,並計算pps,bps和更多的統計數據。

由於它是開源的,您可以更改代碼以符合您的目的或按原樣使用它。

下面是該實用程序的輸出示例:

STATS SUMMARY 
============= 

General stats 
-------------------- 

Sample time:          18.374 [Seconds] 
Number of HTTP packets:       5662 [Packets] 
Rate of HTTP packets:       291.910 [Packets/sec] 
Number of HTTP flows:        55 [Flows] 
Rate of HTTP flows:        2.836 [Flows/sec] 
Number of HTTP pipelining flows:      0 [Flows] 
Number of HTTP transactions:      322 [Transactions] 
Rate of HTTP transactions:      16.601 [Transactions/sec] 
Total HTTP data:        5916120 [Bytes] 
Rate of HTTP data:       305011.600 [Bytes/sec] 
Average packets per flow:      102.945 [Packets] 
Average transactions per flow:     5.963 [Transactions] 
Average data per flow:      107565.818 [Bytes] 

HTTP request stats 
-------------------- 

Number of HTTP requests:       323 [Requests] 
Rate of HTTP requests:       16.653 [Requests/sec] 
Total data in headers:       188596 [Bytes] 
Average header size:       583.889 [Bytes] 

HTTP response stats 
-------------------- 

Number of HTTP responses:       332 [Responses] 
Rate of HTTP responses:       17.117 [Responses/sec] 
Total data in headers:       119577 [Bytes] 
Average header size:       360.172 [Bytes] 
Num of responses with content-length:    320 [Responses] 
Total body size (may be compressed):   5409410 [Bytes] 
Average body size:       16904.406 [Bytes] 

HTTP request methods 
-------------------- 

| Method | Count | 
--------------------- 
| GET  | 321 | 
| POST  | 2  | 
--------------------- 

Hostnames count 
-------------------- 

| Hostname         | Count | 
---------------------------------------------------- 
| images1.teny.co.qq      | 180 | 
| www.teny.co.qq       | 82 | 
| go.teny.co.qq       | 14 | 
| www.niwwin.co.qq       | 8  | 
| az835984.vo.msecnd.net     | 5  | 
| asset.pagefair.com      | 3  | 
| b.scorecardresearch.com     | 3  | 
| cdn.oolala.com       | 3  | 
| asset.pagefair.net      | 2  | 
| dy2.teny.co.qq       | 2  | 
| ecdn.firstimpression.io     | 2  | 
| pagead2.googlesyndication.com   | 2  | 
| server.exposebox.com      | 2  | 
| totalmedia2.teny.co.qq     | 2  | 
| vrp.mybrain.com       | 1  | 
| trc.oolala.com       | 1  | 
| zdwidget3-bs.sphereup.com    | 1  | 
| vrt.mybrain.com       | 1  | 
| www.googletagmanager.com     | 1  | 
| a.visualrevenue.com      | 1  | 
| tpc.googlesyndication.com    | 1  | 
| static.dynamicyield.com     | 1  | 
| st.dynamicyield.com      | 1  | 
| sf.exposebox.com       | 1  | 
| mediadownload.teny.co.qq     | 1  | 
| cdn.firstimpression.io     | 1  | 
| ajax.googleapis.com      | 1  | 
---------------------------------------------------- 

Status code count 
-------------------- 

| Status Code     | Count | 
---------------------------------------- 
| 200 OK      | 327 | 
| 204 No Content    | 1  | 
| 301 Moved Permanently  | 1  | 
| 302 Moved Temporarily  | 1  | 
| 304 Not Modified    | 2  | 
---------------------------------------- 

Content-type count 
-------------------- 

| Content-type     | Count | 
------------------------------------------ 
| application/javascript   | 11 | 
| application/json    | 1  | 
| application/x-javascript  | 23 | 
| image/gif      | 22 | 
| image/jpeg      | 157 | 
| image/png      | 85 | 
| text/css      | 9  | 
| text/html      | 8  | 
| text/javascript    | 13 | 
------------------------------------------