1. 首頁
  2. 問答
  3. 從醜陋的命令行管道創建整潔的外殼腳本

從醜陋的命令行管道創建整潔的外殼腳本

2010-09-23 79 views
2

我寫了一個管道外殼命令，其中有多個管道很好用。我現在想把它放在一個（整齊）shell腳本的形式中。這裏是腳本：從醜陋的命令行管道創建整潔的外殼腳本

#!/bin/bash 
for number in `cat xmlEventLog_2010-03-23T* | sed -nr "/<event eventTimestamp/,/<\/event>/ {/event /{s/^.*$/\n/; p};/payloadType/{h; /protocol/ {s/.*protocol=\"([^\"]*)?\".*/protocol: \1/}; p; x; /type/ {s/.*type=\"([^\"]+)\".*/payload: \1/g}; /type/! {s/.*protocol=\"([^\"]+)\".*/payload: \1/g}; p};/sender/{/sccpAddress/ {s/.*sccpAddress=\"([^\"]*)?\".*/sccpAddress: \1/}; /sccpAddress/! {s/.*/sccpAddress: Unknown/}; p};/result /{s/.*value=\"([^\"]+)\".*/result: \1/g; p};/filter code/{s/.*type=\"([^\"]+)\".*/type: \1/g; p};}"| tee checkThis.txt| awk 'BEGIN{FS="\n"; RS=""; OFS=";"; ORS="\n"} $1~/result: Blocked|Modified/ && $2~/sccpAddress: 353201000001/ && $4~/payload: SMS-MO-FSM-INFO|SMS-MO-FSM/ {$1=$1 ""; print}' | sort | uniq -c| egrep "NUMBER_BLACKLIST|USER_BLACKLIST|NUMBER_WALLEDGARDEN|USER_WALLED_GARDEN|SERVICE_RESTRICTION|BLOCK_VOICE_TO_SMS|PEP_Blacklist_Whitelist" | awk '{print $1}'`; do fil="$fil+$number" 
done 
echo "fil is $fil"

我想整理它，以便它是可讀的。 for sed和awk管道的for循環看起來很醜。有沒有人得到建議來整理這個管道怪物。管道會阻止我把它分解成不同的線路嗎？

感謝

一個

如果複製上面的行到記事本，你會明白我的意思大約醜（但功能）

好鄉親。這是最後清理的版本。

有人提到event_structure函數完全可以在awk中完成。我想知道有沒有人能告訴我一個如何做到這一點的例子。記錄分隔符將被設置爲/ event，這將分隔事件，但它是我感興趣的events.txt（見下文）中的結構。數字結果並不重要。

代碼的核心是在event_structure函數中。我想解析出數據，並將其全部放入數據結構中，以供日後檢查時使用。以下工作正常。在以payloadType開頭的行上，我需要解析出2個值或將任何缺失值設置爲Unknown。這完全是awkable還是sed/awk組合我在這裏有最好的方法來做到這一點？

#!/bin/bash 

event_structure() { 
     sed -nr "/<event eventTimestamp/,/<\/event>/ { 
      /event /{s/^.*$/\n/; p} 
      /payloadType/{h; /protocol/ {s/.*protocol=\"([^\"]*)?\".*/protocol: \1/}; p; x; /type/ {s/.*type=\"([^\"]+)\".*/payload: \1/g}; /type/! {s/.*protocol=\"([^\"]+)\".*/payload: \1/g}; p} 
      /sender/{/sccpAddress/ {s/.*sccpAddress=\"([^\"]*)?\".*/sccpAddress: \1/}; /sccpAddress/! {s/.*/sccpAddress: Unknown/}; p} 
      /result /{s/.*value=\"([^\"]+)\".*/result: \1/g; p} 
      /filter code/{s/.*type=\"([^\"]+)\".*/type: \1/g; p};}" xmlEventLog_2010-03-23T* | 
     tee events.txt| 
     awk 'BEGIN{FS="\n"; RS=""; OFS=";"; ORS="\n"} 
     $1~/result: Blocked|Modified/ && $2~/sccpAddress: 353201000001/ && $4~/payload: SMS-MO-FSM-INFO|SMS-MO-FSM/ {$1=$1 ""; print}' 
} 

numbers=$(event_structure | sort | uniq -c | egrep "NUMBER_BLACKLIST|USER_BLACKLIST|NUMBER_WALLEDGARDEN|USER_WALLED_GARDEN|SERVICE_RESTRICTION|BLOCK_VOICE_TO_SMS|PEP_Blacklist_Whitelist" | awk '{print $1}') 
addition=`echo $numbers | tr -s ' \n\t' '+' | sed -e '1s/^/fil is /' -e '$s/+$//'` 
for number in $numbers 
do 
     fil="$fil+$number" 
done 
echo $addition=$(($fil))

這裏是產生的events.txt文件的一部分：

result: Blocked 
sccpAddress: 353869000000 
protocol: SMS 
payload: COPS 
type: SERVICE_BLACK_LIST 
result: Blocked 


result: Blocked 
sccpAddress: 353869000000 
protocol: SMS 
payload: COPS 
type: SERVICE_BLACK_LIST 
result: Blocked 

result: Modified 
sccpAddress: Unknown 
protocol: IM 
payload: IM 
type: NUMBER_BLACKLIST 
result: Modified 

result: Allowed 
sccpAddress: Unknown 
protocol: MM1 
payload: MM1

這裏是輸出：

$ ./bashShell.sh 
fil is 2+372+1+1+214+73+1+20=684

這裏只是函數調用的輸出：

$ ./bashShell.sh | head -10 
result: Blocked;sccpAddress: 353201000001;protocol: SMS;payload: SMS-MO-FSM;type: TEXT_ANALYSIS;result: Blocked 
result: Blocked;sccpAddress: 353201000002;protocol: SMS;payload: SMS-MT-FSM;type: TEXT_ANALYSIS;result: Blocked 
result: Blocked;sccpAddress: 353201000005;protocol: SMS;payload: SMS-MO-FSM;type: SERVICE_BLACKLIST;result: Blocked 
result: Blocked;sccpAddress: 353201000021;protocol: SMS;payload: SMS-MT-FSM;type: NUMBER_BLACKLIST;result: Blocked 
result: Blocked;sccpAddress: 353201000033;protocol: IM;payload: IM;type: NUMBER_BLACKLIST;result: Blocked 
result: Blocked;sccpAddress: 353401009001;protocol: SMS;payload: SMS-MO-FSM;type: NUMBER_BLACKLIST;result: Blocked 
result: Blocked;sccpAddress: 353201000001;protocol: SMS;payload: SMS-MO-FSM;type: NUMBER_BLACKLIST;result: Blocked 
result: Blocked;sccpAddress: 353201000005;protocol: SMS;payload: SMS-MO-FSM;type: NUMBER_BLACKLIST;result: Blocked 
result: Blocked;sccpAddress: 353401000001;protocol: SMS;payload: SMS-MO-FSM;type: NUMBER_BLACKLIST;result: Blocked 
result: Blocked;sccpAddress: 353201000001;protocol: SMS;payload: SMS-MO-FSM;type: NUMBER_BLACKLIST;result: Blocked

ps我把腳本命名爲ba shShell.sh無特殊原因

打破多行當A

來源

2010-09-23 amadain

回答

3

管道不攔你，但使用$(...)，而不是反引號的。像這樣的東西應該工作：

#!/bin/bash 

for number in $(
    cat xmlEventLog_2010-03-23T* | 
    sed -nr "/<event eventTimestamp/,/<\/event>/ {/event /{s/^.*$/\n/; p};/payloadType/{h; /protocol/ {s/.*protocol=\"([^\"]*)?\".*/protocol: \1/}; p; x; /type/ {s/.*type=\"([^\"]+)\".*/payload: \1/g}; /type/! {s/.*protocol=\"([^\"]+)\".*/payload: \1/g}; p};/sender/{/sccpAddress/ {s/.*sccpAddress=\"([^\"]*)?\".*/sccpAddress: \1/}; /sccpAddress/! {s/.*/sccpAddress: Unknown/}; p};/result /{s/.*value=\"([^\"]+)\".*/result: \1/g; p};/filter code/{s/.*type=\"([^\"]+)\".*/type: \1/g; p};}"| 
    tee checkThis.txt | 
    awk 'BEGIN{FS="\n"; RS=""; OFS=";"; ORS="\n"} $1~/result: Blocked|Modified/ && $2~/sccpAddress: 353201000001/ && $4~/payload: SMS-MO-FSM-INFO|SMS-MO-FSM/ {$1=$1 ""; print}' | 
    sort | 
    uniq -c | 
    egrep "NUMBER_BLACKLIST|USER_BLACKLIST|NUMBER_WALLEDGARDEN|USER_WALLED_GARDEN|SERVICE_RESTRICTION|BLOCK_VOICE_TO_SMS|PEP_Blacklist_Whitelist" | 
    awk '{print $1}' 
); do fil="$fil+$number" 
done 
echo "fil is $fil"

當然，更大的部分是分裂AWK和SED skripts成多行也...

但我相信，即使之後的結果將是還是相當不可讀。

我會建議在Perl，Ruby或任何其他比Bash更可讀的腳本語言中完全重寫腳本。這只是我個人經驗的建議 - 每次從shell腳本開始，我最終都會用Ruby重寫它。我喜歡Bash，但它似乎沒有規模。

來源

2010-09-23 13:36:05

+0

我也是用python寫的（非常漂亮），但是我喜歡把bash版本清理乾淨。謝謝 – amadain 2010-09-23 13:57:29

+0

反引號有什麼問題？ – 2010-09-23 17:28:05

+0

我和雷內在一起。您需要使用適合於處理XML文件的東西。你不應該嘗試使用正則表達式。不足之處是，'sed'部分可以在AWK中重寫，'egrep'和'uniq'可以在AWK中完成。如果你有'gawk'，你也可以做'排序'。但最後，您應該使用Python或Perl XML模塊。 – 2010-09-23 17:46:27

2

兩個小備註：

把「列表」在一個單獨的功能：

number_list() { 
    # complete pipe command list 
    # divided over multiple lines 
} 

for number in `number_list` 
do 
    # ... 
done

嘗試結合一些命令：該cat是不需要的，最終egrep和awk可以合併。

來源

2010-09-23 13:39:46 schot

+0

爲什麼貓不需要？ – amadain 2010-09-23 14:20:21

+1

@amadain：'sed'也需要多個文件參數，所以你可以替換'cat file ... | sed [pat]'與'sed [pat]文件...'。 – schot 2010-09-23 14:30:16

1

你可以加入使用TR和預先設置不同的令牌使用SED「FIL是」：

first-command \ 
    | second-command \ 
    | third-command \ 
    ... 
    | last-command

來源

2010-09-23 13:59:48

1

shell腳本居然是：

pipeline | tr -s ' \n\t' '+' | sed -e '1s/^/fil is /' -e '$s/+$//'

管道可以使用分隔成了多行簡單的部分。 sed腳本是可怕的位。這個腳本可以用這裏的文檔來改進，但是看到這個評論：

#!/bin/bash 

seds=/tmp/seds.$$ 
awks=/tmp/awks.$$ 
gres=/tmp/gres.$$ 

trap "rm -f $seds $awks $gres" 0 1 2 3 15 

# this is a noble and hairy attempt to parse xml with sed 
# it is extremely fragile and strongly dependent upon 
# the form of the source file never changing 
# I'm alternately proud or disgusted that I've been able 
# to get away with this 

cat > $seds <<'EOF' 
/<event eventTimestamp/,/<\/event>/ {/event /{s/^.*$/\n/; p}; 
/payloadType/{h; /protocol/ {s/.*protocol=\"([^\"]*)?\".*/protocol: \1/}; p; x; 
/type/ {s/.*type=\"([^\"]+)\".*/payload: \1/g}; 
/type/! {s/.*protocol=\"([^\"]+)\".*/payload: \1/g}; p}; 
/sender/{/sccpAddress/ {s/.*sccpAddress=\"([^\"]*)?\".*/sccpAddress: \1/}; 
/sccpAddress/! {s/.*/sccpAddress: Unknown/}; p}; 
/result /{s/.*value=\"([^\"]+)\".*/result: \1/g; p}; 
/filter code/{s/.*type=\"([^\"]+)\".*/type: \1/g; p};} 
EOF 

cat > $awks <<'EOF' 
BEGIN {FS="\n"; RS=""; OFS=";"; ORS="\n"} 
$1~/result: Blocked|Modified/ && \ 
$2~/sccpAddress: 353201000001/ && \ 
$4~/payload: SMS-MO-FSM-INFO|SMS-MO-FSM/ {$1=$1 ""; print} 
EOF 

cat > $gres <<EOF 
NUMBER_BLACKLIST 
USER_BLACKLIST 
NUMBER_WALLEDGARDEN 
USER_WALLED_GARDEN 
SERVICE_RESTRICTION 
BLOCK_VOICE_TO_SMS 
PEP_Blacklist_Whitelist 
EOF 

cat xmlEventLog_2010-03-23T* | \ 
sed -nr -f $seds | \ 
tee checkThis.txt | \ 
awk -f $awks | \ 
sort | uniq -c | \ 
fgrep -f $gres | \ 
awk '{print $1}'

來源

2010-09-23 14:55:29 msw

相關問題

 相關問題