Python3 BigQuery或Google Cloud Python通過HTTP代理

Question

如何路由BigQuery client通過調用HTTP代理？

在發佈之前，我嘗試了下面的內容，但它仍然沒有通過http代理進行路由。而谷歌雲服務憑據通過shell環境變量設置GOOGLE_APPLICATION_CREDENTIALS

import httplib2 
import socks 
import google.auth 

credentials, _ = google.auth.default() 
http_client = httplib2.Http(proxy_info = httplib2.ProxyInfo(socks.PROXY_TYPE_HTTP, 'someproxy', 80)); 

bigquery_client = bigquery.Client(credentials=credentials, _http=http_client) 

傳出通信（172.217.xx屬於googleapis.com）通過HTTP代理不路由，

$ netstat -nputw 
Local Address   Foreign Address 
x.x.x.x     172.217.6.234:443  SYN_SENT 

編輯：找到解決方案，發表它作爲答案。

Answer 1

回答這個問題我自己，因爲我找到了原因/解決方案。

原因：

谷歌雲的Python庫使用httplib2的，截至發稿httplib2的已兩個代碼庫爲Python 2和Python 3 Python的3.0版本httplib2的的未實現與襪子/代理支持。請參閱httplib2's repo#init_py。

解決方法：

有一個discussion從httplib2的移動谷歌雲，蟒蛇urllib3，但在平均時間可以使用httplib2shim

import google.auth 
import httplib2shim 
import google_auth_httplib2 

// More declarative way exists, but left for simplicity 
os.environ["HTTP_PROXY"] = "someproxy:80" 
os.environ["HTTPS_PROXY"] = "someproxy:80" 
http_client = httplib2shim.Http() 
credentials, _ = google.auth.default() 

# IMO, Following 2 lines should be done at the google-cloud-python 
# This exposes client speicific logic, and it already does that 
credentials = google.auth.credentials.with_scopes_if_required 
       (credentials, bigquery.Client.SCOPE) 
authed_http = google_auth_httplib2.AuthorizedHttp(credentials,http_client) 

bigquery_client = bigquery.Client(credentials=credentials, _http=authed_http) 

Answer 2

我發現創建這些憑證的唯一方法是直接在我的os環境中設置它。

假設你有你的json credential file已經，也許這會爲你工作：

import httplib2 
import socks 
import os 
import google.auth 

os.environ['GOOGLE_APPLICATION_CREDENTIALS'] = 'path/to/your/credentials_file.json' 

credentials, _ = google.auth.default() 
http_client = httplib2.Http(proxy_info = httplib2.ProxyInfo(socks.PROXY_TYPE_HTTP, 'someproxy', 80)) 
bigquery_client = bigquery.Client(_http=http_client, credentials=credentials)