2017-07-06 47 views
0

我正在清理我的生產數據集,我正在尋找一些無用的條目。Bigquery查找條目所屬的表

SELECT count(pid_like) FROM TABLE_DATE_RANGE(DATASET.TABLE_PRODUCTION_, DATE_ADD(CURRENT_TIMESTAMP(), -30, 'DAY'),CURRENT_TIMESTAMP()) where c1 is null and c2 is null and c3 is null and c4 is null 

所以我想刪除其中C1,C2,C3和C4空條目。

之前,我在最後30桌清潔爲由:

# done for each last 30 tables 
DELETE FROM DATASET.TABLE_PRODUCTION_YYYYMMDD where c1 is null and c2 is null and c2 is null and c4 is null. 

但清洗後,一些無用的條目將會保留,但我不能對他們是什麼臺找到。

回答

2

試試這個發現哪些錶行與空值:

#standardSQL 
SELECT 
    _TABLE_SUFFIX AS suffix, 
    COUNT(*) AS null_count 
FROM `DATASET.TABLE_PRODUCTION_*` 
WHERE _TABLE_SUFFIX BETWEEN 
    FORMAT_DATE('%Y%m%d', DATE_SUB(CURRENT_DATE(), INTERVAL 30 DAY)) AND 
    FORMAT_DATE('%Y%m%d', CURRENT_DATE()) AND 
    c1 IS NULL AND 
    c2 IS NULL AND 
    c3 IS NULL AND 
    c4 IS NULL 
GROUP BY suffix 
HAVING null_count > 0 
ORDER BY null_count DESC; 

它將與空行和計數返回後綴(日期)。

+0

不錯!你能指點我的相關文件嗎? –

+1

也許這個? https://cloud.google.com/bigquery/docs/querying-wildcard-tables#filtering_selected_tables_using_table_suffix。這裏是FORMAT_DATE,但您也可以在這裏找到其他功能:https://cloud.google.com/bigquery/docs/reference/standard-sql/functions-and-operators#format_date –

+0

非常感謝:) –