2016-11-01 36 views
0

我試圖掃描使用Java API谷歌的BigTable的表中的所有行(1.3M〜行),但我得到以下錯誤:谷歌的Cloud Bigtable StatusRuntimeException錯誤掃描一個表中的所有行的時候

Error while reading table 'projects/firm-link-147413/instances/some-bigger-table/tables/media-location-demo' : Response was not consumed in time; terminating connection. (Possible causes: row size > 256MB, slow client data read, and network problems) 

整個數據集大小約爲2GB,並且單​​個行大小非常​​小(< 100k)。網絡連接非常適合下載和上傳。客戶端有能力讀取數據(我有類似的代碼,執行上傳時,它表現很好)。

感謝您的幫助!

整個錯誤是:

com.google.cloud.bigtable.grpc.io.IOExceptionWithStatus:錯誤響應流 在com.google.cloud.bigtable.grpc.scanner.ResultQueueEntry $ ExceptionResultQueueEntry.getResponseOrThrow (ResultQueueEntry.java:88)〜[bigtable-hbase-1.2-0.9.3.jar!/:na] at com.google.cloud.bigtable.grpc.scanner.ResponseQueueReader.getNextMergedRow(ResponseQueueReader.java:95)〜 [bigtable-hbase-1.2-0.9.3.jar!/:na] at com.google.cloud.bigtable.grpc.scanner.StreamingBigtableResultScanner.next(StreamingBigtableResultScanner.java:60)〜[bigtable-hbase-1.2-0.9 .jar!/:na] at com.google.cloud.bigtable.grpc.scanner.StreamingBigt您可以在com.google.cloud.bigtable.grpc.scanner.ResumingStreamingResultScanner.next(ResumingStreamingResultScanner.java:)上使用ableResultScanner.next(StreamingBigtableResultScanner.java:34)〜[bigtable-hbase-1.2-0.9.3.jar!/:na] 。 89)[bigtable-hbase-1.2-0.9.3.jar!/:na] at com.google.cloud.bigtable.grpc.scanner.ResumingStreamingResultScanner.next(ResumingStreamingResultScanner.java:35)[bigtable-hbase-1.2- 0.9.3.jar!/:na] at com.google.cloud.bigtable.hbase.adapters.read.BigtableResultScannerAdapter $ 1.next(BigtableResultScannerAdapter.java:58)[bigtable-hbase-1.2-0.9.3.jar! /:na] at org.apache.hadoop.hbase.client.AbstractClientScanner $ 1.hasNext(AbstractClientScanner.java:94)[hbase-client-1.2.1.jar!/:1.2.1] at com.shutterfly。 corp.migrator.locspec.service.BigTableConnector.scanAllRows(BigTableConnector.java:157)[classes!/:0.0.1-SNAPSHOT] at com.shutterfly.corp.migrator.locspec.service.MediaIALookupMigrator.verifyMigratedData(MediaIALookupMigrator.java:86)[classes!/:0.0.1-SNAPSHOT] at com.shutterfly.corp.migrator.locspec.LocspecMigratorApplication.main (LocalspecMigratorApplication.java:21)[classes!/:0.0.1-SNAPSHOT] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)〜[na:1.8.0_102] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl .java:62)〜[na:1.8.0_102] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)〜[na:1.8.0_102] at java.lang.reflect.Method.invoke(Method .java:498)〜[na:1.8.0_102] at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:48)[locspec-migrator-0.0.1-SNAPSHOT.jar:0.0.1- SNAPSHOT] at org.springframework.boot.loader.Launcher.launch(Launcher.java:87)[locspec-migrator-0.0.1-SNAPSHOT.jar:0.0.1-SNAPSHOT] at org.springframework.boot.loader.Launcher .launch(Launcher.java:50)[locspec-migrator-0.0.1-SNAPSHOT.jar:0.0.1-SNAPSHOT] at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:58)[locspec -migrator-0.0.1-SNAPSHOT.jar:0.0.1-SNAPSHOT] 引起:com.google.bigtable.repackaged.io.grpc.StatusRuntimeException:DEADLINE_EXCEEDED:讀取表'projects/firm-link-147413 /實例/一些更大的表/表/媒體位置演示':響應沒有及時消耗;終止連接。 (可能的原因:行大小> 256MB,緩慢的客戶端數據讀取和網絡問題) at com.google.bigtable.repackaged.io.grpc.Status.asRuntimeException(Status.java:536)〜[bigtable-hbase-1.2- 0.9.3.jar!/:na] at com.google.cloud。bigtable.grpc.scanner.StreamObserverAdapter.onClose(StreamObserverAdapter.java:61)〜[bigtable-hbase-1.2-0.9.3.jar!/:na] at com.google.cloud.bigtable.grpc.io.ChannelPool $ InstrumentedChannel $ 2.onClose(ChannelPool.java:201)〜[bigtable-hbase-1.2-0.9.3.jar!/:na] at com.google.bigtable.repackaged.io.grpc.internal.ClientCallImpl $ ClientStreamListenerImpl.close (ClientCallImpl.java:481)〜[bigtable-hbase-1.2-0.9.3.jar!/:na] at com.google.bigtable.repackaged.io.grpc.internal.ClientCallImpl $ ClientStreamListenerImpl.access $ 600(ClientCallImpl。 java:398)〜[bigtable-hbase-1.2-0.9.3.jar!/:na] at com.google.bigtable.repackaged.io.grpc.internal.ClientCallImpl $ ClientStreamListenerImpl $ 1StreamClosed.runInContext(ClientCallImpl.java: 513)〜[bigtable-hbase-1.2-0.9.3.jar!/:na] at com.google.bigtable.repackaged.io.grpc.internal.ContextRunnable.run(Con textRunnable.java:52)〜[bigtable-hbase-1.2-0.9.3.jar!/:na] at com.google.bigtable.repackaged.io.grpc.internal.SerializingExecutor $ TaskRunner.run(SerializingExecutor.java: 154)〜[bigtable-hbase-1.2-0.9.3.jar!/:na] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)〜[na:1.8.0_102] at java。 util.concurrent.ThreadPoolExecutor $ Worker.run(ThreadPoolExecutor.java:617)〜[na:1.8.0_102] at java.lang.Thread.run(Thread.java:745)〜[na:1.8.0_102]

回答

1

我正在看這個。這取決於一些特定於您的使用情況,特別是您正在跨大陸進行工作。你可以在Google Groups上發帖嗎?關於進一步指導的任何對話可能需要保密而不是公開,團體給我們提供了這種選擇。