Azure Data Lake Store - 從文件讀取時出錯

我使用FileSystemOperationsExtensions.Open返回Stream的方法，我可以從中讀取數據。有時，當服務從流中讀取大文件（〜150-300 MB）服務得到以下情況除外：Azure Data Lake Store - 從文件讀取時出錯

System.IO.IOException: The read operation failed, see inner exception. ---> System.Net.WebException: The request was aborted: The request was canceled. 
at System.Net.ConnectStream.Read(Byte[] buffer, Int32 offset, Int32 size) 
at System.Net.Http.HttpClientHandler.WebExceptionWrapperStream.Read(Byte[] buffer, Int32 offset, Int32 count) 

"ClassName": "System.IO.IOException", 
"Message": "Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host." 
at System.Net.ConnectStream.Read(Byte[] buffer, Int32 offset, Int32 size)\r\n 
at System.Net.Http.HttpClientHandler.WebExceptionWrapperStream.Read(Byte[] buffer, Int32 offset, Int32 count)

而且它是隨機發生的。另外，我創建了DataLakeStoreFileSystemManagementClient類的對象，其中有60分鐘的超時時間，但這些錯誤發生在它之前。它可能需要3,10,20或任何分鐘。當然，我可以重新讀取偏移量，但它需要額外的開發時間。也許有另一種方法來避免這些例外。有人可以幫我嗎？

來源

2017-04-07 Andrei Barbolin

回答了這個問題：http://stackoverflow.com/questions/43400730/azure-data- lake-store-existing-connection-was-force-by-the-remote-ho –

我用270M +尺寸文件進行3次演示測試，它對我來說總是正常工作。請嘗試使用以下代碼對其進行測試。我們還可以從獲得更多的數據存儲演示代碼。

演示代碼：

var applicationId = "Application Id"; 
       var secretKey = "secretkey"; 
       var tenantId = "tenant id"; 
       var adlsAccountName = "Account name"; 
       var creds = ApplicationTokenProvider.LoginSilentAsync(tenantId, applicationId, secretKey).Result; 
       var adlsFileSystemClient = new DataLakeStoreFileSystemManagementClient(creds,clientTimeoutInMinutes:60); 
       var srcPath = "/mytempdir/ForDemoCode.zip"; 
       var destPath = @"c:\tom\ForDemoCode1.zip"; 

       Stopwatch stopWatch = new Stopwatch(); 
       stopWatch.Start(); 
       using (var stream = adlsFileSystemClient.FileSystem.Open(adlsAccountName, srcPath)) 
       using (var fileStream = new FileStream(destPath, FileMode.Create)) 
       { 
        stream.CopyTo(fileStream); 
       } 
       var file = new FileInfo(destPath); 
       Console.WriteLine($"File size :{file.Length}"); 
       stopWatch.Stop(); 
       // Get the elapsed time as a TimeSpan value. 
       TimeSpan ts = stopWatch.Elapsed; 
       // Format and display the TimeSpan value. 
       string elapsedTime = $"{ts.Hours:00}:{ts.Minutes:00}:{ts.Seconds:00}.{ts.Milliseconds/10:00}"; 
       Console.WriteLine("RunTime " + elapsedTime); 
       Console.ReadKey();

包的配置文件：

<?xml version="1.0" encoding="utf-8"?> 
<packages> 
    <package id="Microsoft.Azure.Management.DataLake.Store" version="2.1.1-preview" targetFramework="net452" /> 
    <package id="Microsoft.Azure.Management.DataLake.StoreUploader" version="1.0.0-preview" targetFramework="net452" /> 
    <package id="Microsoft.IdentityModel.Clients.ActiveDirectory" version="3.13.8" targetFramework="net452" /> 
    <package id="Microsoft.Rest.ClientRuntime" version="2.3.5" targetFramework="net452" /> 
    <package id="Microsoft.Rest.ClientRuntime.Azure" version="3.3.5" targetFramework="net452" /> 
    <package id="Microsoft.Rest.ClientRuntime.Azure.Authentication" version="2.2.0-preview" targetFramework="net452" /> 
    <package id="Newtonsoft.Json" version="9.0.2-beta1" targetFramework="net452" /> 
</packages>

來源

2017-04-25 08:56:44

，** ApplicationTokenProvider.LoginSilentAsync **的命名空間是什麼。我有這個錯誤。 **「名稱'ApplicationTokenProvider'在當前上下文中不存在」** –

在'Microsoft.Rest.Azure.Authentication'命名空間中，我還在packages.config文件部分中提到了它。請參考以下.dll文件'Microsoft.Rest.ClientRuntime.Azure.Authentication' –

，謝謝，它適用於我。 –

Azure Data Lake Store - 從文件讀取時出錯

回答

相關問題