1. 首頁
  2. 問答
  3. MTOM附件灑進通道

MTOM附件灑進通道

2015-09-25 75 views
2

我用Spring集成INT-WS：出站網關來讀取一個web服務的文件（PDF/...）與XOP/MTOMMTOM附件灑進通道

它的工作原理和正確地獲得從流附件，我可以將它們寫入磁盤。

但 SI spring-ws在傳輸過程中將文件保留在內存中：獲取60 MB的30個文件一起填充內存並失敗（即使在調用方法中，我一次讀取帶有bufferedStream 128kB的Stream ）。

我希望能夠傳輸N個文件與 「N * fileSize> availableMemory」，流式小文件塊。

xml（和響應對象）包含其他信息（如標題等），「文件」是來自DataHandler的InputStream。

編輯： 這不是春天集成的錯。 它也發生在一個獨立的spring-ws客戶端上。

我的豆子：

<bean id="mtomMarshaller" class="org.springframework.oxm.jaxb.Jaxb2Marshaller"> 
     <property name="contextPath" value="aaaaa.bb.filestreamer" /> 
     <property name="mtomEnabled" value="true" /> 
</bean>

我SI配置：

<int:logging-channel-adapter id="logger" level="DEBUG" log-full-message="true"/> 

<!-- All my channels are like this: --> 
<int:channel id="singleFileRespChannel"><int:interceptors><int:wire-tap channel="logger"/></int:interceptors></int:channel> 

<int:gateway id="filewsEntry" 
     service-interface="aaaaa.ws.FileServiceGateway" 
     default-reply-timeout="5000" > 
      <int:method name="getSingleFile" request-channel="singleFileReqChannel" reply-channel="singleFileRespChannel" /> 
    </int:gateway> 

<int-ws:outbound-gateway id="singleFileMarshallingGateway"  
    request-channel="singleFileReqChannel" reply-channel="singleFileRespChannel" 
    requires-reply="true" 
    uri="${filews.singleFile.wsURI}" marshaller="mtomMarshaller" 
    unmarshaller="mtomMarshaller" > 
</int-ws:outbound-gateway>

我與公理也試過：

<bean id="axiomMessageFactory" class="org.springframework.ws.soap.axiom.AxiomSoapMessageFactory"> 
     <property name="payloadCaching" value="false" /> 
     <property name="attachmentCaching" value="false" /> 
     <property name="attachmentCacheThreshold" value="1024"/> 
     <property name="attachmentCacheDir" value="D:/tmp/cache"></property> 
    </bean>

但是，除了緩存磁盤上的文件，這是相同的：JVM始終使用相同的內存（堆）數量。

我以爲是S-I。 splitter，它會返回地圖中的塊......但要做到這一點，它必須讀取所有文件。

結果可以通過彈簧積分通道「傳送」給呼叫者而不需要將其全部加載到內存中嗎？

任何想法我怎麼能做到這一點？

編輯：添加請求信息：在一個Junit的@Test我呼籲在Runnable.run（服務 ）（首發30個線程）來模擬客戶端的併發請求。使用的內存生長至最大（2GB），這意味着它越來越內存中的所有文件：

這裏調用者的代碼部分：

try{ 
     FileResponse sfresp = service.getSingleFile(sfreq); // CALL S.I. GW 

     logger.warn("SINGLEFILE STREAM: "+ sfreq.getFile().getFileId()); 

     InputStream in=new BufferedInputStream(sfresp.getFileAttachment().getFile().getInputStream()); 

     logger.warn("writing the stream"); 
     byte[] buffer = new byte[1024]; 
     int bytesRead; 
     try{ 
      bytesRead = in.read(buffer); 
      logger.warn("just read "+bytesRead+" Bytes from file : " + new String(buffer,"ASCII")); 

     }catch (IOException e) { 
      logger.error("FILE I/O ERROR"+e.getLocalizedMessage()); 
     }finally{ 
      in.close(); 
     } 
}catch....etc etc

初步結果出現了大量的時間後（30秒），因此在將流傳遞迴調用方之前，端點似乎獲取了所有文件。

編輯

我調試的同時 我不明白爲什麼它得到所有的60兆文件，而不是隻是我所要求的幾個字節，它確實是我開始之前甚至讀取流：

2015-09-28 11:20:32,725|WebServiceTemplate|Sent request [AxiomSoapMessage] 
2015-09-28 11:20:32,741|OMOutputFormat|Start getContentType: OMOutputFormat [ mimeBoundary =null rootContentId=null doOptimize=false doingSWA=false isSOAP11=true charSetEncoding=UTF-8 xmlVersion=null contentType=null ignoreXmlDeclaration=false autoCloseWriter=false actionProperty=null optimizedThreshold=0] 
2015-09-28 11:20:32,741|OMOutputFormat|getContentType= {text/xml} OMOutputFormat [ mimeBoundary =null rootContentId=null doOptimize=false doingSWA=false isSOAP11=true charSetEncoding=UTF-8 xmlVersion=null contentType=text/xml ignoreXmlDeclaration=false autoCloseWriter=false actionProperty=null optimizedThreshold=0] 
2015-09-28 11:20:32,741|MTOMXMLStreamWriter|Creating MTOMXMLStreamWriter 
2015-09-28 11:20:32,741|MTOMXMLStreamWriter|OutputStream =class org.springframework.ws.transport.AbstractSenderConnection$RequestTransportOutputStream 
2015-09-28 11:20:32,741|MTOMXMLStreamWriter|OMFormat = OMOutputFormat [ mimeBoundary =null rootContentId=null doOptimize=false doingSWA=false isSOAP11=true charSetEncoding=UTF-8 xmlVersion=null contentType=text/xml ignoreXmlDeclaration=false autoCloseWriter=false actionProperty=null optimizedThreshold=0] 
2015-09-28 11:20:32,741|MTOMXMLStreamWriter|preserveAttachments = false 
2015-09-28 11:20:32,741|StAXUtils|XMLStreamWriter is org.apache.axiom.util.stax.dialect.Woodstox4StreamWriterWrapper 
2015-09-28 11:20:32,741|OMDataSourceExtBase|serialize [email protected] 
2015-09-28 11:20:32,741|MTOMXMLStreamWriter|Returning access to the original output stream: org.springframework.ws.[email protected]50f9c7a0 
2015-09-28 11:20:32,741|MTOMXMLStreamWriter|Calling MTOMXMLStreamWriter.flush 
2015-09-28 11:20:32,788|OMDataSourceExtBase|serialize OutputStream optimisation: true 
2015-09-28 11:20:32,788|OMDataSourceExtBase|serialize output=org.springframework.ws.[email protected]50f9c7a0 format=OMOutputFormat [ mimeBoundary =null rootContentId=null doOptimize=false doingSWA=false isSOAP11=true charSetEncoding=UTF-8 xmlVersion=null contentType=null ignoreXmlDeclaration=false autoCloseWriter=false actionProperty=null optimizedThreshold=0] 
2015-09-28 11:20:32,788|ByteArrayDataSource|getXMLBytes encoding=UTF-8 
2015-09-28 11:20:32,788|SOAPEnvelopeImpl|Could not close builder or parser due to: 
2015-09-28 11:20:32,788|SOAPEnvelopeImpl|builder is null 
2015-09-28 11:20:32,788|MTOMXMLStreamWriter|Calling MTOMXMLStreamWriter.flush 
2015-09-28 11:20:32,788|MTOMXMLStreamWriter|close 
***** 
HERE IT STOPS WHILE TRANSFERRING THE DOC OVER NETWORK 
***** 
2015-09-28 11:20:38,322|MIMEMessage|Attachments contentLength=0, contentTypeString=Multipart/Related; start-info="text/xml"; type="application/xop+xml"; boundary="----=_Part_434_1482047736.1443432076967" 
2015-09-28 11:20:38,342|MIMEMessage|getRootPartContentID rootContentID=null 
2015-09-28 11:20:38,342|MIMEMessage|readHeaders 
2015-09-28 11:20:38,342|MIMEMessage|addHeader: (Content-Type) value=(application/xop+xml; charset=utf-8; type="text/xml") 
2015-09-28 11:20:38,342|PartImpl|getHeader name=(content-id) value=(null) 
2015-09-28 11:20:38,358|MIMEMessage|getRootPartContentID rootContentID=null 
2015-09-28 11:20:38,358|PartImpl|Using blob of type org.apache.axiom.blob.MemoryBlobImpl 
2015-09-28 11:20:38,358|PartImpl|getHeader name=(Content-Transfer-Encoding) value=(null) 
2015-09-28 11:20:38,358|DebugInputStream|EOF reached after reading 516 bytes in 1 chunks 
2015-09-28 11:20:38,358|MIMEMessage|getRootPartContentID rootContentID=null 
2015-09-28 11:20:38,358|PartImpl|getHeader name=(content-type) value=(application/xop+xml; charset=utf-8; type="text/xml") 
2015-09-28 11:20:38,389|DefaultOMMetaFactoryLocator|Starting class path based discovery 
2015-09-28 11:20:38,389|ImplementationFactory|Loading jar:file:/C:/Users/Max/.m2/repository/org/apache/ws/commons/axiom/axiom-impl/1.2.15/axiom-impl-1.2.15.jar!/META-INF/axiom.xml 
2015-09-28 11:20:38,389|ImplementationFactory|Discovered implementations: [llom(metaFactory=org.apache.axiom.om.impl.llom.factory.OMLinkedListMetaFactory,features=[default(priority=100)])] 
2015-09-28 11:20:38,405|PriorityBasedOMMetaFactoryLocator|Meta factories: 
    default: org.apache.axiom.om.impl.llom.factory.OMLinkedListMetaFactory 
    2015-09-28 11:20:38,405|StAXSOAPModelBuilder|Starting to process SOAP 1.1 message 
2015-09-28 11:20:38,405|WebServiceTemplate|Received response [AxiomSoapMessage] for request [AxiomSoapMessage] 
2015-09-28 11:20:38,405|PullSerializer|Pull serializer created; initial state is [email protected]98[cache=false,document=false] 
2015-09-28 11:20:38,421|StAXBuilder|Caching disabled; current element level is 3 
2015-09-28 11:20:38,421|Navigator|Switching to pull-through mode; first event is START_ELEMENT; depth is 1 
2015-09-28 11:20:38,421|PullSerializer|Switching to state org[email protected]35c7d3d3[[email protected]23dd3a] 
2015-09-28 11:20:38,421|XOPDecodingStreamReader|processXopInclude - found href : cid:e8e49803-f357-4757-bcde-6c99abb48cf4%40ws.xxxx.com 
2015-09-28 11:20:38,421|XOPDecodingStreamReader|processXopInclude - decoded contentID : [email protected] 
2015-09-28 11:20:38,421|XOPDecodingStreamReader|Encountered xop:Include for content ID '[email protected]' 
2015-09-28 11:20:38,421|MIMEMessage|readHeaders 
2015-09-28 11:20:38,421|MIMEMessage|addHeader: (Content-Type) value=(application/octet-stream) 
2015-09-28 11:20:38,421|MIMEMessage|addHeader: (Content-ID) value=(<[email protected]>) 
2015-09-28 11:20:38,421|MIMEMessage|addHeader: (Content-Transfer-Encoding) value=(binary) 
2015-09-28 11:20:38,421|PartImpl|getHeader name=(content-id) value=(<[email protected]>) 
2015-09-28 11:20:38,421|PartImpl|Using blob of type org.apache.axiom.blob.MemoryBlobImpl 
2015-09-28 11:20:38,421|PartImpl|getHeader name=(Content-Transfer-Encoding) value=(binary) 
2015-09-28 11:20:38,499|DebugInputStream|EOF reached after reading 64541873 bytes in 15938 chunks 
2015-09-28 11:20:39,499|PullSerializer|Restoring state [email protected]98[cache=false,document=false] 
2015-09-28 11:20:39,499|StAXBuilder|Caching re-enabled; new element level: 2; done=false 
2015-09-28 11:20:39,500|PullSerializer|Switching to state o[email protected]1e229be 
2015-09-28 11:20:39,501|AbstractReplyProducingMessageHandler|handler 'org.springframework.integration.ws.MarshallingWebServiceOutboundGateway#1' sending reply Message: [[email protected]][Headers={timestamp=1443432039501, id=d9ffd3d4-48f3-504e-00a1-9ccd5d1d56e8, errorChannel=org.springfra[email protected]15160f3c, replyChannel=org.springfra[email protected]15160f3c, useStub=false}] 
2015-09-28 11:20:39,501|AbstractMessageChannel$ChannelInterceptorList|preSend on channel 'singleFileRespChannel', message: [[email protected]][Headers={timestamp=1443432039501, id=d9ffd3d4-48f3-504e-00a1-9ccd5d1d56e8, errorChannel=org.springfra[email protected]15160f3c, replyChannel=org.springfra[email protected]15160f3c, useStub=false}] 
2015-09-28 11:20:39,501|AbstractMessageHandler|[email protected] received message: [[email protected]][Headers={timestamp=1443432039501, id=d9ffd3d4-48f3-504e-00a1-9ccd5d1d56e8, errorChannel=org.springfra[email protected]15160f3c, replyChannel=org.springfra[email protected]15160f3c, useStub=false}] 
2015-09-28 11:20:39,501|AbstractReplyProducingMessageHandler|handler '[email protected]' sending reply Message: [[email protected]][Headers={timestamp=1443432039501, id=d9ffd3d4-48f3-504e-00a1-9ccd5d1d56e8, errorChannel=org.springfra[email protected]15160f3c, replyChannel=org.springfra[email protected]15160f3c, useStub=false}] 
2015-09-28 11:20:39,501|AbstractMessageChannel$ChannelInterceptorList|postSend (sent=true) on channel 'singleFileRespChannel', message: [[email protected]][Headers={timestamp=1443432039501, id=d9ffd3d4-48f3-504e-00a1-9ccd5d1d56e8, errorChannel=org.springfra[email protected]15160f3c, replyChannel=org.springfra[email protected]15160f3c, useStub=false}] 
2015-09-28 11:20:39,501|AbstractMessageChannel$ChannelInterceptorList|postSend (sent=true) on channel 'singleFileReqChannel', message: [[email protected]][Headers={timestamp=1443432032632, id=0ce05572-05f6-5e5e-e819-63377ba4e0b1, errorChannel=org.springfra[email protected]15160f3c, replyChannel=org.springfra[email protected]15160f3c}] 
2015-09-28 11:20:39,501|IntegrationObjectSupport|Unable to attempt conversion of Message payload types. Component 'filewsEntry' has no explicit ConversionService reference, and there is no 'integrationConversionService' bean within the context. 
**** 
HERE THE SERVICE GATEWAY RETURNS THE CONTROL TO MY JAVA CODE 
**** 
2015-09-28 11:20:39,501|TestFileWsInvocation|SINGLEFILE STREAM: "myFile" 
2015-09-28 11:20:39,501|TestFileWsInvocation|writing the stream 
2015-09-28 11:20:39,501|TestFileWsInvocation|just read 1024 Bytes from file : "1234567.......... "

USING SAAJ是相同的：

2015-09-28 11:52:28,760|WebServiceAccessor|Opening [[email protected]] to [http://xxxx:8080/filestreamer/ws/] 
2015-09-28 11:52:28,791|AbstractHeaderMapper|headerName=[useStub] WILL NOT be mapped 
2015-09-28 11:52:28,807|WebServiceTemplate|Sent request [SaajSoapMessage {http://ws.xxxx.com/filestreamer}fileRequest] 
****** 
STOPS WHILE GETTING ALL THE FILE 
****** 
2015-09-28 11:52:34,948|WebServiceTemplate|Received response [SaajSoapMessage {http://ws.xxxx.com/filestreamer}fileResponse] for request [SaajSoapMessage {http://ws.xxxx.com/filestreamer}fileRequest] 
2015-09-28 11:52:35,371|AbstractReplyProducingMessageHandler|handler 'org.springframework.integration.ws.MarshallingWebServiceOutboundGateway#1' sending reply Message: [Payload=com.xxxx.w[email protected]][Headers={timestamp=1443433955371, id=0a41ab57-dac8-6f7e-3ad9-c71a745f5b0a, errorChannel=org.springfra[email protected]78561bc6, replyChannel=org.springfra[email protected]78561bc6, useStub=false}] 
2015-09-28 11:52:35,371|AbstractMessageChannel$ChannelInterceptorList|preSend on channel 'singleFileRespChannel', message: [[email protected]][Headers={timestamp=1443433955371, id=0a41ab57-dac8-6f7e-3ad9-c71a745f5b0a, errorChannel=org.springfra[email protected]78561bc6, replyChannel=org.springfra[email protected]78561bc6, useStub=false}] 
2015-09-28 11:52:35,371|AbstractMessageHandler|[email protected] received message: [[email protected]][Headers={timestamp=1443433955371, id=0a41ab57-dac8-6f7e-3ad9-c71a745f5b0a, errorChannel=org.springfra[email protected]78561bc6, replyChannel=org.springfra[email protected]78561bc6, useStub=false}] 
2015-09-28 11:52:35,371|AbstractReplyProducingMessageHandler|handler '[email protected]' sending reply Message: [[email protected]][Headers={timestamp=1443433955371, id=0a41ab57-dac8-6f7e-3ad9-c71a745f5b0a, errorChannel=org.springfra[email protected]78561bc6, replyChannel=org.springfra[email protected]78561bc6, useStub=false}] 
2015-09-28 11:52:35,371|AbstractMessageChannel$ChannelInterceptorList|postSend (sent=true) on channel 'singleFileRespChannel', message: [[email protected]][Headers={timestamp=1443433955371, id=0a41ab57-dac8-6f7e-3ad9-c71a745f5b0a, errorChannel=org.springfra[email protected]78561bc6, replyChannel=org.springfra[email protected]78561bc6, useStub=false}] 
2015-09-28 11:52:35,371|AbstractMessageChannel$ChannelInterceptorList|postSend (sent=true) on channel 'singleFileWSReqChannel', message: [[email protected]][Headers={timestamp=1443433948760, id=42d187e8-c0c6-4077-1000-8840e2c531ba, errorChannel=org.springfra[email protected]78561bc6, replyChannel=org.springfra[email protected]78561bc6, useStub=false}] 
2015-09-28 11:52:35,371|AbstractMessageChannel$ChannelInterceptorList|postSend (sent=true) on channel 'singleFileRouterChannel', message: [[email protected]][Headers={timestamp=1443433948760, id=42d187e8-c0c6-4077-1000-8840e2c531ba, errorChannel=org.springfra[email protected]78561bc6, replyChannel=org.springfra[email protected]78561bc6, useStub=false}] 
2015-09-28 11:52:35,371|AbstractMessageChannel$ChannelInterceptorList|postSend (sent=true) on channel 'singleFileReqChannel', message: [[email protected]][Headers={timestamp=1443433948760, id=6ebcc8b3-0db4-1e67-9619-98716ff362ff, errorChannel=org.springfra[email protected]78561bc6, replyChannel=org.springfra[email protected]78561bc6}] 
2015-09-28 11:52:35,371|IntegrationObjectSupport|Unable to attempt conversion of Message payload types. Component 'filewsEntry' has no explicit ConversionService reference, and there is no 'integrationConversionService' bean within the context. 
2015-09-28 11:52:35,371|TestFileWsInvocation|SINGLEFILE STREAM: 0 
2015-09-28 11:52:35,371|TestFileWsInvocation|writing the stream 
2015-09-28 11:52:35,371|TestFileWsInvocation|just read 1024 Bytes from file : "12345........"

來源

2015-09-25 MaxS

+0

請分享你的XML響應和JAXB表示。你的文件處理過程如何？我的意思是與'filewsEntry'網關的交互。 –

+0

'但是S.I.在傳輸時將文件保存在內存中：'不對SI側提供參數。這可能是Spring WS的責任，甚至是你使用的交通工具...... –

+0

嗨，大家好，謝謝...我將信息添加到帖子中。 – MaxS

回答

1

SAAJ的問題在於它的API不支持流式附件。在另一方面Apache的公理被設計爲支持這一點，但在Spring-WS一個設計缺陷，防止其利用Axiom的能力：

http://veithen.github.io/2015/10/05/spring-ws-mtom.html

來源

2015-10-06 18:36:01

+0

嗨安德烈亞斯，謝謝。 我進一步調試，我發現「整個文件傳輸」恰好發生在調用getWebServiceTemplate（）時。marshalSendAndReceive（req）; 它使用HttpURLConnection，該方法hasResponse（抽象類中的inh。）將全部複製到byte []中以返回true。 看來我的問題甚至在解組部分之前就踢了。我認爲這可能是http-chunking相關的，但我在http raw req中看到chunked-encoding標頭。 從你的文章我假設，即使我會克服這個問題，我會發現其他麻煩後來在您的博客中陳述.. – MaxS

+0

實際上有兩個額外的問題發生在客戶端只。我更新了我的博客帖子。 –

+0

謝謝，我會接受你的回答，因爲你的博客文章是完整和準確的。我希望他們能夠解決這些問題。 爲了不引入外部依賴性（只是爲了防止博客文章被刪除或移動到其他位置），最好在答案中提供答案（或回顧一下） 謝謝。 – MaxS

相關問題

 相關問題