Tuesday, October 14, 2014

DIXF timeout issues

DIXF - Interesting find

I spent the past 2 weeks doing data migration and here are some interesting errors I found and how to fix them:

Several issues can be seen with large amounts of records (more than 200k - 300k) and medium amounts of records (around 200k) with a lot of data.
Here is the error:

System.ServiceModel.CommunicationException: An error occurred while receiving the HTTP response to http://myServer:7000/DMFService/DMFServiceHelper.svc. This could be due to the service endpoint binding not using the HTTP protocol. This could also be due to an HTTP request context being aborted by the server (possibly due to the service shutting down). See server logs for more details. ---> System.Net.WebException: The underlying connection was closed: An unexpected error occurred on a receive. ---> System.IO.IOException: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host. ---> System.Net.Sockets.SocketException: An existing connection was forcibly closed by the remote host 
at System.Net.Sockets.NetworkStream.Read(Byte[] buffer, Int32 offset, Int32 size) 
--- End of inner exception stack trace --- 
at System.Net.Sockets.NetworkStream.Read(Byte[] buffer, Int32 offset, Int32 size) 
at System.Net.PooledStream.Read(Byte[] buffer, Int32 offset, Int32 size) 
at System.Net.Connection.SyncRead(HttpWebRequest request, Boolean userRetrievedStream, Boolean probeRead) 
--- End of inner exception stack trace --- 
at System.Net.HttpWebRequest.GetResponse() 
at System.ServiceModel.Channels.HttpChannelFactory`1.HttpRequestChannel.HttpChannelRequest.WaitForReply(TimeSpan timeout) 
--- End of inner exception stack trace --- 

Server stack trace: 
at System.ServiceModel.Channels.HttpChannelUtilities.ProcessGetResponseWebException(WebException webException, HttpWebRequest request, HttpAbortReason abortReason) 
at System.ServiceModel.Channels.HttpChannelFactory`1.HttpRequestChannel.HttpChannelRequest.WaitForReply(TimeSpan timeout) 
at System.ServiceModel.Channels.RequestChannel.Request(Message message, TimeSpan timeout) 
at System.ServiceModel.Channels.ClientReliableChannelBinder`1.RequestClientReliableChannelBinder`1.OnRequest(TRequestChannel channel, Message message, TimeSpan timeout, MaskingMode maskingMode) 
at System.ServiceModel.Channels.ClientReliableChannelBinder`1.Request(Message message, TimeSpan timeout, MaskingMode maskingMode) 
at System.ServiceModel.Security.SecuritySessionClientSettings`1.SecurityRequestSessionChannel.Request(Message message, TimeSpan timeout) 
at System.ServiceModel.Channels.ServiceChannel.Call(String action, Boolean oneway, ProxyOperationRuntime operation, Object[] ins, Object[] outs, TimeSpan timeout) 
at System.ServiceModel.Channels.ServiceChannelProxy.InvokeService(IMethodCallMessage methodCall, ProxyOperationRuntime operation) 
at System.ServiceModel.Channels.ServiceChannelProxy.Invoke(IMessage message) 

Exception rethrown at [0]: 
at System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage reqMsg, IMessage retMsg) 
at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData& msgData, Int32 type) 
at Microsoft.Dynamics.AX.Framework.Tools.DMF.ServiceProxy.DMFServiceReference.ServiceContract.ShowPreview(DMFEntity entity) 
at Microsoft.Dynamics.AX.Framework.Tools.DMF.ServiceProxy.DmfEntityProxy.DoWork[T](Func`1 work)

We solve this by adding 'recieveTimeout' and 'sendTimeout' into the C:\Program Files\Microsoft Dynamics AX\60\DataImportExportFramework/Microsoft.Dynamics.AX.Framework.Tools.DMF.SSISHelperService.exe.config

        <binding name="DMFService_WsHttpBinding" receiveTimeout="01:00:00"
          maxBufferPoolSize="2147483647" maxReceivedMessageSize="2147483647">
          <readerQuotas maxDepth="2147483647" maxStringContentLength="2147483647"
            maxArrayLength="2147483647" maxBytesPerRead="2147483647"             maxNameTableCharCount="2147483647" />


  1. Thanks for publishing the tip, Brad. We received this message attempting to insert only 8000 customers into the staging table. The source SQL view was slow, however, with many scalar functions used.

  2. I set send time out and receive timeout to 01:30:00, but it still gave the same error. Can you tell me how much values should be given roughly for 50000 records?

    1. Depends on what type of records. The values I set in the post were for around 300k records and it worked ok. I would push your time up even longer (Maybe 3 hours).

      You could also cut your file in half or thirds or fourths and do some testing. See how long each file goes.

    2. Thanks for the response!
      Without this setting 30K records were added successfully but error came on after that.
      Hopefully it will get fixed.