web stats
Mirth Community - View Single Post - Recurring Connection Resets on HTTP Sender to specific backend.
View Single Post
  #1  
Old 09-25-2019, 02:31 PM
thaddeus_k thaddeus_k is offline
What's HL7?
 
Join Date: Sep 2019
Location: Dallas, TX
Posts: 4
thaddeus_k is on a distinguished road
Exclamation Recurring Connection Resets on HTTP Sender to specific backend.

Mirth Connect Version 3.8.0 Open Source

Issue Description:
We have a channel with an HTTP Sender in it that sends a JSON request to an external backend. When the HTTP Sender is run shortly after a previous call, (~5 minutes) everything works fine. However, if there has been a notable delay since the previous call, we instead get either a Connection Reset error, or we get a Read Timed Out error. ( The read timeout is due to an intentionally short timeout value intended to mitigate the connection reset issues).

At the moment we have mitigated this problem by implementing retry logic, but in the case we have to retry, total latency is in the realm of 8000 milliseconds. This channel services an IVR server, so expected response time is at the scale of a phone conversation with a human being on the other end of the line. An 8 second delay greatly aggravates callers.

Channel has been attached. It has two destinations, selected by the URL used by the originating requestor. (In our case, an IVR server)

A typical connection reset error as seen in mirth:
Code:
HTTP Sender error
ERROR MESSAGE: Error connecting to HTTP server
java.net.SocketException: Connection reset
<Stack trace snipped>
And a screenshot of this is attached. Notice how just ten minutes previously, a successful request with the same body went through.

Also attached are wireshark captures. Notes on those transactions follow:
Quote:
The first transaction at 8:50 AM to sales force from postman responded in 1.8 seconds and the one at 10 AM responded in 1.1 seconds. Subsequent requests on both occasions responded in less than 1 second.

The first transaction that we ran after we immediately ran the sales force transaction via mirth took us 8.6 seconds at around 8:55 am and the 2nd transaction that we ran via mirth at 10 am took 14 seconds from postman. Subsequent requests responded in less than 1 second.
Edit: the Mirth Channel is attempting to reach an HTTPS Backend , so port will be 443 if you need to filter the wireshark packets. The servers own IP address is 10.9.1.116/23, according to the Network Properties Screen.

According to nslookup from the server, target IP addresses (non-authoritative) are
Code:
nslookup <snipped>
Address:  10.9.1.230

Non-authoritative answer:
Name:    <snipped>
Addresses:  136.147.43.149
          136.147.41.149
          136.147.42.149
tl;dr This channel is giving us recurring connection resets when other channels are not, and I do not understand why it is happening or how to get it to stop.
Attached Images
File Type: png Connection_reset_error.png (38.7 KB, 5 views)

Last edited by thaddeus_k; 09-27-2019 at 08:06 AM. Reason: Add Backend URL
Reply With Quote