web stats
How to queue/wait multiple destinations - Mirth Community

Go Back   Mirth Community > Mirth Connect > Support

Reply
 
Thread Tools Display Modes
  #1  
Old 07-18-2013, 02:44 AM
chronofish chronofish is offline
Mirth Newb
 
Join Date: Mar 2008
Posts: 25
chronofish
Question How to queue/wait multiple destinations

We have a Mirth channel (synchronized) going to 4 destinations.

The first 3 destinations are web service calls (HTTP Sender) which are simply REST calls to get some basic data.

The 4th channel relies on the previous 3 destinations to have been completed cleanly. The data from those previous destinations are used in the 4th destination's transformer and the outgoing message is modified accordingly.

When the channel was initially released, the destinations were not set to "use persistent queues". Occasionally one of the web service calls would timeout. When this happens the content of the response (paraphrased) was "Error:HTTP Timeout....".

The channel was updated to use persistent queues. Now the response is "Message is queued".

In either case the 4th destination is called despite the result of previous destinations. And the response data is the Mirth error/warning message instead of data from the HTTP call.

What we need is for Mirth to queue on a destination and not continue until the destination has been successful.

Is this possible?

Thank you
Reply With Quote
  #2  
Old 07-18-2013, 08:58 AM
narupley's Avatar
narupley narupley is online now
Mirth Employee
 
Join Date: Oct 2010
Posts: 7,126
narupley is on a distinguished road
Default

Even in 3.0 this specific use-case isn't super-simple to do out of the box. But it's certainly doable, and much easier to implement in 3.0 then it would be in 2.x (because all connectors can queue, and there is a response transformer now). Here's an example.

Say you have a channel with three HTTP Senders and one JavaScript Writer. In 3.0 you can continually queue that JavaScript Writer until some condition is met. So one way to do it is to keep an in-memory cache of the response map values of the previous three destinations, in the global channel map. To ensure that they get updated when the message is eventually sent, we'll create a response transformer, which will run after every send attempt.

First, we set the queue for all four destinations to "Attempt First", indicating that if the queue is empty, the channel will first try to send the message, instead of always just dropping it in the queue and immediately moving on. In the response transformer for each of the first three destinations, we have the following:

Destinations 1-3 Response Transformer
Code:
var newResponse = new com.mirth.connect.donkey.model.message.Response(responseStatus, msg, responseStatusMessage, responseErrorMessage);
$gc('cachedResponseMap').get(connectorMessage.getMessageId()).put('d'+connectorMessage.getMetaDataId(),newResponse);
That will get the cached response map for the particular message we're on, and update the response. Once we've done that for all three destinations, we can validate the cached responses in the final JavaScript Writer:

Destination 4 JavaScript Writer
Code:
var cachedResponseMap = $gc('cachedResponseMap').get(connectorMessage.getMessageId());
var d1 = cachedResponseMap.get('d1');
var d2 = cachedResponseMap.get('d2');
var d3 = cachedResponseMap.get('d3');
if (d1.getStatus() == QUEUED || d2.getStatus() == QUEUED || d3.getStatus() == QUEUED)
	return QUEUED;

// If it gets here, all three previous destinations are no longer queued
logger.info('Responses:\n\n'+d1.getStatus()+': '+d1.getMessage()+'\n\n'+d2.getStatus()+': '+d2.getMessage()+'\n\n'+d3.getStatus()+': '+d3.getMessage());

$gc('cachedResponseMap').remove(connectorMessage.getMessageId());
That grabs the status for each destination. If at least one is QUEUED, then we still want to queue this destination as well, so we just return QUEUED. Otherwise, all three of the previous destinations are no longer queued, so at that point you can do whatever you want.

Now, since that is an in-memory cache, it will be lost if the server goes down for some reason (or if the channel is redeployed and "Clear global channel map on deploy" is checked). To remedy this, we'll create a deploy script that reconstructs the cached response map based on the queued messages currently in the database.

Channel Deploy Script
Code:
var cachedResponseMap = new java.util.HashMap();

var cc = com.mirth.connect.server.controllers.ControllerFactory.getFactory().createChannelController();
var metaDataIds = cc.getConnectorNames(channelId).keySet();
metaDataIds.remove(new java.lang.Integer(0));
metaDataIds.remove(new java.lang.Integer(4));

var dao = com.mirth.connect.donkey.server.Donkey.getInstance().getDaoFactory().getDao();
try {
	var queuedConnectorMessages = dao.getConnectorMessages(channelId,4,com.mirth.connect.donkey.model.message.Status.QUEUED);

	for each (queuedConnectorMessage in queuedConnectorMessages.toArray()) {
		var messageId = queuedConnectorMessage.getMessageId();
		cachedResponseMap.put(messageId, new java.util.HashMap());
		for each (connectorMessage in dao.getConnectorMessages(channelId,messageId,metaDataIds,true).toArray())
			cachedResponseMap.get(messageId).putAll(connectorMessage.getResponseMap());
	}
} finally {
	dao.close();
}

$gc('cachedResponseMap',cachedResponseMap);
First, we grab all QUEUED connector messages for the fourth (JavaScript Writer) destination. For each queued connector message it finds, we'll then grab the other three connector messages and place the current contents of their response maps into our in-memory cached response map. Finally, we'll place the constructed response map into the global channel map.

To ensure that an entry exists in the cached response map for each message, we'll add it in the preprocessor:

Channel Preprocessor Script
Code:
$gc('cachedResponseMap').put(connectorMessage.getMessageId(),new java.util.HashMap());
return message;
I'll attach the channels I used. These are for 3.0.0 RC 1.
Attached Files
File Type: xml Example - HTTP Queues.xml (42.3 KB, 200 views)
File Type: xml HTTP Listener 1.xml (15.8 KB, 95 views)
File Type: xml HTTP Listener 2.xml (15.8 KB, 62 views)
File Type: xml HTTP Listener 3.xml (15.8 KB, 60 views)
__________________
Step 1: JAVA CACHE...DID YOU CLEAR ...wait, ding dong the witch is dead?

Nicholas Rupley
Work: 949-237-6069
Always include what Mirth Connect version you're working with. Also include (if applicable) the code you're using and full stacktraces for errors (use CODE tags). Posting your entire channel is helpful as well; make sure to scrub any PHI/passwords first.


- How do I foo?
- You just bar.

Last edited by narupley; 07-18-2013 at 07:35 PM.
Reply With Quote
  #3  
Old 07-21-2013, 06:21 PM
chronofish chronofish is offline
Mirth Newb
 
Join Date: Mar 2008
Posts: 25
chronofish
Default

Thanks for the detailed response.

That seems like a lot of hoops. Maybe easier to cascade Channels. A bit more of a maintenance issue (having to know that two/three channels are operating in unison) but better than being hidden in code.

-CF
Reply With Quote
  #4  
Old 07-22-2013, 05:50 AM
narupley's Avatar
narupley narupley is online now
Mirth Employee
 
Join Date: Oct 2010
Posts: 7,126
narupley is on a distinguished road
Default

Quote:
Originally Posted by chronofish View Post
Thanks for the detailed response.

That seems like a lot of hoops. Maybe easier to cascade Channels. A bit more of a maintenance issue (having to know that two/three channels are operating in unison) but better than being hidden in code.

-CF
It depends on what you mean by "cascade". Remember that after a message gets queued, the channel considers it "done" and moves on to the next destination. Asynchronously, the destination queue will dispatch the queued message eventually. In 3.0 you have the response transformer which can execute after each send attempt, so you have the ability to forward information when the message is finally successfully sent. In 2.x you do not have that capability; you'd have to go directly to the database instead (which could get even more ugly).

So yes in 3.0 you could break it up into two channels, but the code would only be slightly less complex. You would still need to take advantage of the response transformers of each HTTP Sender and forward some information to the downstream channel. Then the downstream channel would have to use the global channel map to determine when it's received "ACKs" from each of the three destinations, and then act accordingly.
__________________
Step 1: JAVA CACHE...DID YOU CLEAR ...wait, ding dong the witch is dead?

Nicholas Rupley
Work: 949-237-6069
Always include what Mirth Connect version you're working with. Also include (if applicable) the code you're using and full stacktraces for errors (use CODE tags). Posting your entire channel is helpful as well; make sure to scrub any PHI/passwords first.


- How do I foo?
- You just bar.
Reply With Quote
  #5  
Old 12-21-2017, 09:32 AM
rallbritain rallbritain is offline
Mirth Newb
 
Join Date: Jun 2013
Posts: 6
rallbritain is on a distinguished road
Default

Reviving a really old thread here, but wanted to ask a follow up question.

Can you force a non-javascript writer destination to queue?

Last edited by rallbritain; 12-21-2017 at 09:40 AM.
Reply With Quote
Reply

Tags
destinations, queues

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT -8. The time now is 04:49 PM.


Powered by vBulletin® Version 3.8.7
Copyright ©2000 - 2020, vBulletin Solutions, Inc.
Mirth Corporation