web stats
Queueing due to reprocessing large number of messages - Mirth Community

Go Back   Mirth Community > Mirth Connect > General Discussion

Reply
 
Thread Tools Display Modes
  #1  
Old 02-19-2020, 11:12 AM
khobbs khobbs is offline
Mirth Newb
 
Join Date: May 2016
Posts: 14
khobbs is on a distinguished road
Default Queueing due to reprocessing large number of messages

We have the situation where we are reprocessing 2.6 million messages in a channel. The channel has source queue ON. The request to reprocess that many messages take quite a while to retrieve them from the db and put them back into the channel. The source queue is ON and everything works as expected.

In the meantime, this channel is also receiving live incoming new data from an external source, as normal real-time message processing.

My question is, for these live feed incoming messages, when they hit the channel that is in the process of queueing the messages for the reprocessing, what happens to these new live feed incoming messages? When then come into this channel that is busy queueing the reprocess, do these messages get intermixed into the queue, or do they wait somewhere for the channel to finish queueing the reprocessed messages?

I'm concerned about message order, and that maybe we need to be pausing the live stream until the reprocessing messages all get queued?

Thanks for you time!
-kevin
Reply With Quote
  #2  
Old 03-03-2020, 05:10 AM
kirbykn2's Avatar
kirbykn2 kirbykn2 is offline
Mirth Guru
 
Join Date: Sep 2014
Location: Michigan
Posts: 615
kirbykn2 is on a distinguished road
Default

Do you see a backload of messages in the source queue?
__________________
Best,

Kirby

Mirth Certified|Epic Bridges Certified|Cloverleaf Level 2 Certified

Appliance Version 3.11.4
Mirth Connect Version 3.8.0
Java Version 1.6.0_45-b06
Java (64 bit) Version 1.6.0_45-b06
Java 7 (64 bit) Version 1.7.0_151-b15
Java 8 (64 bit) Version 1.8.0_181-b13
PostgreSQL Version 9.6.8
Reply With Quote
  #3  
Old 03-09-2020, 01:09 PM
khobbs khobbs is offline
Mirth Newb
 
Join Date: May 2016
Posts: 14
khobbs is on a distinguished road
Default

Yes, the source queue is queueing as expected, there are many messages (several days of processing worth) actively being added into the source queue. In the meantime we have not turned off our live feed. So, my question is where do the new incoming live feed messages go? At the end of the queue, after all the reprocessing messages finally get queued? Or do the new incoming live feed messages get inserted into the queue as they come in, which would be before all the reprocessing messages get fully queued.

Last edited by khobbs; 03-09-2020 at 01:11 PM.
Reply With Quote
  #4  
Old 03-09-2020, 04:04 PM
agermano agermano is offline
Mirth Guru
 
Join Date: Apr 2017
Location: Indiana, USA
Posts: 1,181
agermano is on a distinguished road
Default

If you are reprocessing as new messages, I'm almost certain that the live messages will be interspersed with reprocessed messages as they are being created.

I'm not sure how it will affect processing order if you are replacing the old messages.
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT -8. The time now is 02:58 AM.


Powered by vBulletin® Version 3.8.7
Copyright ©2000 - 2020, vBulletin Solutions, Inc.
Mirth Corporation