XMLWordPrintable

    • Icon: Story Story
    • Resolution: Done
    • Icon: High High
    • Istanbul Release
    • None
    • None
    • None

      Under current design using internal queue and multi-thread to process from it - there is possibility for events received to be delivered out of order by the collector when events from arrive in quick succession.
       
              1. Let’s assume that events land in the queue in proper order:  E1, E2, E3, etc.
              2. Now we have multiple threads (e.g. 20), T1, T2, T3, etc., each pulling the next message from the queue, doing some processing, and ultimately sending the message to the message bus.
              3. Thread T1 takes event E1 from the queue.  Thread T2 takes E2 from the queue.  These threads are now operating in parallel.
              4. We have no guarantee that T1 will send to MR before T2 does.  T1 could be interrupted by the CPU while T2 proceeds. 

       

      Possible design change to address ordering:

      Syncronous event receipt and publish - validate each message against the schema and publish messages to MR  in the main request handling call without event queue/caching within VES. The response code will be sent to client based on Dmaap publish response. If Dmaap is unavailable, respond 5XX back to client (they should queue and retry based on VES requirment) 

      Pros: Collector can return a exact status code to the client  —  will not send 202 if MR is not available.  Collector can be scaled up performance via server thread count / non-blocking I/O architecture in the server.  No possible event loss scenario in VESCollector

      Cons:  Increase delay in response to VNF (based on MR publish); more asynchronous Dmaap publish call even when messages are destined for same topics.

       

            pawel pawel
            vv770d vv770d
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: