Watched folders in a cluster

In a cluster, Watched Folder depends on the Quartz scheduler and the Job Manager service for load balancing and failover. For more information about Quartz cluster behavior, see Quartz Documentation.

Watched Folder performs these three main tasks at each poll:

  • Scan the folder

  • Invoke the target service

  • Handle the results

The load balancing and failover behavior changes depending on whether the watched folder is configured for synchronous or asynchronous invocation.

Synchronous watched folder in a cluster

For synchronous invocations, the Quartz load balancer decides which node will get the polling event. The node that gets the polling event will perform all the tasks: scan the folder, invoke the target service, and handle the results.

View full size graphic

For synchronous invocations, when one node fails, the Quartz scheduler sends new polling events to other nodes. Invocations that were started on the failed node will be lost. For more information about how to recover the files associated with the failed job, see Failure points and recovery.

Asynchronous watched folder in a cluster

For asynchronous invocations, the Quartz load balancer decides which node will get the polling event. The node that gets the polling event will scan the input folder and invoke the target service by placing the request in the Job Manager service queue. The Job Manager service load balancer, in turn, is responsible for deciding which node will process the invocation request. It is possible that even though node A created the invocation request, node B ends up processing the request. Or the node that started the invocation request may also end up processing the request.

View full size graphic

For asynchronous invocations, when one node fails, the Quartz scheduler sends new polling events to other nodes. Invocation requests that were created on the failed node will be in the Job Manager service queue and will be sent to other nodes for processing. Files for which invocation requests are not created will remain in the stage folder. For more information about how to recover the files associated with the failed job, see Failure points and recovery.

// Ethnio survey code removed