Watched folder endpoint settings

Use the following settings to configure a watched folder endpoint.

Name:
(Mandatory) Identifies the endpoint. Do not include a < character because it will truncate the name displayed in Workspace. If you’re entering a URL as the name of the endpoint, ensure that it conforms with the syntax rules specified in RFC1738.

Description:
A description of the endpoint. Do not include a < character because it will truncate the description displayed in Workspace.

Path:
(Mandatory) Specifies the watched folder location. In a clustered environment, this setting must point to a shared network folder that is accessible from every computer in the cluster.

Asynchronous:
Identifies the invocation type as asynchronous or synchronous. The default value is asynchronous. Asynchronous is recommended for long-lived processes, while synchronous is recommended for transient or short-lived processes.

Cron Expression:
Enter a cron expression if the watched folder must be scheduled by using a cron expression. When this setting is configured, Repeat Interval is ignored. For details about configuring the cron expression see Class CronTrigger. When this setting is configured, the Repeat Interval is ignored.

Repeat Interval:
The interval in seconds for scanning the watched folder for input. Unless the Throttle setting is enabled, Repeat Interval should be longer than the time to process an average job; otherwise, the system may become overloaded. The default value is 5. See the description for Batch Size for additional information.

Repeat Count:
Number of times the watched folder scans the folder or directory. A value of -1 indicates indefinite scanning. The default value is -1.

Throttle:
When this option is selected, it limits the number of watched folder jobs that LiveCycle processes at any given time. The maximum number of jobs is determined by the Batch Size value. (See About throttling.)

User Name:
(Mandatory) The user name that is used when invoking a target service from the watched folder. The default value is SuperAdmin.

Domain Name:
(Mandatory) The user’s domain. The default value is DefaultDom.

Batch Size:
The number of files or folders to be picked up per scan. Use to prevent an overload on the system; scanning too many files at one time can cause a crash. The default value is 2.

The Repeat Interval and Batch Size settings determine how many files Watched Folder picks up in every scan. Watched Folder uses a Quartz thread pool to scan the input folder. The thread pool is shared with other services. If the scan interval is small, the threads will scan the input folder often. If files are dropped frequently into the watched folder, then you should keep the scan interval small. If files are dropped infrequently, use a larger scan interval so that the other services can use the threads.

If there is a large volume of files being dropped, make the batch size large. For example, if the service invoked by the watched folder endpoint can process 700 files per minute, and users drop files into the input folder at the same rate, then setting the Batch Size to 350 and the Repeat Interval to 30 seconds will help Watched Folder performance without incurring the cost of scanning the watched folder too often.

When files are dropped into the watched folder, it lists the files in the input, which can reduce performance if scanning is happening every second. Increasing the scan interval can improve performance. If the volume of files being dropped is small, adjust the Batch Size and Repeat Interval accordingly. For example, if 10 files are dropped every second, try setting the Repeat Interval to 1 second and the Batch Size to 10.

Wait Time:
The time, in milliseconds, to wait before you scan a folder or file after it is created. For example, if the wait time is 3,600,000 milliseconds (one hour) and the file was created one minute ago, this file will be picked up after 59 or more minutes have passed. The default value is 0.

This setting is useful to ensure that a file or folder is completely copied to the input folder. For example, if you have a large file to process and the file takes ten minutes to download, set the wait time to 10*60 *1000 milliseconds. This prevents the watched folder from scanning the file if it is not ten minutes old.

Exclude File Pattern:
A semi-colon (;) delimited list of patterns that a watched folder uses to determine which files and folders to scan and pick up. Any file or folder with this pattern will not be scanned for processing.

This setting is useful when the input is a folder with multiple files. The contents of the folder can be copied into a folder with a name that will be picked up by the watched folder. This prevents the watched folder from picking up a folder for processing before the folder is completely copied into the input folder.

You can use file patterns to exclude:

  • Files with specific filename extensions; for example, *.dat, *.xml, .pdf, *.*

  • Files with specific names; for example, data.* would exclude files and folders named data1, data2, and so on.

  • Files with composite expressions in the name and extension, as in these examples:

    • Data[0-9][0-9][0-9].[dD][aA][tT]

    • *.[dD][Aa][Tt]

    • *.[Xx][Mm][Ll]

For more information about file patterns, see About file patterns.

Include File Pattern:
(Mandatory) A semi-colon (;) delimited list of patterns that the watched folder uses to determine which folders and files to scan and pick up. For example, if the Include File Pattern is input*, all files and folders that match input* are picked up. This includes files and folders named input1, input2, and so on.

The default value is * and indicates all files and folders.

You can use file patterns to include:

  • Files with specific filename extensions; for example, *.dat, *.xml, .pdf, *.*

  • Files with specific names; for example, data.* would include files and folders named data1, data2, and so on.

  • Files with composite expressions in the name and extension, as in these examples:

    • Data[0-9][0-9][0-9].[dD][aA][tT]

    • *.[dD][Aa][Tt]

    • *.[Xx][Mm][Ll]

For more information about file patterns, see About file patterns.

Result Folder:
The folder where the saved results are stored. If the results do not appear in this folder, check the failure folder. Read-only files are not processed and will be saved in the failure folder. This value can be an absolute or relative path with the following file patterns:
  • %F = filename prefix

  • %E = filename extension

  • %Y = year (full)

  • %y = year (last two digits)

  • %M = month

  • %D = day of month

  • %d = day of year

  • %H = hour (24-hour clock)

  • %h = hour (12-hour clock)

  • %m = minute

  • %s = second

  • %l = millisecond

  • %R = random number (between 0 and 9)

  • %P = process or job id

For example, if it is 8 PM on July 17, 2009 and you specify C:/Test/WF0/failure/%Y/%M/%D/%H/, the result folder is C:/Test/WF0/failure/2009/07/17/20.

If the path is not absolute but relative, the folder will be created inside the watched folder. The default value is result/%Y/%M/%D/, which is the Result folder inside the watched folder. For more information about file patterns, see About file patterns.

Note: The smaller the size of the result folders, the better Watched Folder performance will be. For example, if the estimated load for the watched folder is 1000 files every hour, try a pattern like result/%Y%M%D%H so that a new subfolder is created every hour. If the load is smaller (for example, 1000 files per day), you could use a pattern like result/%Y%M%D.

Preserve Folder:
The location where files are stored after successful scanning and pick-up. The path can be an absolute, a relative, or a null directory path. You can use file patterns, as described for Result Folder. The default value is preserve/%Y/%M/%D/.

Failure Folder:
The folder where failure files are saved. This location is always relative to the watched folder. You can use file patterns, as described for Result Folder.

Read-only files are not processed and will be saved in the failure folder.

The default value is failure/%Y/%M/%D/.

Preserve On Failure:
Preserve input files in case of failure to execute the operation on a service. The default value is true.

Overwrite Duplicate Filenames:
When set to True, files in the results folder and preserve folder are overwritten. When set to False, files and folders with a numeric index suffix are used for the name. The default value is False.

Purge Duration:
(Mandatory) Files and folders in the result folder are purged when they are older than this value. This value is measured in days. This setting is useful in ensuring that the result folder does not become full.

A value of -1 days indicates to never delete the results folder. The default value is -1.

Operation Name:
(Mandatory) A list of operations that can be assigned to the watched folder endpoint.

Input Parameter Mappings:
Used to configure the input required to process the service and operation. The settings available depend on which service is using the watched folder endpoint. Here are the two types of inputs:
Literal:
The watched folder uses the value entered in the field as it is displayed. All basic Java types are supported. For example, if an API uses input such as String, long, int, and Boolean, the string is converted to the proper type and the service is invoked.

Variable:
The value entered is a file pattern that the watched folder uses to pick the input. For example, in the case of the encrypt password service, where the input document must be a PDF file, the user can use *.pdf as the file pattern. The watched folder will pick up all files in the watched folder that match this pattern and invoke the service for each file. When a variable is used, all input files are converted to documents. Only APIs that use Document as the input type are supported.

Output Parameter Mappings:
Used to configure the outputs of the service and operation. The settings available depend on which service is using the watched folder endpoint.

Watched Folder output can be a single document, a list of documents, or a map of documents. These output documents are then saved in the result folder, using the pattern specified in the Output Parameter Mapping.

Note: Specifying names that result in unique output filenames improves performance. For example, consider the case where the service returns one output document and the Output Parameter Mapping maps it to %F.%E (the file name and extension of the input file). In this case, if users drop files with the same name every minute, and the result folder is configured to result/%Y/%M/%D, and the Overwrite Duplicate Filename setting is off, Watched Folder will try to resolve the duplicate file names. The process of resolving duplicate file names can affect performance. In this situation, changing the Output Parameter Mapping to %F_%h_%m_%s_%l to add hours, minutes, seconds, and milliseconds to the name, or ensuring that dropped files have unique names may improve performance.