AEM forms Backup and Recovery

Backup and recovery strategy for AEM forms

If your AEM forms implementation stores additional custom data in a different database, you are responsible for implementing a strategy to back up this data and ensuring that it remains in sync with the AEM forms data. Also, the application must be designed so that it is robust enough to handle a scenario where the additional databases get out of sync. It is highly recommended that any database operation that is performed is done in the context of a transaction to help maintain a consistent state.

After you identify how AEM forms is used, determine which files must be backed up, how often, and the backup window to be made available.

Important: As with any other aspect of your AEM forms implementation, your backup and recovery strategy must be developed and tested in a development or staging environment before being used in production in order to ensure that the entire solution is working as expected with no data loss.

Adobe Experience Manager (AEM) is an integral part of AEM forms. Therefore, you need to back up AEM as well in sync with AEM forms backup as Correspondence Management Solution and services, such as forms manager are based on data stored in AEM part of AEM forms.To prevent any data loss, the AEM forms specific data must be backed up in a way to ensure that GDS and AEM (repository) correlate with database references.The database, GDS, AEM, and Content Storage Root directories must be restored to a computer with the same DNS name as the original.

Types of backups

The AEM forms backup strategy involves two types of backups:

System image:
A complete system backup that you can use to restore the contents of your computer if your hard drive or entire computer stops working. A system image backup is necessary only prior to production deployment of AEM forms. Internal corporate policies then dictate how often system image backups are required.

AEM forms specific data:
Application data exists in database, Global Document Storage (GDS), and AEM repository, and must be backed up in real time. GDS is a directory that is used to store long-lived files that are used within a process. These files may include PDFs, policies, or form templates.
Note: If Content Services (Deprecated) is installed, also back up the Content Storage Root directory. (See Content Storage Root directory (Content Services only) .)

The database is used to store form artifacts, service configurations, process state, and database references to GDS files. If you enabled document storage in the database, persistent data and documents in the GDS are also stored in the database. The database can be backed up and recovered by using the following methods:

  • Snapshot backup mode indicates that the AEM forms system is in backup mode either indefinitely or for a specified number of minutes, after which backup mode is no longer enabled. To enter or leave snapshot backup mode, you can use one of the following options. After a recovery scenario, snapshot backup mode should not be enabled.

  • Rolling backup mode indicates that the system is always in backup mode, with a new backup mode session being initiated as soon as the previous session is released. No time out is associated with rolling backup mode. When the LCBackupMode script or APIs are called to leave rolling backup mode, a new rolling backup mode session begins. This mode is useful for supporting continuous backups but still allowing old and unneeded documents to be cleaned out of the GDS directory. Rolling backup mode is not supported through the Backup and Recovery page. After a recovery scenario, rolling backup mode is still enabled. You can leave the continuous backup mode (rolling backup mode) by using the LCBackupMode script with the leaveContinuousCoverage option.

Note: Leaving rolling backup mode immediately causes a new backup mode session to begin. To disable rolling backup mode completely, use the leaveContinuousCoverage option in the script, which overwrites the existing rolling backup session. When in snapshot backup mode, you may leave backup mode as you usually do.

To prevent data loss, the AEM forms specific data must be backed up in a way that ensures that GDS and Content Storage Root directory documents correlate with database references.

Important: When the GDS is stored on the file system and not in the database, perform the database backup before the GDS backup.

Special considerations for backup and recovery

Use the following guidelines if you must recover AEM forms into a different environment because of the following changes:

  • Change in the IP address, hostname, or port of the AEM forms server

  • Change in the drive letters or directory path

  • Change to a different database host, port, or name

Typically, such recovery scenarios are caused by hardware failure of the server that hosts the application server, database server, or forms server. In addition to the AEM forms-specific configurations that are described in this section, you should also make the necessary changes for other parts of the AEM forms deployment such as load balancers and firewalls, if the hostname or IP address of a AEM forms server changes.

What cannot be changed

Even though you can change the database server and many other parameters, you cannot change the application server type or database type when you recover AEM forms from a backup. For example, if you are recovering a AEM forms backup, you cannot change the application server from JBoss to WebLogic or database from Oracle to DB2. In addition, recovered AEM forms must use the same file system paths such as the fonts directory.

Restarting after a recovery

Before you restart the forms server after a recovery, do the following:

  1. Start the system in maintenance mode.

  2. Do the following to ensure that Form Manager is synced with AEM forms in the maintenance mode:

    1. Go to http://< server >:< port >/lc/fm and log in using adminstrator/password credentials.

    2. Click the name of the user (Super Administrator in this case) at top-right corner.

    3. Click Admin Options .

    4. Click Start to synchronize assets from the repository.

  3. In a clustered environment, the master node (with respect to AEM) should be up before the slave nodes.

  4. Ensure that no processes are initiated from either internal or external sources such as the Web, SOAP, or EJB process initiators until the normal operation of the system is validated.

If the main AEM forms database is moved or changed, review the install Guides relevant to your application server for information on updating the database connection information for the AEM forms data sources IDP_DS and EDC_DS.

Changing the AEM forms hostname or IP address

In a cluster, if you use TCP caching instead of UDP, you must update the cache locator configuration. See “Configuring the caching locators (caching using TCP only)” in the configuration guide relevant to your application server.

Changing the AEM forms node file system paths

If you change the file system paths for a standalone node, you must update the appropriate references in preferences, other system configurations, custom applications, and deployed AEM forms applications. On the other hand, for a cluster, all nodes must use the same file system path configuration. You must set the Global Document Storage (GDS) root directory and ensure that it points to a copy of the recovered GDS which is in sync with the recovered database. Setting the GDS path is important because the GDS can contain data intended to persist across application server restarts.

In a clustered environment, the file system path configuration of the repository should be same for all the cluster nodes before the backup and after the recovery.

Use the LCSetGDS script in the [ aem-forms root] \sdk\misc\Foundation\SetGDSCommandline folder to set the GDS path after you change the file system paths. See the ReadMe.txt file in the same folder for details. If the old GDS directory path cannot be used, LCSetGDS script must be used to set the new path to the GDS before you start AEM forms.

Important: This circumstance is the only one under which you should use this script to change the GDS location. To change the GDS location while AEM forms is running, use Administration Console. (See Configure general AEM forms settings .)

After you set the GDS path, start the forms server in maintenance mode, and use the administration console to update the remaining file system paths for the new node. After you verify that all of the necessary configurations are updated, restart and test AEM forms.

Files to back up and recover

The application and data files that must be backed up are described in more detail in the following sections.

Consider the following points regarding backup and recovery:

  • The database should be backed up before GDS and AEM repository.

  • If you need to bring down the nodes in a clustered clustered environment for backup, ensure that the slave nodes are shut down before the master node. Otherwise, it can lead to inconsistency in the cluster or server. Also, the master node should be made live before any slave node.

  • For the restore operation of a cluster, application server should be stopped for each node in the cluster.

Global Document Storage directory

The GDS is a directory used to store long-lived files that are used within a process. The lifetime of long-lived files is intended to span one or more launches of a AEM forms system, and can span days and even years. These long-lived files can include PDFs, policies, and form templates. Long-lived files are a critical part of the overall state of many AEM forms deployments. If some or all long-lived documents are lost or corrupted, the forms server may become unstable.

Input documents for asynchronous job invocation are also stored in the GDS and must be available to process requests. Therefore, it is important that you consider the reliability of the file system that hosts the GDS and employ a redundant array of independent disks (RAID) or other technology as appropriate for your quality and level of service requirements.

The location of the GDS is determined during the AEM forms installation process or later by using administration console. In addition to keeping a high-availability location for GDS, you can also enable database storage for documents. See Backup options when database is used for document storage .

GDS location

If you leave the location setting empty during installation, the location defaults to a directory under the application server installation. You must back up the following directory for your application server:

  • (JBoss) [appserver root] /server/ [server] /svcnative/DocumentStorage

  • (WebLogic) [appserverdomain] / [server] /adobe/AEMformsserver/DocumentStorage

  • (WebSphere) [appserver root] /installedApps/adobe/ [server] /DocumentStorage

If you changed the GDS location to a non-default location, you can determine it as follows:

  • Log in to administration console and click Settings > Core System Settings > Configurations.

  • Record the location that is specified in the Global Document Storage Directory box.

In a clustered environment, the GDS typically points to a directory that is shared on the network and is read/write accessible for every cluster node.

The location of the GDS may be changed during a recovery if the original location is no longer available. (See Changing the GDS location during recovery .)

Backup options when database is used for document storage

You can enable AEM forms document storage in the AEM forms database using the administration console. Even though this option keeps all persistent documents in the database, AEM forms still requires the file system-based GDS directory because it is used to store permanent and temporary files and resources related to sessions and invocations of AEM forms.

When you select the “Enable document storage in the database” option in the Core System Settings in the administration console or by using Configuration Manager, AEM forms does not allow snapshot backup mode and rolling backup mode. Therefore, you do not need to manage backup modes using AEM forms. If you use this option, you should back up the GDS only once after enabling the option. When you recover AEM forms from a backup, you do not need to rename the backup directory for the GDS or restore GDS.

AEM repository

AEM repository (crx-repository) is created if crx-repository is configured while installing AEM forms. The location of the crx-repository directory is determined during the AEM forms installation process. AEM repository backup and restore is required along with database and GDS for consistent AEM forms data in AEM forms. AEM repository contains data for Correspondence Management Solution, forms manager, and HTML Workspace.

Correspondence Management Solution

Correspondence Management Solution centralizes and manages the creation, assembly and delivery of secure, personalized, and interactive correspondences. It enables you to quickly assemble correspondence from both pre-approved and custom-authored content in a streamlined process from creation to archival. As a result, your customers get timely, accurate, convenient, secure, and relevant communication. Your business maximizes the value of customer interactions and minimizes cost and risk with a process that is streamlined for ease, speed, and productivity.

A simple Correspondence Management Solution setup includes an author instance and a publish instance on the same machine or on different machines

forms manager

forms manager streamlines the process of updating, managing, and retiring forms.

HTML Workspace

HTML Workspace matches the capabilities of the (Deprecated for AEM forms on JEE) Flex Workspace and adds new capabilities to extend and integrate Workspace and make it more user-friendly.

Note: The Flex Worksapce is deprecated for AEM forms release.

It allows for task management on clients without Flash Player and Adobe Reader. It facilitates rendition of HTML Forms, besides PDF forms and Flex forms.

AEM forms database

The AEM forms database stores content such as form artifacts, service configurations, process state, and database references to files in the GDS and the Content Storage Root directory (for Content Services). Database backups can be performed in real time without an interruption in service, and recovery can be to a specific point in time or to a particular change. This section describes how to configure your database so that it can be backed up in real time.

On a properly configured AEM forms system, the system administrator and the database administrator can easily collaborate to recover the system to a consistent, known state.

To back up the database in real time, you must either use snapshot mode or configure your database to run in the specified log mode. This allows your database files to be backed up while the database is open and available for use. Furthermore, the database preserves its rollback and transaction logs when it is running in these modes.

Note: Adobe® LiveCycle® Content Services ES (Deprecated) is a content management system installed with LiveCycle. It enables users to design, manage, monitor, and optimize human-centric processes. Content Services (Deprecated) support ends on 12/31/2014. See Adobe product lifecycle document . To know about configuring Content Services (Deprecated), see Administering Content Services .

DB2

Configure your DB2 database to run in archive log mode.

Important: If your AEM forms environment was upgraded from a previous version of AEM forms and uses DB2, online backup is not supported. In this case, you must shut down AEM forms and perform an offline backup. Future versions of AEM forms will support online backup for upgrade customers.

IBM has a suite of tools and help systems to help database administrators manage their backup and recovery tasks:

DB2 has built-in capabilities to back up a database to Tivoli Storage Manager. By using Tivoli Storage Manager, DB2 backups can be stored on other media or the local hard drive.

For more information about DB2 database backup and recovery, see Developing a backup and recovery strategy for DB2 .

Oracle

Use snapshot backups or configure your Oracle database to run in archive log mode. (See Oracle Backup: An Introduction .) For more information about backing up and recovering your Oracle database, go to these sites:

Oracle Backup and Recovery: Explains the concepts of backup and recovery and the most common techniques for using Recovery Manager (RMAN) for backup, recovery, and reporting in more detail, as well as providing more information about how to plan a backup and recovery strategy.

Oracle Database Backup and Recovery User's Guide: Provides in-depth information about RMAN architecture, backup and recovery concepts and mechanisms, advanced recovery techniques such as point-in-time recovery and database flashback features, and backup and recovery performance tuning. It also covers user-managed backup and recovery, using host operating system facilities instead of RMAN. This volume is essential for backup and recovery of more sophisticated database deployments and for advanced recovery scenarios.

Oracle Database Backup and Recovery Reference: Provides complete information about syntax and semantics for all RMAN commands, and describes the database views that are available for reporting on backup and recovery activities.

SQL Server

Use snapshot backups or configure your SQL Server database to run in transaction log mode.

SQL Server also provides two backup and recovery tools:

  • SQL Server Management Studio (GUI)

  • T-SQL (command line)

See Backup Strategies and Backup and Restore .

MySQL

Use MySQLAdmin or modify the INI files in Windows to configure your MySQL database to run in binary log mode. (See MySQL binary logging .) A hot backup tool for MySQL is also available from InnoBase software. (See Innobase Hot Backup .)

Note: The default binary logging mode for MySQL is "Statement", which is incompatible with tables used by Content Services (Deprecated). Using binary logging in this default mode causes Content Services (Deprecated) to fail. If your system includes Content Services (Deprecated), use "Mixed" logging mode. To enable "Mixed" logging, add the following argument to the my.ini file: 
binlog_format=mixed 
log-bin=logname

You can use the mysqldump utility to obtain the full database backup. Full backups are required, but they are not always convenient. They produce large backup files and take time to generate. To do an incremental backup, ensure that you start the server with the – log-bin option as described in the previous section. Each time the MySQL server restarts, it stops writing to the current binary log, creates a new one and, from then on, the new one becomes the current one. You can force a switch manually with the FLUSH LOGS SQL command. After the first full backup, subsequent incremental backups are done by using the mysqladmin utility with the flush-logs command, which creates the next log file.

See Backup Strategy Summary .

Content Storage Root directory (Content Services only)

The Content Storage Root directory contains the Content Services (Deprecated) repository where all documents, artifacts and indexes are stored. The Content Storage Root directory tree must be backed up. This section describes how to determine the location of the Content Storage Root directory for both stand-alone and clustered environments.

Content Storage Root location (stand-alone environment)

The Content Storage Root directory is created when Content Services (Deprecated) is installed. The location of the Content Storage Root directory is determined during the AEM forms installation process.

The default location for the Content Storage Root directory is [aem-forms root] /lccs_data.

Back up the following directories located in the Content Storage Root directory:

/audit.contentstore

/contentstore

/contentstore.deleted

/backup-lucene-indexes

If the /backup-lucene-indexes directory is not present, back up the /lucene-indexes directory, also located in the Content Storage Root directory. If the /backup-lucene-indexes directory is present, do not back up the /lucene-indexes directory because it may cause errors.

Content Storage Root location (clustered environment)

When you install Content Services (Deprecated) in a clustered environment, the Content Storage Root directory is split into two separate directories:

Content Storage Root directory: Typically, a shared network directory that is read/write accessible for all nodes in the cluster

Index Root directory: A directory that is created on each node in the cluster, always having the same path and directory name

The default location for the Content Storage Root directory is [GDS root] /lccs_data, where [GDS root] is the location described in GDS location . Back up the following directories located in the Content Storage Root directory:

/audit.contentstore

/contentstore

/contentstore.deleted

/backup-lucene-indexes

If the /backup-lucene-indexes directory is not present, back up the /lucene-indexes directory, also located in the Content Storage Root directory. If the /backup-lucene-indexes directory is present, do not back up the /lucene-indexes directory because it may cause errors.

The default location for the Index Root directory is [aem-forms root] /lucene-indexes on each node.

Customer-installed fonts

If you installed additional fonts on your AEM forms environment, you must back them up separately. Back up all Adobe and customer font directories that are specified in administration console under Settings > Core System > Configurations. Ensure that you back up the entire font directory.

Note: By default, the Adobe fonts installed with AEM forms are located in the [aem-forms root]/fonts directory.

If you are reinitializing the operating system on the host computer and want to use fonts from the previous operating system, the contents of the system fonts directory should also be backed up. (For specific instructions, see the documentation for your operating system).

Backing up the AEM forms data

This section describes the steps that are required to complete a hot, or online, backup of the AEM forms database, the GDS, and Content Storage Root directories.

After AEM forms is installed and deployed to production areas, the database administrator should perform an initial full, or cold, backup of the database. The database must be shut down for this backup. Then, differential or incremental (or hot) backups of the database should be done regularly.

To ensure a successful backup and recovery, a system image backup must be available at all times. Then, if a loss occurs, you can recover your entire environment to a consistent state.

Backing up the database at the same time as the GDS, AEM repository, and Content Storage Root directory backups helps keep these systems synchronized if recovery is ever required.

The backup procedure described in this section requires you to enter safe backup mode before you back up the AEM forms database, AEM repository, GDS, and Content Storage Root directories. When backup is complete, you must exit safe backup mode. Safe backup mode is used to mark long-lived and persistent documents that reside in the GDS. This mode ensures that the automated file cleanup mechanism (the file collector) does not delete expired files until the safe backup mode is released. It is necessary to keep a GDS backup in synchronization with a database backup.

How often the GDS location must be backed up depends on how AEM forms is used and the backup windows available. The backup window can be affected by long-lived processes because they can run for several days. If you are continually changing, adding, and removing files in this directory, you should back up the GDS location more often.

If the database is running in a logging mode, as described in the previous section, the database logs also must be backed up frequently so that they can be used to restore the database in case of media failure.

Note: Files that are not referenced may persist in the GDS directory after the recovery process. This is a known limitation at this time.

Back up the database, GDS, AEM repository, and Content Storage Root directories

You must put AEM forms in either the safe backup (snapshot) mode or the rolling backup (continuous coverage) mode. Before you set AEM forms to enter either of the backup modes, ensure the following:

  • Verify the system version and record the patches or updates that were applied since the last complete system image backup was performed.

  • If you are using either rolling or snapshot mode backups, ensure that your database is configured with the correct log settings to allow for hot backups of the database. (See AEM forms database .)

In addition to these, observe the following guidelines for the backup/restore process.

  • Back up the GDS directory by using an available operating system or a third-party backup utility. (See GDS location .)

  • (Optional) Back up the Content Storage Root directory by using an available operating system or a third-party backup and utility. (See Content Storage Root location (stand-alone environment) or Content Storage Root location (clustered environment) .)

  • Back up author and publish instances (crx-repository backup).

    To back up the Correspondence Management Solution environment, perform the steps on the author and publish instances as described in Backup and Restore .

    Consider the following points when backing up the author and publish instances:

    • Ensure that backup for author and publish instances are synchronized to start at the same time.Although you can continue to use author and publish instances while the backup is being performed, it is recommended not to publish any asset during the backupto avoid any uncaptured changes. Wait for the backup of both author and publish instances to end before publishing new assets.

    • The complete backup of Author node includes backup of Forms Manager and HTML Workspace data.

    • Workbench developers can continue to work on their processes locally. They should not deploy any new processes during the backup phase.

    • The decision about the length of each backup session (for rolling backup mode) should be based on the total time taken to back up all the data in AEM forms (DB,GDS, AEM repository, and any other additional custom data).

You should back up the AEM forms database, including any transaction logs. (See AEM forms database .) For more information, see the appropriate knowledge base article for your database:

These articles provide guidance to basic database features for the backup and recovery of data. They are not intended as all-inclusive technical Guides of a specific vendor's database backup and recovery feature. They outline commands that are required to create a reliable database backup strategy for your AEM forms application data.

Note: The database backup must be complete before you begin backing up the GDS. If the database backup is not complete, your data will be out of sync.

Entering the backup modes

You can use either administration console, the LCBackupMode command, or the API available with the AEM forms installation to enter and leave backup modes. Note that for rolling backup (continuous coverage), the administration console option is not available; you should use either the command line option or the API. For information about using the API to enter and leave backup modes, see AEM forms API Reference .

Note: If you configured SSL on the forms server, then you cannot place the forms server in back up mode using LCBackupMode.CMD script.

Using the administration console to enter safe backup mode

  1. Log in to administration console.

  2. Click Settings > Core System Settings > Backup Utilities.

  3. Select Operate In Safe Backup Mode and click OK.

    This method puts AEM forms into backup mode indefinitely (no time out), and it enters snapshot mode rather than rolling backup mode.

Using the command line option to enter safe backup mode

You can use the command line interface LCBackupMode scripts to put AEM forms in safe backup mode.

  1. Set ADOBE_LIVECYCLE and start the application server.

  2. Go to the [aem-forms root] /sdk/misc/Foundation/BackupRestoreCommandline folder.

  3. Depending on your operating system, edit the LCBackupMode.cmd or LCBackupMode.sh script to provide default values that are appropriate for your system.

  4. At the command prompt, run the following command on a single line:

    • (Windows) LCBackupMode.cmd enter [-Host= hostname ] [-port= portnumber ] [-user= username ] [-password= password ] [-label= labelname ] [-timeout= seconds ]

    • (Linux, UNIX) LCBackupMode.sh enter [-host= hostname ] [-port= portnumber ] [-user= username ] [-password= password ] [-label= labelname ]

    In the previous commands, the placeholders are defined as follows:

    Host is the name of the host where AEM forms is running.

    port is the WebServices port of the application server on which AEM forms is running.

    user is the user name of the AEM forms administrator.

    password is the password of the AEM forms administrator.

    label is the text label, which can be any string, for this backup.

    timeout is the number of seconds after which the backup mode is automatically left. It can be 0 to 10,080. If it is 0, which is the default, the backup mode never times out.

    For more information about the command line interface to the backup mode, see the Readme file in the BackupRestoreCommandline directory.

Leaving backup modes

You can use either the administration console or the command line option to leave backup modes.

Leave safe backup mode (snapshot mode)

To use Administration Console to take AEM forms out of safe backup mode (snapshot mode), perform the following tasks.

  1. Log in to Administration Console.

  2. Click Settings > Core System Settings > Backup Utilities.

  3. Deselect Operate In Safe Backup Mode and click OK.

Leave all backup modes

You can use the command line interface to take AEM forms out of safe backup mode (snapshot mode) or to end the current backup mode session (rolling mode). Note that you cannot use the administration console to leave rolling backup mode. While in rolling backup mode, the Backup Utilities controls on the Administration Console are disabled. You must use either API call or use the LCBackupMode command.

  1. Go to the [aem-forms root] /sdk/misc/Foundation/BackupRestoreCommandline folder.

  2. Depending on your operating system, edit the LCBackupMode.cmd or LCBackupMode.sh script to provide default values that are appropriate for your system.

    Note: You must set the JAVA_HOME directory as described in the appropriate chapter for your application server in Preparing to Install AEM forms .

  3. Run the following command on a single line:

    • (Windows) LCBackupMode.cmd leaveContinuousCoverage [-Host= hostname ] [-port= portnumber ] [-user= username ] [-password= password ]

    • (Linux, UNIX) LCBackupMode.sh leaveContinuousCoverage [-Host= hostname ] [-port= portnumber ] [-user= username ] [-password= password ]

      In the previous commands, the placeholders are defined as follows:

      Host is the name of the host where AEM forms is running.

      port is the port on which AEM forms is running on the application server.

      user is the user name of the AEM forms administrator.

      password is the password of the AEM forms administrator.

      leaveContinuousCoverage Use this option to disable rolling backup mode completely.

    Important: For the time that backup mode is off, continuous coverage cannot be reestablished. Any changes during that time are not protected.
    Note: If you enabled document storage in database, the snapshot backup mode and rolling backup modes are not applicable.

    For more information about the command line interface to the backup mode, see the readme file in the BackupRestoreCommandline directory.

Recovering the AEM forms data

This section describes the steps required to recover the AEM forms data. Also see Special considerations for backup and recovery .

Important: The database, GDS, AEM repository, and Content Storage Root directories must be restored to a computer with the same DNS name as the original.

AEM forms should recover reliably from the following failures:

Disk Failure: The latest backup media is required to recover the database content.

Data Corruption: File systems do not record past transactions, and systems may accidentally overwrite required process data.

User Error: Recovery is limited to the data that is made available by the database. If the data was stored and is available, recovery is simplified.

Power Outage, System Crash: File system APIs are often not designed or used in a robust manner that guards against unexpected system failures. If a power outage or system crash occurs, document content that is stored in the database is more likely to be up to date than content that is stored on a file system.

If you are using rolling backup mode, you are still in backup mode after recovery. If you are using snapshot backup mode, you are not in backup mode after recovery.

When restoring from backup to a new system, the following configurations may be different. This difference should not affect a successful recovery of the AEM forms application:

  • IP address

  • Physical system configuration (CPUs, disk, memory)

  • GDS location

Important: The backup of the Content Storage Root directory must be restored to the location of that directory as it was set during Content Services configuration.

If a single node of a multinode cluster failed and the remaining nodes of the cluster are operating properly, perform the cluster single-node recovery procedure.

Recover the AEM forms data

  1. Stop the AEM forms services and application server if running.

  2. If necessary, re-create the physical system from a system image. For example, this step may not be necessary if the reason for recovery is a faulty database server.

  3. Apply patches or updates to AEM forms that were applied since the image was made. This information was recorded in the backup procedure. AEM forms must be patched to the same patch level as it was when the system was backed up.

  4. (WebSphere Application Server) If you are recovering to a new instance of WebSphere Application Server, run the restoreConfig.bat/sh command.

  5. Recover the AEM forms database by first running a database restore operation using the database backup files and then applying the transaction redo logs to the recovered database. (See AEM forms database .) For more information, see one of these knowledge base articles:

  6. Recover the GDS directory by first deleting the contents of the GDS directory on the existing installation of AEM forms and then copying the contents of the GDS directory from the backed-up GDS. If you changed the GDS directory location, see Changing the GDS location during recovery .

  7. Rename the GDS backup directory to be restored as shown in these examples:

    Note: If the /restore directory already exists, back it up and then delete it before you rename the /backup directory that contains the latest data.

    • (JBoss) Rename [appserver root] /server/ [server] /svcnative/DocumentStorage/backup to:

      [appserver root] /server/ [server] /svcnative/DocumentStorage/restore.

    • (WebLogic) Rename [appserverdomain] / [server] /adobe/AEMformsserver/DocumentStorage/backup to:

      [appserverdomain] / [server] /adobe/AEMformsserver/DocumentStorage/restore.

    • (WebSphere) Rename [appserver root] /installedApps/adobe/ [server] /DocumentStorage/backup to:

      [appserver root] /installedApps/adobe/ [server] /DocumentStorage/restore.

  8. Recover the Content Storage Root directory by first deleting the contents of the Content Storage Root directory on the existing installation of AEM forms and then recovering the contents by following the tasks for either stand-alone or clustered environments:

    Important: The backup of the Content Storage Root directory must be restored to the location of the Content Storage Root directory as it was set during Content Services (Deprecated) configuration.

    Standalone: During the recovery process, restore all the directories that were backed up. When these directories are restored, if the /backup-lucene-indexes directory is present, rename it to /lucene-indexes. Otherwise, the lucene-indexes directory should already exist and no action is required.

    Clustered: During the recovery process, restore all the directories that were backed up. To restore the Index Root directory, perform the following steps on each node of the cluster:

    • Delete all content in the Index Root directory.

    • If the /backup-lucene-indexes directory is present, copy the contents of the Content Storage Root directory /backup-lucene-indexes directory to the Index Root directory and delete the Content Storage Root directory /backup-lucene-indexes directory.

    • If the /lucene-indexes directory is present, copy the contents of the Content Storage Root directory /lucene-indexes directory to the Index Root directory.

  9. Restore/recover the CRX-repository.

    • Standalone

      Restore author and publish instances : If a disaster occurs, you can restore the repository to the last backed up state by performing the steps described in Backup and Restore.

      The complete restoration of Author node ascertains the restoration of Forms Manager and HTML Workspace data as well.

    • Clustered

      For restoration in a clustered environment, see Strategy for backup and restore in a clustered environment .

  10. Delete any AEM forms temporary files that were created in the java.io.temp directory or in the Adobe temp directory.

  11. Start AEM forms (see Starting and stopping services ) and the application server(s) (see Maintaining the Application Server ).

Changing the GDS location during recovery

If your GDS is restored to a location other than where it was originally, run the LCSetGDS script to set the GDS to the new location. The script is in the [ aem-forms root] \sdk\misc\Foundation\SetGDSCommandline folder. The script takes two parameters, defaultGDS and newGDS . See the ReadMe.txt file in the same folder for instructions on how to run the script.

Note: If you had enabled document storage in database, you don’t need to change the GDS location.
Important: This circumstance is the only one under which you should use this script to change the GDS location. To change the GDS location while AEM forms is running, use Administration Console. (See Configure general AEM forms settings .)
Important: Component deployment will fail on Windows if the GDS directory is at the drive root (for example, D:\). For GDS, you must make sure that the directory is not located at the root of the drive but is located in a subdirectory. For example, the directory should be D:\GDS and not simply D:\.

Recovering the GDS to a clustered environment

To change the GDS location in a clustered environment, shut down the entire cluster and run the LCSetGDS script on a single node of the cluster. (See Changing the GDS location during recovery .) Start only that node. When that node is fully started, other nodes in the cluster may be started safely and will correctly point at the new GDS.

Note: If you cannot ensure starting one node completely before starting other nodes, you must run the LCSetGDS script on every node in the cluster before you start the cluster.

PDF Generator backup limitations

The temporary directory that PDF Generator uses to convert files cannot be backed up. Even though the service will be restored properly, data can get lost because PDF Generator reviews and clears the contents of the temporary directory at set intervals.

Backup strategy for Connector for EMC Documentum users

If you have Connector for EMC Documentum installed, in addition to the instructions in this chapter, your backup and recovery strategy must include backing up (or recovering) the computer that the respective ECM system is installed on. (See the ECM Documentum documentation).

Back up your AEM forms environment by using ECM repository and performing the following tasks:

Restore AEM forms environment by using ECM repository and performing the following tasks:

Backup strategies for watched folders

This content describes how watched folders are affected by different backup and recovery scenarios, the limitations and outcomes of these scenarios, and how to minimize data loss.

Watched Folder is a file system-based application that invokes configured service operations that manipulate the file within one of the following folders in the watched folder hierarchy:

  • Input

  • Stage

  • Output

  • Failure

  • Preserve

A user or client application first drops the file or folder in the input folder. The service operation then moves the file into the stage folder for processing. After the service performs the specified operation, it saves the modified file in the output folder. Successfully processed source files are moved to the preserve folder, and failed processing files are moved to the failure folder. When the Preserve On Failure attribute for the watched folder is enabled, failed processed source files are moved to the preserve folder. (See Configuring watched folder endpoints .)

You can back up watched folders by backing up the file system.

Note: This backup is independent of the database or document storage backup and recovery process.

How watched folders work

This content describes the watched folder file manipulation process. It is important to understand this process before developing a recovery plan. In this example, the Preserve On Failure attribute for the watched folder is enabled. The files are processed in the order in which they arrive.

The following table describes the file manipulation of five sample files (file1, file2, file3, file4, file5) throughout the process. In the table, the x axis represents time, such as Time 1 or T1, and the y axis represents folders within the watched folder hierarchy, such as Input.

Folder

T1

T2

T3

T4

T5

T6

T7

Input

file1, file2, file3, file4

file2, file3, file4

file3, file4

file4

empty

file5

empty

Stage

empty

file1

file2

file3

file4

empty

file5

Output

empty

empty

file1_out

file1_out, file2_out

file1_out, file2_out

file1_out, file2_out, file4_out

file1_out, file2_out, file4_out

Failure

empty

empty

empty

empty

file3_fail, file3

file3_fail, file3

file3_fail, file3

Preserve

empty

empty

file1

file1, file2

file1, file2

file1, file2, file4

file1, file2, file4

The following text describes file manipulation for each time:

T1: The four sample files are placed in the input folder.

T2: The service operation moves file1 into the stage folder for manipulation.

T3: The service operation moves file2 into the stage folder for manipulation. It places the results of file1 in the output folder, and it moves file1 to the preserve folder.

T4: The service operation places file3 in the stage folder for manipulation. It places the results of file2 in the output folder, and it places file2 in the preserve folder.

T5: The service operation places file4 in the stage folder for manipulation. The manipulation of file3 fails, and the service operation places it in the failure folder.

T6: The service operation places file5 in the input folder. It places the results of file4 in the output folder, places file4 in the preserve folder.

T7: The service operation places file5 in the stage folder for manipulation.

Backing up watched folders

It is recommended that you back up the entire watched folder file system to another file system.

Restoring watched folders

This section describes how to restore watched folders. Watched folders often invoke short-lived processes that complete within a minute. In such cases, restoring the watched folder with a backup that is done hourly will not prevent data loss.

For example, if a backup is taken at time T1 and the server fails at T7, then file1, file2, file3, and file4 are already manipulated. Restoring the watched folder with a backup taken at T1 does not prevent data loss.

If a more recent backup was taken, you can restore the files. When restoring the files, consider which watched folder hierarchy folder the current file resides in:

Stage: Files in this folder are processed again after the watched folder is restored.

Input: Files in this folder are processed again after the watched folder is restored.

Result: Files in this folder are not processed.

Output: Files in this folder are not processed.

Preserve: Files in this folder are not processed.

Strategies to minimize data loss

The following strategies can minimize output and input folder data loss when restoring a watched folder:

  • Back up output and failure folders frequently, such as hourly, to avoid loss of result and failure files.

  • Back up the input files in a folder other than the watched folder. This ensures file availability after recovery in case you cannot find the files in either the output or failure folder. Ensure that your file naming scheme is consistent.

    For example, if you are saving the output with %F. extension , the output file will have the same name as the input file. This helps you to determine which input files are manipulated and which ones must be resubmitted. If you see only file1_out file in the result folder and not file2_out, file3_out and file4_out, you must resubmit file2, file3, and file4.

  • If the watched folder backup that is available is older than the time it takes to process the job, you should allow the system to create a new watched folder and automatically place the files in the input folder.

  • If the latest available backup is not recent enough, the backup time is less than the time it takes to process the files, and the watched folder is restored, the file was manipulated in one of the following different stages:

    • Stage 1: In the input folder

    • Stage 2: Copied to the stage folder but the process is not yet invoked

    • Stage 3: Copied to the stage folder and the process is invoked

    • Stage 4: Manipulation in progress

    • Stage 5: Results returned

    If files are in Stage 1, they will be manipulated. If files are in Stage 2 or 3, place them in the input folder for manipulation to take place again.

    Note: If manipulation of a file occurs more than once, data loss will be prevented but results may be duplicated.

Conclusion

Due to the dynamic and constantly changing nature of a watched folder, restoration of watched folders should be done with files that are backed up within a day. A best practice would be backing up the results, storing the input folder on a server, and tracking input files so that you can resubmit the job in case of failure.

Backing up and recovering the EMC Documentum repository

This section describes the tasks required to back up and recover the EMC Documentum repository configured for your AEM forms environment.

Note: These instructions assume that AEM forms with Connectors for ECM and EMC Documentum Content Server are installed and configured as required.

For both the backup and restore processes, there are two main tasks:

  • Backing up (or restoring) the AEM forms environment.

  • Backing up (or restoring) the EMC Documentum Content Server.

Note: Back up your AEM forms data before you back up the EMC Documentum system, and subsequently restore the EMC Documentum system before you restore the AEM forms environment.

Software requirements

To perform the necessary backup tasks on your EMC Documentum Content Server, purchase an appropriate third-party utility such as EMC NetWorker from EMC or CYA SmartRecovery for EMC Documentum from CYA. The following instructions describe the steps for using EMC NetWorker Module version 7.2.2.

You need the following EMC NetWorker modules:

  • NetWorker Module

  • NetWorker Configuration Wizard

  • NetWorker Device Configuration Wizard

  • NetWorker Module for the database type used by your Content Server

  • NetWorker Module for Documentum

Preparing the EMC Document Content Server for backup and recovery

This section describes installing and configuring the EMC NetWorker software on the Content Server.

Prepare the EMC Documentum server for backup

  1. On the EMC Documentum Content Server, install the EMC NetWorker modules, accepting all defaults.

    During the installation processes, you are prompted to enter the server name of the Content Server computer as the NetWorker Server Name . When installing the EMC NetWorker Module for your database, choose a "Complete" installation.

  2. Using the sample content below, create a configuration file named nsrnmd_win.cfg and save it to an accessible location on the Content Server. This file will be called by the backup and restore commands.

    The following text contains formatting characters for line breaks. If you copy this text to a location outside this document, copy a portion at a time and remove the formatting characters when you paste it to the new location. 

    ################################################ 
# NetWorker Module for Documentum v1.2 nsrnmd_win.cfg D5.3+ example with 
# typical set of working parameters.  THIS FILE MUST BE SITE-CUSTOMISED. 
# 
# Parameters not shown can be set in this file (as per site customisation) #or from the command-line. 
# 
# Please refer to the user Guides for details on all parameters, including  
# those not listed below. 
# Note: DCTM environment for D6 is slightly different from D5, refer to D6 
# Installation Guide to update the values. 
################################################ 
#Can get values for most of below from doing as the dctm inst owner: cmd> set DOCUMENTUM=C:\Documentum 
DM_HOME=C:\Documentum\product\6.0 
JAVA_HOME=C:\Progra~1\Documentum\java\1.5.0_12 
JAVA_PATH=C:\Progra~1\Documentum\java\1.5.0_12\bin 
PATH=C:\WINNT\system32;C:\WINNT;C:\WINNT\system32\WBEM;C:\Documentum\product\6.0\bin;C:\Documentum\fulltext\fast;C:\Documentum\product\6.0\fusion;C:\Program Files\Documentum\Shared;C:\Program Files\Legato\nsr\bin;C:\Program Files\Microsoft SQL Server\80\Tools\Binn;C:\Program Files\Microsoft SQL Server\90\DTS\Binn\;C:\Program Files\Microsoft SQL Server\90\Tools\binn;C:\Program Files\Microsoft SQL Server\90\Tools\Binn\VSShell\Common7\IDE;C:\Program Files\Documentum\java\1.5.0_12\bin;C:\Documentum\config;C:\Documentum 
#See also manifest dctm.jar file for class path locations 
CLASSPATH=.;C:\Program Files\Legato\nsr\bin;C:\Program Files\Legato\nsr\bin\nsrnmdde.jar;C:\Program Files\Documentum\java\1.5.0_12\lib\tools.jar;C:\Program Files\Documentum\Shared\dfc.jar;C:\Program Files\Documentum\Shared\aspectjrt.jar;C:\Program Files\Documentum\dctm.jar;C:\Program Files\Documentum\Shared\workflow.jar;C:\Program Files\Documentum\Shared\log4j.jar;C:\Program Files\Documentum\java\1.5.0_12\lib\dt.jar;C:\Documentum\config 
 
################################################ 
#If not using nsrnmdsv -m ALL|DB|DB_LOG|FTI|FTI_ALL|ICF|SA|SA_ALL, set #for backup 
NMD_SCOPE=ALL 
#Mandatory when scope (backup or restore) is FTI/SA without -a option 
#NMD_OBJECT_NAME=filestore_01 
################################################ 
NMDDE_DM_DOCBASE=Docbase 
NMDDE_DM_USER=Administrator 
#NMDDE_DM_PASSWD must be set via running: nsrnmdsv -f <nmdcfg> -P <pwd>. 
################################################ 
#DB related hooks to invoke arbitrary scripts: 
#Set if DB is on a remote host 
#NMD_DB_HOST=dbhost 
#Pure basename implies remote host execution; absolute path ... local 
#execution as in NMD v1.0. 
# 
#Remote execution requires script be put in remote nsrexecd bin directory 
#and D5.3+ host be added to remote nsr\res\servers file w/ nsrexecd recycled 
# 
#Refer to user Guides for sample script code. 
NMD_DB_FULL_BACKUP_CMD=C:\DocumentumBackup\Scripts\nsrnmddbf.bat 
NMD_DB_LOG_BACKUP_CMD=C:\DocumentumBackup\Scripts\nsrnmddbl.bat 
NMD_DB_INCR_BACKUP_CMD=C:\DocumentumBackup\Scripts\nsrnmddbi.bat 
################################################ 
#For D5.3+ only: NMD_FTI_INCLUDED, NMD_FTI_HOST, NMD_FTI_USER, 
#NMD_FTI_PASSWD & NMD_FTI_SUBDIRS. 
#Optional for remote D5.3+ FTI server 
NMD_FTI_INCLUDED=no 
#NMD_FTI_HOST=ftihost 
#Recommended for D5.3+ FTI server quiesce/unquiesce 
#NMD_FTI_USER=ftiadmin 
#The index name: optional for D5.3+ FTI server, used with -M FTI_ALL or 
#-M ALL   
#NMD_FTI_NAME=rep_name_ftindex_01 
#Recommended for D5.3+ FTI server quiesce/unquiesce 
#NMDDE_FTI_PASSWD must be set via running: nsrnmdsv -f <nmdcfg> -P <pwd> 
#-M FTI. 
#Pure basename implies remote host execution; absolute path ... local execution 
#as in NMD v1.0. 
# 
#Remote execution requires script be put in remote nsrexecd bin directory 
#and D5.3+ host be added to remote nsr\res\servers file w/ nsrexecd  
#recycled. 
# 
#See example nsrnmdfti*.bat examples. 
# 
#Mandatory for D5.3+ 
#NMD_BACKUP_FTI_QUIESCE=nsrnmdftiq.bat 
#NMD_BACKUP_FTI_UNQUIESCE=nsrnmdftiu.bat 
#Used for D5.3+ to get InstallProfile.xml FTI file in multinode 
#discovery. 
#Optional, if not specified, will treat as single-node FTI. 
#NMD_FTI_GET_INSTPROF=nsrnmdfti_instprof.bat 
#Mandatory for D5.3+. No spaces in paths or around comma separators. 
#For remote FTI, paths must be valid at the FTI host. 
#NMD_FTI_SUBDIRS=F:\Documentum_idx\data\fulltext\fixml,F:\Documentum_idx 
#\data\fulltext\index 
################################################ 
#Mandatory. No spaces in paths or before comma 
#separators in NMD_ICF_SUBDIRS_xxx: 
#NMD_ICF_INCLUDED=yes 
#NMD_ICF_SUBDIRS=C:\Documentum\dba,C:\Documentum\product\5.3 
################################################ 
NMD_FILEREPORT_INCLUDED=yes 
NMDDE_METADATA_OUTPUT_DEST=C:\docbase_freports\ 
################################################ 
#Other misc recommended NMD_xxx parameters 
#Recommended to get more meaningful saveset names 
NMD_USE_DEFAULT_SAVESET_NAMES=yes 
#Use following to skip unwanted ICF, FTI and non-SnapImage SA dirs/files. 
#For example, "<</>> +skip: dm_ftwork_dir" line will skip non-data 
#files. 
# 
#The path will be the same and must exist on D5.3+, remote FTI host, and 
#RCS hosts correspondingly if used. 
#NMD_DIRECTIVES_FILE=E:\Program Files\Legato\nsr\res\nsrnmddirectives.txt 
#For non-SnapImage SA backup 
#NMD_SA_FULL_NUM_SAVESET=16 
#NMD_SA_INCR_NUM_SAVESET=4 
#NMD_USE_SNAPIMAGE=yes 
################################################ 
# DSA setup 
################################################ 
# Name of the config file at the remote sites; 
# Mandatory, listed in the config file at the primary host. 
# (if skipped, backup is treated as local) 
# NMD_RCS_CFG_FILE=rep_name_rcs.cfg 
 
# SA-host mapping add, optional, will override far-store list info. 
# No space around comma. 
# NMD_HOST_SAS_MAP=host01,sa_01,sa_02 
# NMD_HOST_SAS_MAP=host02,sa_03 
################################################ 
NSR_SERVER=con-dctm 
#NSR_CLIENT=d5svrhost 
NSR_GROUP=Default 
NSR_DATA_VOLUME_POOL=Default 
#NSR_SNAPIMAGE_DATA_VOLUME_POOL=Default 
#Relocation dir will be the same on D5+ and remote FTI/SA hosts. 
NSR_RELOCATION=C:\reloc 
#NSR_PARALLELISM=16 
NSR_DEBUG_FILE=C:\Program Files\Legato\nsr\applogs\nmd.log 
NSR_DEBUG_LEVEL=9 
################################################ 
NMDDE_DM_PASSWD=XAtup9pl

    Keep the configuration file password field NMDDE_DM_PASSWD blank. You will set the password in the next step.

  3. Set the configuration file password as follows:

    • Open a command prompt, and change to [NetWorker_root] \Legato\nsr\bin.

    • Run the following command: -nsrnmdsv.exe -f <path_to_cfg_file> -P <password>

  4. Create the executable batch (.bat) files that are used to back up the database. (See the NetWorker documentation.) Set the details in the batch files according to your installation.

    • Full database backup (nsrnmddbf.bat):

      [NetWorker_database_module_root] -s <NetWorker_Server_Name> -U <username> -P <password> -l full <database_name>

    • Incremental database backup (nsrnmddbi.bat):

      [NetWorker_database_module_root] -s <NetWorker_Server_Name> -U <username> -P <password> -l 1 -R <database_name>

    • Database log backup (nsrnmddbl.bat):

      [NetWorker_database_module_root] -s <NetWorker_Server_Name> -U <username> -P <password> -l incr -R <database_name>

      Where:

      [NetWorker_database_module_root] is the installation directory of the NetWorker module. For example, the default installation directory for NetWorker Module for SQL Server is C:\Program Files\Legato\nsr\bin\nsrsqlsv.

      NetWorker_Server_Name is the server that NetWorker is installed on.

      username & password are the user name and password of the database administrator user.

      database_name is the name of the database to back up.

Create a backup device

  1. Create a new directory on the EMC Documentum server and share the folder by giving full rights to all users.

  2. Start EMC NetWorker Administrator and click Media Management > Devices.

  3. Right-click Devices and select Create.

  4. Enter the following values and click OK:

    Name: The full path of the shared directory

    Media Type: File

  5. Right-click the new device and select Operations.

  6. Click Label, enter a name, click OK, and then click Mount.

A device is added to which the backed up files will be saved. You can add multiple devices of different formats.

Back up the EMC Documentum Content Server

Perform the following tasks after you complete a full backup of your AEM forms data. (See Backing up the AEM forms data .)

Note: The command scripts require the full path to the nsrnmd_win.cfg file that you created in Preparing the EMC Document Content Server for backup and recovery .

  1. Open a command prompt, and change to [NetWorker_root] \Legato\nsr\bin.

  2. Run the following command: 

    - nsrnmdsv.exe -f <path_to_cfg_file>

Restore the EMC Documentum Content Server

Perform the following tasks before you restore your AEM forms data. (See Recovering the AEM forms data .)

Note: The command scripts require the full path to the nsrnmd_win.cfg file that you created in Preparing the EMC Document Content Server for backup and recovery .

  1. Stop the Docbase service that you are restoring.

  2. Start the NetWorker User utility for your database module (for example, NetWorker User for SQL Server ).

  3. Click the Restore tool and then select Normal.

  4. On the left side of the screen, select the database for your Docbase and click the Start button on the toolbar.

  5. When the database is restored, restart the Docbase service.

  6. Open a command prompt and change to [NetWorker_root] \Legato\nsr\bin

  7. Run the following command: 

    - nsrnmdrs.exe -B <docbase_name> -f <path_to_cfg_file> -C SA

Enabling and disabling safe backup mode

On the Backup Settings page, you can operate AEM forms in safe backup mode so that you can reliably back up your database and Global Document Storage (GDS) (GDS) directory.

While AEM forms is in safe backup mode, it operates normally, except that it does not actively remove files from the GDS directory.

Note: Setting this option does not back up your system; it prepares your system for backup.

Enable safe backup mode

  1. In administration console, click Settings > Core Systems Settings > Backup Settings.

  2. On the Backup Settings page, select Operate In Safe Backup Mode and click OK.

Note: If the system is already running in safe backup mode, a new reservation will not be created when you click OK.

Disable safe backup mode

  1. In administration console, click Settings > Core Systems Settings > Backup Settings.

  2. On the Backup Settings page, deselect Operate In Safe Backup Mode and click OK.

Strategy for backup and restore in a clustered environment

Note: If your AEM forms implementation stores additional custom data in a different database, you must implement a strategy to back up this data ensuring that it remains in sync with the AEM forms data. Also, the application must be designed so that it is robust enough to handle a scenario where the additional databases get out of sync. It is highly recommended that any database operation that is performed is done in the context of a transaction to help maintain a consistent state.

You need to back up the following parts of the AEM forms system to recover from any error:

  • Database used by AEM forms

  • GDS that has long lived data and other persistent documents

  • AEM database (crx-repository)

Note: You need to backup any other data that is being used by your AEM forms setup, such as customer fonts, connecters data, and so on.

Back up a clustered environment

This topic discusses the following strategies to back up any AEM forms clustered environment:

  • Offline backup with downtime

  • Offline backup with no downtime (backup of a slave node which is shutdown)

  • Online Backup with no downtime but delay in response

  • Back up the Bootstrap properties file

Offline backup with downtime

  1. Shut down the entire cluster and related services. (see Starting and stopping services )

  2. On any node, back up the database, GDS, and Connectors. (see Files to back up and recover )

  3. Perform the following steps to back up AEM repository offline:

    1. For each cluster node, back up the file that contains the cluster node id.

    2. Back up all files of any slave cluster node, including subdirectories.

    3. Back up repository/system id of each cluster node separately.

    For detailed steps, see Backup and Restore .

  4. Back up any other data, such as customer fonts.

  5. Start the cluster again.

Offline backup with no downtime

  1. Enter the rolling backup mode. (see Entering the backup modes )

    Note that we need to leave the rolling backup mode after a recovery.

  2. Shut down any of the slave nodes of the cluster with respect to AEM. (see Starting and stopping services )

  3. On any node, back up the database, GDS, and Connectors. (see Files to back up and recover )

  4. Perform the following steps to back up AEM repository offline:

    1. For each cluster node, back up the file that contains the cluster node id.

    2. Back up all files of any slave cluster node, including subdirectories.

    3. Back up repository/system.id of each cluster node separately.

    For detailed steps, see Backup and Restore .

  5. Back up any other data, such as customer fonts.

  6. Start the cluster again.

Online Backup with no downtime but delay in response

  1. Enter the rolling backup mode. (see Entering the backup modes )

    Note that you need to leave the rolling backup mode after a recovery.

  2. Shut down any of the slave nodes of the cluster with respect to AEM. (see Starting and stopping services )

  3. On any node, back up the database, GDS, and Connectors. (see Files to back up and recover )

  4. Perform the following steps to back up AEM repository online:

    1. For each cluster node, back up the file that contains the cluster_node.id.

    2. Back up repository/system.id of each cluster node separately.

    3. On any slave node, take an online backup of the repository for detailed steps see Online backup.

  5. Back up any other data, such as customer fonts.

  6. Start the cluster again.

Back up the Bootstrap properties file

When we create an AEM cluster, a properties file is created in the application server for all slave nodes. It is recommended to back up the Bootstrap properties file. You can find the file at the following location on your application server:

  • JBoss: in the BIN directory

  • WebLogic: in the domain directory

  • WebSphere: in the profile directory

You need to back up the file for disaster recovery scenario of AEM slave node and replace it at the specified location on the application server, if restored.

Recovery in a clustered environment

In case of any failure of the entire cluster or a single node, you need to restore it using the backup.

For a single node recovery, you just need to shut down the single node and run the single node recovery procedure.

In case the entire cluster fails due to failures like database crash, you need to perform the following steps. Restoration depends on the method of backup used.

Restoring a single node

  1. Stop the corrupted node.

    Note: If the corrupted node is an AEM master node, shut down the entire cluster node.

  2. Re-create the physical system from a system image.

  3. Apply patches or updates to AEM forms that were applied since the image was made. This information was recorded during the backup procedure. AEM forms must be recovered to the same patch level as it was when the system was backed up.

  4. ( Optional ) If all other nodes are working fine, it is possible that the AEM repository is alsocorrupted. In this case, you will see a repository unsync message in the error.log file of the AEM repository.

    To restore the repository, perform the following steps.

    Note: If a zipped crx-repository backup was taken online, unzip it at any location and follow the offline restoration process.

    1. Delete the repository, shared, version, and workspaces directories in the clusterNode directory of the node.

    2. Restore the backup of the cluster node (including subdirectories) to the node.

    3. Delete the file clusterNode/revision.log on the node.

    4. Delete the .lock on the node, if exists.

    5. Delete the repository/system.id on the node, if exists.

    6. Delete the files **/listener.properties on the node, if exist.

    7. Restore repository/cluster_node.id for individual cluster nodes.

Note: Consider the following points:

  • If the failed node was an AEM master node, copy all the content from the slave repository folder (crx-repository\crx.0000 where 0000 can be any digits) to the crx-repository\ repository folder and delete the slave repository folder.

  • Before restarting any cluster node, ensure that you delete the repository /clustered.txt from the master node.

  • Ensure that the master node is started first and once it is completely up, start other nodes.

Restoring the entire cluster

  1. Stop all the cluster nodes.

  2. Recreate the physical system from a system image.

  3. Apply patches or updates to AEM formsAEM formsthat were applied since the image was made. This information was recorded in step 1 of the backup procedure. AEM forms must be recovered to the same patch level as it was when the system was backed up.

  4. Restore the database, GDS, and Connectors.

  5. Do the following to recover the AEM repository offline:

    Note: If a zipped crx-repository backup was taken online, unzip it at any location and follow the offline restoration process.

    1. On all cluster nodes, delete the repository, shared, version, and workspaces directories in the clusterNode directory.

    2. Delete all files and directories in the shared directory.

    3. Restore the backup of the cluster node (including subdirectories) to one cluster nodes.

    4. Copy all files of the restored cluster node to all other cluster nodes. Once done, each cluster node contains the same data.

    5. Delete the file clusterNode/revision.log on all cluster nodes.

    6. Delete the .lock on all cluster nodes, if exists.

    7. Delete the repository/system.id all cluster nodes, if exists.

    8. Delete the files **/listener.properties on all cluster nodes, if exist.

    9. Restore repository/cluster_node.id for individual cluster nodes.

Note: Consider the following points:

  • If the failed node was an AEM master node, copy all the content from the slave repository folder (it looks like crx-repository\crx.0000 where 0000 can be any digits) to the crx-repository\ repository folder.

  • Before restarting any cluster node, ensure that you delete the repository /clustered.txt from the master node.

  • Ensure that the master node is started first and once it is completely up, start other nodes.

Back up and restore Correspondence Management Solution publish node

The publisher node does not have any master-slave relationship in a clustered environment. You can take backup of any Publisher node by following Backup and Restore .

Recover a single publisher node

  1. Shutdown the node that needs to be recovered and do not do any publish activity until the node is up again.

  2. Restore the Publish node using Restoring the Backup .

Recover a cluster

  1. Shutdown the cluster.

  2. Restore the Publish node using Restoring the Backup .

  3. Start the master node followed by the slave node of the author cluster.

// Ethnio survey code removed