3.0 - 3.8 - Backing up and Restoring Workspaces

Overview

ProcessMaker holds vital information about your organization's users and their work; therefore, it is very important to setup a backup system to safeguard the data being stored in ProcessMaker. If periodic backups aren't being made, one lightning strike or an accidental deletion of key files on your system could bring your organization to a grinding halt. Backups can also be very useful for returning to a previous state after making experimental changes to your system or backing up ProcessMaker.

It is recommended that your entire system be backed up in case the entire operating system needs to be restored. For the data stored in ProcessMaker, it is recommended to periodically backup your workspaces. Depending on how critical your data is and how often it is being updated, a daily backup or a weekly backup would be appropriate.

In general, it is recommended to backup and restore workspaces in the same version of ProcessMaker, because the database structure and file structure can change from one version of ProcessMaker to the next. Nonetheless, it is possible to backup a workspace from version 2.5.X and restore it in version 2.8 or version 3.0.

Note: It is recommended to have MySQL installed on the server at least as a service. Otherwise, MySQL will be executed externally, which will slow the response time of the backup/restore process.

Tested Systems

Backing up and restoring workspaces has been tested in the following systems. It will probably work in other systems as well, but it hasn't been tested.

(*) Even though there is a warning with mysql_connect(), the restore procedure finishes successfully.

Note that some issues were found when restoring backups from Windows 8.1 to Windows 8.1 and from Windows Server 2012 to Windows v. 8.1. These issues will be solved in future versions.

Note: The restoration and backup of a workspace can only be made in the same platform (Windows -> Windows, Linux -> Linux), because the names of the MySQL tables in Windows are not case sensitive, which may cause ProcessMaker to have problems.

Before Backing Up and Restoring Workspaces

Before backing or restoring a workspace, follow the next steps:

Step 1: In the server where ProcessMaker is running, open a command line prompt.

To open a command line prompt in Windows go to Start > All Programs > Accessories > Command Prompt. In most Linux/UNIX systems, it can be found under Accessories > Command Prompt.

Step 2: In the server where ProcessMaker is running, login as the root user or administrator. On Windows systems, start the command prompt as the administrator. On Linux/UNIX systems, open a terminal and login as root user or use the sudo command to gain root privileges:

Step 3: Navigate to the directory where ProcessMaker is installed. For example:

Backing Up Workspaces

The workspace backup utility creates a tar file that contains a dump of the database for each workspace and its associated files, which are stored in the INSTALL-DIRECTORY/shared/sites/{WORKSPACE}/ directory. Additional workspaces can be created in ProcessMaker, but most installations of ProcessMaker will have the default "workflow" workspace, which uses the wf_workflow database and stores its files in the INSTALL-DIRECTORY/shared/sites/workflow/ directory.

Issue the command to back up a workspace:

LINUX/UNIX:

Windows:

Remember that workspace names are case sensitive.

By default, the backup file will be created at:
    INSTALL-DIRECTORY/shared/backups/workspace.tar

However, the location of the backup file can be specified with the optional [BACKUP-FILE] parameter. This parameter can include forward slashes "/" to specify a path in Linux/UNIX or backward slashes "\" to specify a path in Windows. If the path or filename contains spaces, enclose it in quotation marks. If using a relative path, then it will start from the location of the processmaker script. If no path is specified, then the file will be created in the INSTALL-DIRECTORY/shared/backups directory.

Examples:

For help using the workspace-backup option, enter the command:

While executing a workspace backup, the output will indicate which tables are being backed up and print a summary of the backup at the end, similar to the following:

Backing Up Bigger Workspaces

Note: This parameter can NOT be used on Windows systems.

If ProcessMaker is installed on a 32 bit Linux/UNIX system that does not have Large File Support enabled, then the .tar backup file can not be larger than 2GB in size.
To back up a workspace that is larger than 2 GB, use the -s:[MAX-SIZE] or --filesize=[MAX-SIZE] option to break the backup file into multiple compressed files:

By default, the maximum size of the backup file is 1000 megabytes. To use a different maximum file size, set the [MAX-SIZE] in megabytes.

For example, if this option is set to 512 when backing up the "dev" workspace on a Linux system:

Then, a 1.5 GB backup will produce 3 backup files named dev.tar.01, dev.tar.02 and dev.tar.03.

Backing Up Multiple Workspaces

Back up multiple workspaces at the same time with the command:

Separate each workspace name with a space. The last parameter will be considered the name of the backup file.

For example, to backup the "workflow", "sales" and "Finance" workspaces to the file "pmBackup.tar":

Automating Backups

Backups should be made periodically, so it is best to automate the backup process.

Linux/UNIX

In Linux/UNIX, the backups can be scheduled as a cron job. For more information on the format of the crontab file, see Configuring crontab in Linux/UNIX.

For example, the following line can be added to the /etc/crontab file to automatically backup the "workflow" workspace every 12 hours and save it to a file at /opt/processmaker/backups/workflow.tar:

Inserting Date/Time in File Names

Since the backup file will be overwritten every 12 hours, it will not be possible to use the processmaker script's workspace-restore command to roll the workspace back to a state older than 12 hours.

For example, if a database was corrupted 3 days ago, the backup file will have already been overwritten, so it isn't possible to recover the old database. Therefore, it is recommended to use a time stamp in the backup file name, so multiple backup files can be maintained. However, old backup files also need to be periodically eliminated so that the backups don't eat up all the hard drive space on the server.

These cron jobs can be used to make a backup file every 12 hours and save it to a file at /root/backups/workflow_YYYY-MM-DD_HH-MM.tar, as well as automatically delete the old backup files more than 30 days old:

The current date and time is automatically inserted into the filename with the date command, such as workflow_2011-03-21_12-01.tar. Make sure to use `` (backticks), so Linux/UNIX will execute the date command, rather than treating it like a quoted string.

The second command automatically deletes any files older than 30 days (30days x 24hours = 720hours) in the /root/backups/ directory every night at midnight, so old backup files will not occupy too much space. If on a Debian/Ubuntu/Mint server, use the tmpreaper command instead of tmpwatch.

Storing Backups Remotely

It is recommended to keep backup copies on a different server (in a different location), so that a hard drive failure or lightning strike won't also destroy the backup copies on the ProcessMaker server. The following script located at /usr/local/bin/pmbackup.sh can be used to make a backup of the "workflow" workspace and store it in another location:

Make sure that the script file is executable:

To copy the backup files to the remote server at backups.example.com without having to enter a password for the root user, generate a public/private key with the command:

Add the public key to the /root/.ssh/authorized_keys file in the backups.example.com server. For more information, see this article.

Then, add the pmbackup.sh script as a cron job on the ProcessMaker server by adding the following line to the /etc/crontab file:

Windows

On Windows systems, workspaces can be periodically backed up as a scheduled task. Use the SCHTASKS command in DOS to create a scheduled task that periodically executes the processmaker workspace-backup command. For more information on using the SCHTASKS command, see Executing cron.php with the SCHTASKS command.

Examples:

Inserting Date/Time in File Names

It is not possible to insert the current date and time into the backup file name using the system %date% and %time% variables, since SCHTASKS treats them as literal strings. Instead, create a .bat file to execute the processmaker workspace-backup command. For example, the file C:\Users\Bob\pmbackup.bat contains:

Then, use the SCHTASKS command to schedule the periodic execution of the C:\Users\Bob\pmbackup.bat file:

Storing Backups Remotely

To securely copy files from Windows, install PuTTY's pscp program on the ProcessMaker server. Then, create a .bat file to execute the processmaker workspace-backup command and use pscp to copy the backup file to a remote server.

For example, the file C:\Documents and Settings\Bob\pmbackup.bat contains the following code to copy the backup file to the /root/backups directory on a remote Linux/UNIX server with a domain name of "example.com":

Then, use the SCHTASKS command to schedule the periodic execution of the C:\Documents and Settings\Bob\pmbackup.bat file:

To copy the backup files to a remote Windows server, install an SSH server on the remote Windows server (with programs such as freeSSHd, CygWin or WinSSHD), so it can accept secure file transfers. Another option is using PuTTY's psftp command to transfer the files using the SFTP protocol.

Errors While Backing Up Workspaces

The following section displays possible error while backing up workspaces.

Mysqldump Error

By default, mysqldump locks all the tables it's about to dump. This ensures the data is in a consistent state during the dump.

The following error can occur:

It shows because the requested database breaks the data consistency while dumping (e.g. invalid tables, columns, functions or definer/invoker). Usually this error occurs when there is mysqldump with big data. Since ProcessMaker uses a motor engine in various workspaces, this error should occur because a new thread starts that locks tables and the next user can see this error.

There are two solutions to this error:

• Use one of these mysqldump commands: mysqldump --opt --lock-tables=false -u [USER] -p [DBNAME] > dump.sql mysqldump --opt --single-transaction -u [USER] -p [DBNAME] > dump.sql
• Add the 'LOCK TABLES' permission to the user: GRANT SELECT,LOCK TABLES ON wf_[DBNAME_original].* TO 'wf_[new_DBNAME]'@'[ServerIP]';

Restoring Workspaces

The restoration and backup of a workspace can only be made on the same platform (Windows -> Windows, Linux -> Linux), because the names of the MySQL tables in Windows are not case sensitive, which may cause ProcessMaker to have problems.

Note: Before the restoration takes place, it is necessary to apply the File Permissions as it was a new installation of ProcessMaker in order to work properly.

To restore a workspace from a backup use the command:

LINUX/UNIX

Windows

If the backup file has a .tar extension and it is located in the INSTALL-DIRECTORY/shared/backups directory, then it is not necessary to specify the path or the file extension.

If it is not located in the shared/backups directory, or it doesn't have a .tar extension, then specify the path. Remember to enclose it within double quotes if the file name or path contains spaces.

By default, the backup will be restored to the same workspace name and use the same database names as the original. To overwrite an existing workspace, use the -o or --overwrite option. For example, to overwrite the existing "workflow" workspace:

To restore to a different workspace name, specify the optional [WORKSPACE-NAME] parameter. The files from the backup workspace will be copied into the INSTALL-DIRECTORY/shared/sites/[WORKSPACE-NAME]/ directory.

Remember that the workspace name should only contain 13 valid characters in MySQL databases or less, so it can not contain hyphens "-", spaces, periods, commas, semicolons or other special characters. To avoid problems, it is recommended to only use ASCII letters, numbers and underscores "_" for workspace names, since the workspace name should be easy to represent in a web address. Remember that workspace names are case sensitive when being entered in a URL in a web browser, but MySQL databases in Windows are not case sensitive, whereas MySQL databases in Linux are case sensitive.

Restoring a workspace takes a large amount of time. If restoring a workspace that users need to access, it is recommended to do the restore operation during a period when users will not need to access the workspace. The execution time of the processmaker workspace-restore command has been improved to some degree, but the restoration of a workspace with a few cases will still take at least 10 minutes, and large workspaces with many cases can take much longer.

Examples:

While the processmaker backup-restore command is executing, it will not display any output on the screen for many minutes. When it has terminated, it will display information similar to the following on the screen:

Warning: A workspace from a newer version of ProcessMaker can NOT be restored in an older version of ProcessMaker. For example, restoring from v.3.0 to v.2.5 will not work. However, it may be possible to restore a workspace from an older version to an newer version of ProcessMaker, although error messages may be displayed during the restore process.

After logging into the new workspace for the first time, a dialog box will appear, asking that the Enterprise license file be uploaded for the workspace. If the same organization is still using the workspace, then the same license_XXXXXXXXXXXXX.dat file can be used for the new workspace as well. There is no limit on the number of workspaces that can be created with an Enterprise License. If a different organization will be using the workspace, then contact ProcessMaker Sales to obtain a new Enterprise license.

Click on Browse and select the license_XXXXXXXXXXXXX.dat file to upload it. If the license file is valid, then the login screen will appear again with the message "License installed successfully":

To only show information about a backup file, use the -i option. For example, to show information about the workspace(s) in the shared/backups/sales.tar backup file:

To restore a workspace from a backup file that has multiple workspaces, use the -wWORKSPACE or --workspace=WORKSPACE option to specify which workspace should be restored. For example, to restore the "finance" workspace from the file /root/pmBackup.tar and overwrite the existing "accounting" workspace:

or:

For help using the processmaker workspace-restore command:

The -m option can be used to restore workspaces from multiple backup files (created with the -s option). Specify the name of the backup file, but do not include the numbers at the end of the file name. For example, if restoring from three backup files named pmbackup.tar.00, pmbackup.tar.01 and pmbackup.tar.02, then issue the command:

ProcessMaker will automatically join all the files whose file names start with "pmbackup.tar" when restoring the workspace(s).

BACKUP-FILE1 is the name of the first backup file, which typically has the extension tar.00.

Note: All the workspace files from the backup will be created in the INSTALL-DIRECTORY/shared/sites/WORKSPACE/ directory, with the same file permissions and same file owner as the shared directory. If not, for greater security, change folders and files permissions of the restored workspace to the same permissions as in the shared folder.

After Restoring a Workspace

After using the processmaker workspace-restore command, it is strongly recommended to issue the following command that repairs the cache of cases in the APP_CACHE_VIEW table: php processmaker cacheview-repair [WORKSPACE]

If no workspace is specified, then the cache will be repaired for all workspaces.

If this command is not executed, the number of cases listed under the Home menu will not be correct. To sort the cases with a language other than English (which is the default), use the -lISO-CODE or --lang=ISO-CODE option to specify the ISO 639-1 code for another language, such as es (Spanish), pt-BR (Brazilian Portuguese), or zh-CN (mainland Chinese). For example, to repair the case cache of the "workflow" and "development" workspaces in French in a Linux system:

In addition, if restoring from a different a workspace from a different version of ProcessMaker, the following command must be executed to update the case schema used by the new version of ProcessMaker:

For example, to migrate the case schema for the "workflow" and "development workspaces on a Linux server:

Note: Take into account that until ProcessMaker version 3.0.1.5, this command only worked for workspaces that had only one database wf_<WORKSPACE> and not for workspaces that kept on working with the three databases ProcessMaker used to manage.

Cleaning and restoring table PLUGIN_REGISTRY table

After you have restored the workspace, you can follow these steps to clean the cache and force ProcessMaker to read the updated table PLUGIN_REGISTRY:

1. Remove all records from table PLUGINS_REGISTRY.

Warning: Do not remove the enterprise plugin (ProcessMaker Enterprise Core Edition). If you delete the enterprise plugin, ProcessMaker becomes unstable. Therefore, it is important to not erase that plugin.

2. Apply the following command to clear the application cache.

3. Remove the directory cache inside ProcessMaker.

4. Reload ProcessMaker site.