3.0 - 3.8 - FTP Monitor

Plugin Version: 3.1.2 | Release Notes | Old Versions

Overview

FTP Monitor is a plugin that creates a new case for each new file in the directory by checking the FTP or Shared/Local directory periodically.

Requirements

ProcessMaker Compatibility

This plugin is fully compatible with the following ProcessMaker versions:

• v. 2.5.2.x
• v. 2.8.0.x
• v. 3.0.1.x
• v. 3.1.x
• v. 3.2.x
• v. 3.3.x

How the Plugin Works

The FTP Monitor needs to have access to a valid FTP or Shared/Local directory. For each file uploaded to the directory, a new case will automatically be created and routed to the second task in the process. The file is added as an Input Document file in the new case. For this reason, the process needs to include an initial task, which will automatically be skipped by the FTP Monitor, and an Input Document. The FTP Monitor checks for new files in the directory when ProcessMaker's cron.php file is executed. To ensure that FTP Monitor is regularly checking the directory, a cron job in Linux/UNIX or a Scheduled Task in Windows should be created to periodically execute ProcessMaker's cron.php file.

In addition, if an XML file is placed in a directory with the same filename as the file that initiated the case, then its XML data will be parsed and added as case variables in the new case. The passed variables can be used in fields in Dynaforms, triggers, conditions and evaluation routing rules, or to assign users to tasks.

Installation and Configuration

Log in with a user, such as "admin", who has the PM_SETUP_ADVANCE permission in his/her role and then go to Admin > Plugins > Enterprise Manager. Either install the FTP Monitor plugin by clicking on Install from File and uploading the plugin file or by clicking on its Install Now button in the list of available plugins.



After installing the plugin, make sure that the FTP Monitor plugin is  enabled.



Creating an FTP Monitor Connection

Once the plugin is enabled, go to Admin > Plugins and the  FTP Monitor Setup option will be available in the sidebar. Click on it and all the available connections will be listed:



In connection list, the menu offers the following options:



• New: Create a new FTP or SHARED/LOCAL connection by clicking on  New.
• Edit: Edit an existing connection by choosing one in the list and clicking on Edit.
• Delete: Delete an existing connection by choosing one in the list and clicking on Delete.
• Disable: Disable an existing connection by choosing one in the list and clicking on Disable.
• Enable: Enable an existing connection by choosing one in the list and clicking on Enable.

To create a new connection, click on New and a form will be displayed to define the properties of the new connection. The form contains two sections and provides a Test connection button to check whether the connection is correct.



Select whether the Connection Type is:

• FTP
• LOCAL

Depending on the selection, different properties will need to be defined.

FTP Connection Properties

Creates cases from a specific directory on an FTP server.

Basic Information FTP

• Host: The server address where the FTP server is located.
• Port: The server port for FTP. Leave blank to use the default port (21 for FTP or 22 for SSH).
• User: Enter the username if FTP server needs authentication. Otherwise, leave it blank.
• Password: The password if the FTP server needs authentication. Otherwise, leave it blank.
• Path: Specify the directory where files will be uploaded to initiate new ProcessMaker cases.
• Search Pattern: Use glob pattern matching to specify which files will be used to initiate new cases.
  
  • Use * to match any number of characters (including zero).
  • Use ? to match any single character.
  • Use [xyz] to match one of the specified characters.
  • Use \ (backslash) to escape a character so it won't be treated as a wildcard.

  Examples:

  • * would accept all files.
  • *.doc would except only Word documents.
  • *invoice* would accept any files that have the word "invoice" in the filename
  • [aeiou]*.pdf would accept any PDF files that start with a vowel in the filename.

• Pass Variables in an XML file: Mark this option if variables in an XML file will be added as case variables to the new cases. See XML File Format for more information.

Process Information

• Process: Select the process (in the current workspace) where new cases will be created.
• Task: Select the initial task where the new case will begin.
• User: Select the user who will be designated to work on the first task in the case. Only users who are assigned to the first task are available to be selected.
• Input Document: Choose the Input Document that the uploaded file will be attached to.
• Active: Select this option if the connection will be used; otherwise, unselect it to deactivate the connection so the directory will not be monitored and new cases will not be created.

Local Connection Properties

A local directory located on the server where ProcessMaker is installed (or a NAT addressable from the local directory structure) is used to initiate new ProcessMaker cases.



This option does not require authentication, but make sure that this directory can be read by Apache, which is running ProcessMaker. On Linux/UNIX systems, the user running Apache (apache in Red Hat/Fedora, www-data in Debian/Ubuntu, and wwwrun (or the www group) in SuSE/OpenSUSE) should have read access to the directory.

XML File Format

To automatically create case variables in the new case, an XML file can be placed in the specified directory to initiate new cases. The XML file must have the following structure:

<metadata file='filename.extension'>
  <field1>value1</field1>
  <field2>value2</field2>
  ...
</metadata>

Where filename.extension is the filename of the file initiating the case, and field1 and field2 will become case variables in the new case, which can be used in Dynaform fields or in trigger code. Note that variable names are case sensitive and that all values will be treated as strings and can contain spaces and new lines.

If the value will be used in a ProcessMaker date field, the date should be in YYYY-MM-DD or YYYY-MM-DD HH:MM:SS format, such as 2014-12-31 or 1970-01-25 18:59:00.

If the value is a number that will be used in a mathematical calculation by ProcessMaker, it should not have a thousands separator and "." (dot) should be used as the decimal point. For example, 123456 (integer) or 89797.99 (floating point number).

The XML file must have the same filename as the file that is used to initiate the new case; otherwise, the FTP Monitor won't recognize the XML file and associate its data with the right case.

For example, to start a case with a file named clientAcmeConsulting.pdf, the XML file should be named clientAcmeConsulting.xml. To create a new case with the case variables "clientFirstName", "clientLastName", "address" and "contractAmount", the following XML file can be used:

<metadata file='clientAcmeConsulting.pdf'>
   <firstName>John Robert</firstName>
   <lastName>Doe</lastName>
   <address>234 Oak St.
Littletown CA 73613</address>
   <contractAmount>2800.00</contractAmount>
 </metadata>

Warning: Please take note that it is NOT possible to pass grid variables through the mentioned XML file.

Examples Creating Cases

Using an FTP Connection

Uploading Images to the Server

This first example will show how to create cases from image files uploaded to an FTP server.

Step 1 - Creating the connection: Create an FTP connection to specify the location where the image files will be uploaded:



Note that in this example, the FTP Monitor will only create new cases, but will not create case variables in those cases, since the the option Pass variables in an XML file was not selected.

Step 2 - Upload Images: Upload image files to the FTP directory:



Step 3 - Execute cron.php: After creating the FTP Monitor connection, configure the server to periodically execute the cron.php file.

It is a good idea to manually execute the cron.php file at least one time to make sure that new cases are being created correctly. When cron.php is executed, it will report how many new files were found in the directory to initiate new cases. In the following example, two files were found to initiate new cases.



The second part of the execution details information about each case created with the corresponding case number:



In ProcessMaker two new cases were created in the Inbox:



Step 4 - Route the case: Choose a case and open it to see the image file that initiated the case and was attached as an Input Document file:



Using a Local/Shared Connection

Follow this example if the FTP Monitor starts cases by using files placed in the local directory on the server where ProcessMaker is installed. Follow the steps below to create cases using a local connection.

Step 1 - Create the connection:Create the connection by choosing the "LOCAL" connection type. Select a process with at least two tasks:



Step 2 - Upload files to the ProcessMaker server: Now, place files in the directory on the same server where ProcessMaker is installed. In this example, the monitored directory is inside the ProcessMaker main directory. Two files are uploaded, so two cases will be created.



An XML file has been placed in the directory, but the Pass variables in an XML files option wasn't marked, so no variables will be passed and the file will be treated as an independent file when starting a case.

Step 3 - Execute cron.php: Now execute the cron.php file:



In the details output by cron.php, the two new cases are listed, along with the names of the uploaded files. Log in to ProcessMaker with the user who was assigned to the two cases to see them in the Inbox:



Open one of the cases to see the file that was used to initiate the case. It should have been added as an Input Document file:



Using an XML File and Conditional Routing

The following example will show how to start a case from a file and its corresponding XML file, which passes variables to the case to automatically fill fields in a Dynaform. One of the passed variables named "shipToCountry" is used in a exclusive gateway to route the case.

Step 1 - Create a process with a exclusive gateway: Create a process whose first task is named Dummy Task, since this task will automatically be executed by the FTP Monitor and route onto the next task using an evaluation routing rule. This routing rule will choose between 3 possible tasks, which terminate the process.



Create an Input Document named "Invoice" and assign it as the first step of the Dummy Task.



The exclusive gateway will route the process to the "USA", "Canada" or "Unrecognized Country" tasks, based upon the value of the @@shipToCountry case variable, which comes from the XML file.



Step 2 - Create Dynaform: Create the following DynaForm with the following fields:



Remember that the variables assigned to these fields must be the variables that contain the data in the XML file.



Set this Dynaform as the second step in the tasks "USA", "Canada" and "Unrecognized Country", so the user can view the input document and the data when running a case. Remember that all steps in the initial task ("Dummy Task") are automatically skipped by FTP Monitor when it initiates a case.



Step 3 - Create the connection: Create a local FTP Monitor connection, mark the Pass variables in an XML file option and include *invoice* as a search pattern so that only files with the word "invoice" can initiate cases.



Step 4 - Create the XML data file: For every case that will be initiated by the FTP Monitor, create an XML file with the following structure:

<metadata file='35307980_invoice.png'>
  <invoiceNumber>123</invoiceNumber>
  <totalAmount>780.25</totalAmount>
  <invoiceDate>2016-07-4</invoiceDate>
  <shipToName>Harry Mason</shipToName>
  <shipToAddress>7366 Clear Robin Front</shipToAddress>
  <shipToCity>Never Spur</shipToCity>
  <shipToState>MT</shipToState>
  <shipToZipCode>59893-6673</shipToZipCode>
  <shipToCountry>US</shipToCountry>
  <shipToPhone>(406) 972-2585</shipToPhone>
</metadata>

Each field name in the XML file must have the same name as the variable names used in the Dynaform. Remember that field names are case sensitive.

The XML can be automatically created using a database trigger when a new invoice is added to the database, or using a stored procedure that is periodically executed. For example, the following MySQL query in the INVOICES table will create an XML data file for a specified invoice number:

SELECT CONCAT(
  "<metadata file='", INV_NUMBER, "_invoice.png'>\n",
  "  <invoiceNumber>", INV_NUMBER, "</invoiceNumber>\n",
  "  <invoiceDate>", INV_DATE, "</invoiceDate>\n",
  "  <totalAmount>", INV_AMOUNT, "</totalAmount>\n",
  "  <shipToName>", SHIP_NAME, "</shipToName>\n",
  "  <shipToAddress>", SHIP_ADDRESS, "</shipToAddress>\n",
  "  <shipToCity>", SHIP_CITY, "</shipToCity>\n",
  "  <shipToState>", SHIP_STATE, "</shipToState>\n",
  "  <shipToZipCode>", SHIP_ZIPCODE, "</shipToZipCode>\n",
  "  <shipToCountry>", SHIP_COUNTRY, "</shipToCountry>\n",
  "  <shipToPhone>", SHIP_PHONE, "</shipToPhone>\n",
  "</metadata>"
)
FROM INVOICES WHERE INV_NUMBER = XXXX
INTO OUTFILE /path/dir/XXXX_invoice.xml;

Another way to automatically create the XML file is to use a script, such as the following PHP code, which creates an XML file for every record in the INVOICE table whose CASE_START_DATE field is set to NULL (no data). The script uses the ftp_connect() and ftp_login() functions to connect to the FTP server and log in. It also uses mysqli to connect to a MySQL database.

<?php
$ftp_server = "ftp.example.com";
$ftp_username = "myuser";
$ftp_password = "p@ssW0rD";
$ftp_conn = ftp_connect($ftp_server) or die("Could not connect to $ftp_server");
$ftp_log  = ftp_login($ftp_conn, $ftp_username, $ftp_userpass);

$db = new mysqli('localhost', 'myuser', 'p@ssw0rd', 'BILLING_DATABASE');

if ($db->connect_errno > 0) {
    die('Unable to connect to database [' . $db->connect_error . ']');
}

$sql = "SELECT * FROM INVOICES WHERE CASE_START_DATE IS NULL";

if (!$result = $db->query($sql)) {
    die('There was an error running the query [' . $db->error . ']');
}

while ($row = $result->fetch_assoc()) {
   $xml = "<metadata file='{$row['INV_NUMBER']}_invoice.png'>\n" .
      "  <shipToName>{$row['SHIP_NAME']}</shipToName>\n" .
      "  <shipToAddress>{$row['SHIP_ADDRESS']}</shipToAddress>\n" .
      "  <shipToCity>{$row['SHIP_CITY']}</shipToCity>\n" .
      "  <shipToState>{$row['SHIP_STATE']}</shipToState>\n" .
      "  <shipToZipCode>{$row['SHIP_ZIPCODE']}</shipToZipCode>\n" .
      "  <shipToCountry>{$row['SHIP_COUNTRY']}</shipToCountry>\n" .
      "  <shipToPhone>{$row['SHIP_PHONE']}</shipToPhone>\n" .
      "  <totalAmount>{$row['INV_AMOUNT']}</totalAmount>\n" .
      "  <invoiceNumber>{$row['INV_NUMBER']}</invoiceNumber>\n" .
      "  <invoiceDate>{$row['INV_DATE']}</invoiceDate>\n" .
      "</metadata>";
   $xmlFilename = $row['INV_NUMBER'] . "_invoice.xml";
   file_put_contents("/local/path/" . $xmlFilename, $xml);

   //use the following code to upload the $xmlFilename to an FTP directory:
   if (ftp_put($ftp_conn, "/server/path/" . $xmlFilename, "/local/path/" . $xmlFilename, FTP_ASCII)) {
      echo "Successfully uploaded file '$xmlFilename'\n";
      //set the CASE_START_DATE to the current time:
      $db->query("UPDATE INVOICES SET CASE_START_DATE = NOW() WHERE INV_NUMBER = " . $row['INV_NUMBER']);
   }
   else {
      echo "Error uploading $xmlFilename\n";
   }
}

ftp_close($ftp_conn);
$db->close();
?>

Set the above script to be periodically executed as a cron job in Linux/UNIX or as a Scheduled Task in Windows.

Step 5 - Upload files: Now upload the XML file and the PNG image file to the monitored directory. Remember that both files must have the same filenames as in the image below:



Step 6 - Execute cron.php: Execute cron.php to start cases for each file uploaded to the directory monitored. In this case just one case will be started: