WarningSaveDatabase

EndWarningSaveDatabase

Data integration is performed using an integration model. It can be run after a preimport processing that will first prepare the data import, e.g. tables purged, data normalized. 

Automating a data integration consists of scheduling the sequencing of the preimport processing followed by the integration model. You can use different tools to automate the processing.

Conditions for accessing the tools for automating integration

• SaaS-based customers:
  • You cannot access the application server. You cannot run the smoBackOfficeClient and smoIntegration tools. You cannot run any batches to run these tools.
  • You can access the \preimport folder via ftps/ftp. This enables you to store data files and access configuration files, SQL scripts, log files (\log subfolder) and temp files for data extraction (\temp subfolder).
  • You can create new subfolders in the \preimport folder based on your requirements.

• On-premises customers:
  • You can access the smoBackOfficeClient and smoIntegration tools on your application server.
  • You can access the \preimport folder freely and organize it based on your requirements.

Notes

• You can use an external scheduler such as the Windows Task Scheduler or a third-party application to schedule an integration. This enables you to:
  • Leverage more advanced scheduling options than those available for the preimport and integration model performed respectively by the Service Manager smoBackOffice and smoScheduler schedulers.
  • Centralize all scheduled tasks within a single tool.
  • Automatically sequence the running of an integration model at the end of the preimport processing.

• The smoBackOfficeClient and smoIntegration tools are available in the <easyvista>\tools folder (Note: <easyvista> identifies the EasyVista folder on the application server. You can also download them from the documentation Web page for each tool.

Caution

• You should always perform a backup of the Service Manager database prior to running an integration model. This ensures that you can restore the database if there are too many problems or errors.
• Integration models are unable to perform data integration on an incremental basis. All of the data in the CSV file will be imported into the Service Manager database, regardless of whether or not the data was modified after the last integration was performed.

Best Practice

• You should first run integration models in the test database before running them in the production database.
• If there are too many errors at the end of the integration processing, restore the Service Manager database using the backup. Correct the source data and run the integration again.

Procedures

How to automate an integration using the Service Manager schedulers

Step 1: Test the preimport using the smoBackOffice scheduler.

1. Specify the INI configuration files to be run by the smoBackOffice scheduler.

• Go to the folder called <easyvista>\tools\smobackoffice\QUERIES\SQL\PREIMPORT.
• Open the PREIMPORT_SQL.sql file.
• Go to the [PREIMPORT] section.
• Specify the list of INI configuration files containing the SQL scripts to be run.
  Replace <PreimportFileName.ini> with the names of the INI configuration files to be run by the preimport processing.

2. Force the execution of the INI configuration files containing the SQL scripts to be run by the preimport processing.  See the description.

• Go to the folder called <easyvista>\tools\smoBackOffice\Preimport\.
  Replace <easyvista> with the easyvista folder on your application server.

• Open each INI configuration file.
• Go to the [DEFINITION] section and delete the two keys, NLASTRUN and SLASTRUN that store the last execution date of the file.
  The keys will automatically be regenerated when the file is next run. Deleting them enables you to force the execution.

 The preimport will be placed in the smoBackOffice scheduler queue. It will run after the tasks currently being processed.

3. Once the preimport processing is completed, check the contents of the tables loaded by the processing and correct the errors, if any.

Step 2: Test the integration model using the smoScheduler scheduler.

1. Run the integration model using the smoScheduler scheduler.

• Select Administration > Integration > Integration Models in the menu.
• Click  next to the template.
• Proceed to the last step of the integration model.
• Click Execute.
   The integration model will be placed in the scheduler queue. It will run after the tasks currently being processed.

2. Once the integration processing is completed, check the data integrated in Service Manager and correct the errors, if any.

Step 3: Schedule the integration using the Service Manager schedulers.

1. Schedule the running of the preimport SQL scripts.

• Open each INI configuration file.
• Go to the [DEFINITION] section.
  • Use the ACTIVE key to indicate that the file must be run.
  • Use the FREQUENCY key to define the frequency (in number of days) for running the file.
  • Use the BEGINTIME key to specify the start time (in HH:NN:SS format) for running the file.
  • Delete the two keys, NLASTRUN and SLASTRUN that store the last execution date of the file. This will force the execution of the file.

2. Schedule the execution of the integration model.

• Select Administration > Integration > Integration Models in the menu.
• Click  next to the template.
• Select the Enabled box in the window for scheduling the task.
• Specify the scheduling information required for running the integration model.  See the description.

How to automate an integration using an external scheduler

Step 1: Test the preimport using the smoBackOfficeClient tool.

1. Specify that the INI configuration files should be run only by the smoBackOfficeClient tool. Disable their execution by the Service Manager smoBackOffice scheduler.

• Go to the folder called <easyvista>\tools\smobackoffice\QUERIES\SQL\PREIMPORT.
• Open the PREIMPORT_SQL.sql file.
• Comment out the lines for executing the INI files. 

2. Force the execution of the INI configuration files containing the SQL scripts to be run by the preimport processing.  See the description of INI files.

• Go to the folder called <easyvista>\tools\smobackoffice\QUERIES\SQL\PREIMPORT.
  Replace <easyvista> with the easyvista folder on your application server.

• Open each INI configuration file.
• Go to the [DEFINITION] section and delete the two keys, NLASTRUN and SLASTRUN that store the last execution date of the file.
  The keys will automatically be regenerated when the file is next run. Deleting them enables you to force the execution.

3. Run the preimport processing using the smoBackOfficeClient tool.

• Create a batch file.

• Insert a command line for each INI configuration file to be run using the smoBackOfficeClient tool.
  Replace <PreimportFileName.ini> with the name of the INI configuration file run by the preimport processing.

• Run the batch file using your external scheduler.

4. Once the preimport processing is completed, check the contents of the tables loaded by the processing and correct the errors, if any.
 

Step 2: Test the integration model using the smoIntegration tool.

1. Specify that the integration model should be run only by the smoIntegration tool. Disable its execution by the smoScheduler scheduler.

• Select Administration > Integration > Integration Models in the menu.
• Click  next to the template.
• Unselect the Enabled box in the window for scheduling the task. 

2. Run the integration model using the smoIntegration tool.

• Create a batch file.

• Insert a command line to run the integration model using the smoIntegration tool.
  Replace <IntegrationModel_EnglishName> with the English name of the integration model.

• Run the batch file using your external scheduler.

3. Once the integration processing is completed, check the data integrated in Service Manager and correct the errors, if any.

Step 3: Schedule the integration model using the external scheduler.

1. Create a batch file.

2. Insert command lines to run the INI configuration files using the smoBackOfficeClient tool.

3. Insert a command line to run the integration model using the smoIntegration tool.

4. Define the parameters for scheduling the batch file using your external scheduler.

Note: You do not need to allow for an adequate interval between the running of the preimport processing and the integration model using the smoBackOfficeClient tool. The smoIntegration tool will only run once the smoBackOfficeClient tool has stopped.