Purging of Archives
Definition
The purging of archives is used to clean databases permanently by physically deleting records from tables.
EndDefinition
- A purge is built around a parent query that accesses a set of standardized data from the database. It is used to perform an individual purge of specific objects based on the selected parent query.
- The data is extracted using filters.
- When enabled, it will run based on the frequency defined.
- You can configure a CSV file to be sent automatically in a notification email. This will list the deleted records.
- Purges can be stored in folders that are organized by function.
Reasons for purging archives
- Reduce the size of the database by deleting the oldest records. This will optimize processing speed and system performance.
- Refresh the list of records by deleting those that are no longer viewed.
Examples
Creation of a purge called Equipment discarded more than 3 years ago
- Execution frequency: Every December 31 at 7 pm
- CSV file: Display the type, model and asset
Type | Model | Asset |
---|---|---|
IT/Printers | LaserJet 1100 | 10221P |
IT/Printers | LaserJet 1100 | 10224C |
Enable a purge called Tickets over 3 years old
The purging of tickets over three years old will be performed on a continuous basis as long as the purge remains enabled.
Notes
- You can create or modify a purge as long as you are authorized to access the Administration > Purge > Purge Archives menu.
- Once they are enabled, purges will appear as scheduled tasks.
- Parent Query:
- Each purge is associated with a parent query selected from a list defined by EasyVista. You cannot add new queries.
- The queries are limited to the following main tables: Contracts, CIs, Incidents, Service Requests, History (Discovery), Equipment (Discovery), Employees, Events, Licenses, Equipment (Asset Management), Knowledge Base search results, Equipment Life Cycle.
- You can modify a query as long as you have the appropriate rights, and only in each purge.
See User Profiles > Edit rights and display rights.
- Purges do not take domains into account.
- When you purge data, any file attachments associated with the records will automatically be purged.
- CSV file:
- You can define its structure freely based on the information you want to save.
- Name of the CSV file = Purge name followed by the processing date/time.
- Configuration of the Archives folder:
- smoserver.ini file on the application server
- Schema: A_COMPANY
- Table: A_PARAMETERS
- Field to modify: SHARED_DOCUMENT_PATH: Specify the folder for storing the files
- When the purge is first run, the folder will automatically be created on the application server.
- The sender of the email and the sender's email address must be specified in the A_PARAMETERS table.
- Schema: A_COMPANY
- Table: A_PARAMETERS
- Fields to modify:
- MAILER_SCHEDULER: Specify the sender name, e.g. EVO_SCHEDULER
- MAILER_SCHEDULER_ADDR: Specify the sender's email address, e.g. scheduler@name_of_customer.com
- If the execution of a purge is not triggered, check that the scheduler is started and correctly configured.
Caution
- If you inadvertently perform a purge, you cannot undo the action. The EasyVista team will not resolve any issues related to the deletion of data and its impact on your platform.
- If you inadvertently purged the wrong records, you can create an integration model to retrieve the data from the CSV file.
- You should always check the execution of the purge by generating the CSV file in the Calendar screen to ensure that the records displayed in the file are the ones to be processed by the purge.
- Define a filter to ensure that only the relevant records are deleted.
- Remember to specify the dates in the filter. If you do not specify a date, all records in the database will be deleted.
- Before enabling a purge, remember to perform a full backup of the database.
- If you want purges to run regularly, you should schedule the backup before each purge.
- The purging of data will run as a background task, based on the specified frequency, as long as the purge is enabled. If you want to interrupt the purging of data, you must stop the task using the Disable wizard.
- Depending on the volume of data selected by the parent query, the processing time of the purge may be long. We recommend that you refrain from selecting too many fields to avoid complexifying the process.
- Do not confuse archiving with the purging of archives.
- Archiving is performed based on the end dates of different elements managed by Service Manager.
example Employee departure date, equipment discard date
- Archived elements remain available and can be viewed using the Archived filters.
- Archiving is performed based on the end dates of different elements managed by Service Manager.
Best Practice
- You should always check the execution of the purge by generating the CSV file in the Calendar screen to ensure that the records displayed in the file are the ones to be processed by the purge.
- Store the purged data in a CSV file. This enables you to keep a history of your data.
- If you want to save information whose fields are not available in the CSV file, you should add them manually to the parent query.
- You should only purge archived elements, i.e. records displayed using the Archived filter.
- To purge integration logs, use the dedicated parent query called Integration log.
See the procedure.
- To purge temp files and log files, see the rules for purging that describe the resources to be purged based on operational constraints.
- Avoid scheduling purges whose frequency is based on a specific time. We recommend you run the processing yearly.
- If the running of purge B depends on the completion of purge A whose processing time is unknown (due to large data volumes, access time, etc.), we recommend that you schedule the tasks in a sequence instead of defining a specific time for running purge B.
Menu access
Administration > Purge > Purge Archives
Screens description
Main Information
Name: Name of the purge.
Comment: Comment field that describes the purge.
Folder Name: Name of the folder where the purge is saved.
Parent Query: Name of the parent query used to configure the purge.
- The fields available are parent query fields and other fields manually added to Service Manager internal reporting tools, e.g. trends, alerts, reports, and other purges.
Filter
Note: Except for the Integration log parent query
Filter used to extract the data to be included in the purge. See Visual filter editor.
- All fields in the purge's parent query as well as other manually added fields will be available for the filter.
Define the CSV Format
Order: Order of columns in the CSV file. See the procedure.
Display: Columns to be displayed in the CSV file.
Field: List of fields from the parent query associated with the purge and other manually added fields.
Calendar
Note: Universal Time (UTC): Current date and time in Service Manager, expressed in Universal Time (UT).
Frequency: Used to define the frequency for running the purge.
- The frequency will be applicable after the date specified in the Next Execution Date field.
- The parameters depend on the selected frequency.
- Every [N] Months / Days / At (Hour UT): Define the relevant frequency using Universal Time (UT).
- Depends on: Used to sequence the running of two purges successively. Select the purge whose processing precedes the execution of your purge from the drop-down list.
Next Execution Date: (Note: Only if the frequency is not Depends on): Date and time at which the purge should next run.
- The date initializes the start of the frequency.
example Frequency: Every seven days at 7 am; the next execution date is a Saturday ==> The purge will run every Saturday
- You can also update the date using the Enable wizard.
Warn
Recipients (Employees) / Recipients (Groups): List of recipients of the notification email.
- These recipients must be listed in the Employee Directory or Group Directory.
- You must select one or more recipients, i.e. employee or group.
- Each recipient must be separated using the semicolon (;).
Title: Subject of the notification email sent to recipients.
Message: Body of the notification email sent to recipients.
Procedures and Wizards
How to create a purge
Step 1: Create the new purge
1. Select Administration > Purge > Purge Archives in the menu.
2. Click + New.
Step 2: Specify the main information on the purge
1. Enter the information on the purge.
2. Select the parent query.
- You can add fields to the parent query.
- Click
next to the field.
The list of fields from the parent query of the purge and other fields manually added to Service Manager internal reporting tools and other purges will appear. - Add the fields you want.
See How to modify a parent query
- Click
3. Click Next.
Step 3: Define the conditions for the filter used by the purge
Note: Except for purges using the Integration log parent query
1. Define the filter conditions. See Visual filter editor.
- All fields in the purge's parent query as well as other manually added fields will be available.
2. Click Next.
Step 4: Format the CSV file to be generated by the purge
1. Define the order of the columns in the file.
- Select the Order box next to the field to be moved.
- Click Insert Here next to the field above which you want to move the selected field.
2. Specify the options depending on the type of field.
- Only for tree fields that are not in the last level: Indicate the first and last level of the hierarchy to be displayed.
- The value 0 is used to display all grouping levels in the hierarchy.
example Display the tree field, Location = North America/USA/New York/Manhattan/Broadway
- Start = 0, End = 0 ==> North America/USA/New York/Manhattan/Broadway
- Start = 1, End = 3 ==> USA/New York/Manhattan
- Start = 4, End = 5 ==> Manhattan/Broadway
- Location (last level) = Broadway
- The value 0 is used to display all grouping levels in the hierarchy.
3. Select the fields to be displayed in the file.
Step 5: Define the frequency for running the purge
1. Specify the frequency of the purge.
2. Click Next.
Step 6: Complete the notification email
1. Specify the recipients of the notification email.
Step 7: Save the purge
1. Click Finish.
The list of purges will be refreshed.
Step 8: Enable the new purge
1. Select the purge.
2. Run the Enable wizard.
3. Enter the first execution date, click Next and click Finish.
The CSV file will display the records that will be purged when the purge is next run.
4. Check that the purge is enabled.
- Select Administration > Scheduled Tasks > Purge Archives in the menu.
- Check that the purge appears in the list of scheduled tasks.
How to create a purge using a template
Step 1: Select the purge
1. Display the list of purges.
2. Click next to the template.
The properties window of the purge will appear.
Step 2: Create the new purge
1. Modify the relevant information in the template purge.
2. Click Save as to avoid overwriting the template purge. Give a name to the new purge.
How to check if a purge is enabled or disabled
Step 1: Check if the purge is enabled or disabled
1. Select Administration > Purge > Purge Archives in the menu.
2. Display the List view.
3. Look at the Enabled box for the relevant purge.
- If the box is checked, the purge is enabled.
- If the box is not checked, the purge is disabled.
Step 2: Check that the corresponding scheduled task exists
1. Select Administration > Scheduled Tasks > Purge Archives in the menu.
2. Check the presence of the purge.
- If the purge appears in the list, it is scheduled to run based on the frequency defined.
- If the purge does not appear in the list, it is not scheduled to run.
Wizards
Rules for purging temp files and log files
Server with Service Manager and Service Apps
Resources to be purged based on operational constraints
Server | Folder | Rule | ||
---|---|---|---|---|
Application server | REP_APP\tools\SERVERS\mssql\log | All XML files whose last modification date exceeds one month | ||
Application server | REP_APP\tools\servers\EASYVISTASERVER\log | All XML files whose last modification date exceeds one month | ||
Application server | REP_APP\tools\smobackoffice\log | All LOG files whose last modification date exceeds one month | ||
Application server | REP_APP\tools\smomail\log | All LOG files whose last modification date exceeds one month | ||
Application server | REP_APP\tools\SERVERS\smoauth\log | All LOG files whose last modification date exceeds one month | ||
Web server | REP_LIN_LOG/apache | All files whose last modification date exceeds one month | ||
Web server | PHP.INI file in the folder defined by the session.save_path variable or the REP_SESS_PHP variable | All files whose last modification date exceeds seven days | ||
Web server | REP_LIN_LOG/php | All PHP logs whose last modification date exceeds one month | ||
Web server | REP_LIN_LOG/REST | All logs with REST calls whose last modification date exceeds one month | ||
Web server | REP_LIN_LOG/SOAP | All logs with outgoing SOAP calls whose last modification date exceeds one month | ||
Web server |
|
All files whose last modification date exceeds seven days | ||
Web server | /easyvista/argo | All Service Apps logs whose last modification date exceeds seven days |
Examples of purge scripts on Windows servers
In each example, replace the folder displayed below in yellow by the folders to be purged.
- PowerShell v3
# One script per Log folder would be required.
# MSSQL Log Folder files older than 30 days
Get-ChildItem –Path "C:\EasyVista\tools\servers\MSSQL\Log" –Recurse | Where-Object CreationTime –lt (Get-Date).AddDays(-30) | Remove-Item
- DOS Batch file
forfiles -p "C:\EasyVista\tools\servers\MSSQL\Log" -s -m *.xml -d 30 -c "cmd /C del @path"
Dedicated Service Apps server
Resources to be purged based on operational constraints
Server | Folder | Rule | ||
---|---|---|---|---|
Web server | REP_LIN_LOG/apache | All Service Apps logs whose last modification date exceeds seven days | ||
Web server | PHP.INI file in the folder defined by the session.save_path variable or the REP_SESS_PHP variable | All files whose last modification date exceeds seven days | ||
Web server | REP_LIN_LOG/php | All PHP logs whose last modification date exceeds one month |
Configuration of the LOG.INI file
On the Service Manager GUI servers, log files are automatically purged by the application. You can configure the purge frequency in the LOG.INI file.
expire = 7 days
; Indicate what you want to delete
; root => /argo/<folder logs>: delete only this folder but not subfolders
; tenant => actual tenant of user is logged
; all => root folder and all the tenants that exist
folder_delete = "all"
; log_cleanup_frequency
; Keep this value to 100 unless necessary
Use case: Creation of a purge Delete attachments from closed tickets
The purge will delete documents associated to closed tickets both in the database and on the web server thanks to two dedicated parent requests. This allows to save disk space on the web server.
Implementation:
- Select the dedicated parent query.
- Use the dedicated parent request Documents Operation to delete documents associated to closed incidents.
- Use the dedicated parent request Documents Transition to delete documents associated to closed requests.
- Extract the data by specifying the filter. Indicate the period you want to delete the documents associated to closed incidents.
- Purge execution: Indicate the frequency you want.