Date & Time Logic for Transferring Log Files with MOVEit Automation
If you are a regular reader of our blogs, you would know that we often like to highlight any particular technical achievements or solutions we developed, so that knowledge can be shared amongst our customers and visitors alike. This week we not only have a clever solution - for transferring log files using MOVEit Automation - to showcase but that solution has been submitted by one of our customers.
The Brief
Early last week we received an query from one of our customers asking for our opinion on a workflow he would like to develop in MOVEit Automation. Normally we would be very keen to assist - maybe even help to develop the workflow ourselves in our demo labs - but in this case, he was very insistent that as long as we thought it was possible, that was all that he needed to investigate for himself.
Each day, throughout the day, an application in the customer's network would output its activity into a log file, which the customer needed the content of to be transferred to another location frequently.
At the end of a day, the application would stop writing to that log file and would create a new log file for the following day. The customer want to transfer the previous days log file to an archiving solution which would have sufficient space for long terms storage.
The Solution
We were very certain that this would be achievable using MOVEit Automation macros. Dynamic placeholders for variables such as times and dates.
An example of this would be [yyyy]-[mm]-[dd].
Which would output the date at the time of executing that macro, with hyphens between the year, month and day values.
In addition to this, MOVEit Automation permits the use of a "+" or "-" on any of those macros to denote either an increment of a decrement of its value. For example [dd-] effectively output the day date value of yesterday. Something we would be able to make use of when archiving the previous days log file.
You can review the official MOVEit Automation documentation regarding time and date based macros here: https://docs.ipswitch.com/MOVEit/Automation2020/Help/AdminConsole/en/index.htm#27813.htm
The Outcome
Later that week we received an update from our customer who come to the same conclusion as we had, regarding the use of date and time based macros; and had successfully put together a workflow which was achieving the desired outcome.
- On a schedule, the workflow looks at all files on the configured host. This is indicated by the "*.*" filename and file type filter.
- The workflow then invokes a loop for each file encountered and test each file to see whether it is named with today's date and has a file type and suffix of ".log". There also includes an AND operator to also test that the log file has been written to in the last hour. Not having written to the file in this time could indicate that it is not the active log file.
- If the active log file for that date is found, it is transferred to a centralised log location. But crucially, the file is not deleted nor renamed so that the application can continue to populate the same log file.
- If, a log file is found for the previous day using the "[dd-]" macro we discussed above, the file is moved to the archive location and the original is deleted. Which should result in only that days log file being present on the source at any time.
This workflow achieved the objective that our customer set out to achieve; and he was able to configure it himself with the knowledge he had acquired through his time using MOVEit Automation. Maybe we can take some credit for having helped in him the past... 😀
If you would like to learn more about MOVEit Automation; automated file transfers or would like to talk about how how we might be able to help your organisation implement a managed file transfer solution. You can book a call with us today by clicking here.
|
