Triggering
In general, the first place where something goes wrong is the actual triggering of an action. Depending on the system that you are working with, this may be an event, job, task or similar. Actions are generally triggered by an event matching a rule – for a scheduled action this is a time event corresponding to a specific time. More commonly however, the event will be the arrival of a file and the rule will be a filename or folder match. Common errors in this area include conflicting rules in multiple actions and files arriving just after a scheduled transfer.
Sources and Targets
Broadly speaking, the most common role of automation software is to move files from one place to another – unfortunately, this is where things most often go awry. Here are some common problem that occur.Firewall incorrectly configured – affecting both inbound and outbound traffic. Caused by either some or none of the required ports being opened, or even incorrect NATing of the automation components IP address
Whitelisting and Blacklisting – Unfortunately as the automation administrator, you don’t necessarily have a view of this. It is however worthwhile being able to validate that your IP address hasn’t changed unexpectedly.
Password, Key or Certificate Expiry – There is always a play-off between security and operability, but invariably in more secure environments non-expiring certificates, keys or passwords are disallowed. Be aware that many secure transfer servers will not confirm that this is the cause of the problem, so it may not be immediately obvious. You should also note that repeated failures may result in IP blacklisting or a locked account.
Connectivity – The internet is a great place to get lost in and we all expect to have occasional issues reaching certain destinations. The same can often hold true inside your own network however. Remember that you may sometimes need to flush a DNS cache in order to make the right connections (especially true after a post maintenance DNS change). For those connections that just can’t be made (whether internal or external), you will need to put a plan in place to reconnect when the target server becomes available again.
Space – Maybe not the final frontier, but often the last straw. Many platforms place a hard limit on the maximum permissible file size; some network administrators have a hard limit on how big a file they allow through their network (especially during peak times). And of course, inevitably there will always be a disk running out of space somewhere.
Programs and Scripts
Most MFT systems will provide you with the opportunity to execute a program or script at some point during the transfer process. This allows you perform some basic transformation or processing of a file during transit. Invariably the MFT software will not by default provide all of the debug information that you need, so be prepared to write in extra logging, or redirect STDERR to STDOUT and capture it somewhere.
Notifications
Often, MFT software cannot/will not report back failures to send email notifications as they potentially may be stopped somewhere outside of the MFT system. To counter this it is common to send yourself a daily status email, the arrival of which demonstrates that at least some emails are leaving the MFT system. If the MFT software does not provide the opportunity to test emails, try logging on to the server and running telnet on port 25 to test SMTP (Windows); on Linux systems you may use sendmail instead.
Visibility
Managed File Transfer solutions – or MFT – provide excellent visibility of transfers. At a basic level, that might include email message alerts when a file has been delivered. BUT, this relies on you noticing you haven’t received the email.These systems record events between the server and client, so – with the right module or add ons – you can usually get a detailed level of reporting. This real-time transfer monitoring allows you to keep an eye on the most important transfers as they happen.
A good MFT system will provide the following:
- Real time status of your servers and sites
- Views of transfers in and out of your system
- A dashboard giving key system statistics
- The run history of event rules configured on your system
With many systems you can design customisable reports showing transfers, which you can then export to save or share. And – because prevention is better than cure – the IT department can uncover factors which may lead to future errors, such as connection failures, firewall misconfigurations, and data corruption.How you achieve this will depend on your MFT solution and other monitoring systems in your environment. Ideally they will interact, but if this isn’t possible, you could use SNMP traps, or write to a Syslog server. Many monitoring systems read Windows event logs to detect errors, and happily most MFT systems allow this directly. Alternatively you could use a database as an intermediary location for storing monitoring events. Our technical consultants provide professional services to help you if you need it.
Never miss an SLA again
With a good MFT system, you can build a rule to check if a particular file has been transferred by a certain time. The system will generate an email, alerting the administrator or another specified user, if the transfer has failed. Setting the rule to check before the file is needed gives you advanced warning. So if – for example – an order isn’t sent or payment not made, you have time to contact the sender and address any difficulties before the SLA is breached.
These rules can be set for file names, file sizes or specific senders. You can also track the number of files sent. For example – you can check that at least three files of 100KB or more were uploaded from a specific group of users, with a file name starting ‘finance’.
Why the transfer failed?
The more sophisticated MFT solutions on the market will allow you to pinpoint which of several steps in an event rule has failed. This means you can quickly identify the problem, without the cumbersome process of analysing logs. Some solutions will display high-level information in a dashboard view, which can be customised to highlight errors, such as failed logins or PCI compliance. You may be able to display other KPIs as well.
Managed File Transfer Buyer’s Guide – Start your research here
What is MFT and how does it enhance security, productivity, compliance and visibility? This free guide is compiled from 16+ years’ experience. It includes definition, product features & use cases.