Open Source Managed File Transfer Software: Current & Past Options

Open Source Managed File Transfer Software

Current & Past Options [Updated January 2019]

 

If you are looking for an open source Managed File Transfer solution, this blog is for you. It’s written by Pro2col Managing Director James Lewis, who is a self-confessed file transfer geek with over two decades’ experience working with file transfer technologies and vendors. He’s been following open source projects for many years and regularly updates this blog as new options become available.

 

I originally wrote this post back in July 2012, after a number of requests for open source Managed File Transfer from potential customers. They’d found us via our website, which clearly promoted a wide variety of commercial products, with no reference to open source, however they were only interested in open source options.

Free clearly doesn’t pay the bills, but being a bit of an industry geek, I decided to do the research and find out what was available. I identified a couple of SourceForge projects, which I’ve been following over the years.

For the record, I’m a fan of open source. Our previous Technical Director was a thrifty Northerner, who converted me. We ran various elements of the business very successfully on open source projects. Our support ticketing system was based on OTRS and our monitoring system used Nagios. Both were mature applications, widely used in the open source community and provided considerable functionality at a price that suited!

 

open_source

When it comes to open source Managed File Transfer though, the landscape is patchy at best. Open source FTP servers exist in abundance and can provide the landing point for incoming and outgoing files, but open source Managed File Transfer projects appear to be scarce. I suspect that this is because Managed File Transfer hasn’t been one of those technologies that every company deploys.

Recent changes in EU regulation in the form of GDPR, also impacts this space. GDPR legislation requires increased levels of security, audit trails and reporting on any transfers involving personal data. (See our blog posts Encryption at rest for GDPR and Where is your data going and why?)

Open source Managed File Transfer trends

My research over the past 6-7 years has highlighted a few trends that don’t bode well for open source Managed File Transfer projects, and may be a reflection of the wider open source landscape. In general, they have slotted into the following categories:

  • The company is acquired and the free option is removed. Some or all of the functionality is incorporated into a commercial offering.
  • The part-time developer gets a contract or new job and the project gets shelved.
  • The project is labeled as Managed File Transfer, but doesn’t contain the key functions of all commercial offerings in the market.
  • Commercial vendors offer a pseudo open source product with limited functionality and encourage migration to their commercial solution.

Genuine open source projects

There are just two genuine projects that I’ve found so far. I couldn’t recommend either, as I’ve not invested any time personally or asked our technical consultants to review them, however they have ongoing development and support packages.

Yade – an open source project, previously going under the name of SOSFTP. This project has been around since at least 2012. SOS Berlin lists a number of customers on their website and provides support and consulting packages. Currently my favourite option, based upon the length of time they’ve been around.

WAARP – a relative newcomer to the market but it looks to have all the basics covered. It also provides commercial support options and their website provides visibility of who is involved in the project. This is certainly one to watch.

If you’re a user of either Yade or WAARP, I’d be interested in hearing from you. I’m keen to understand how complete the project is, how responsive the development team are and what your experience of their support offering has been like.

Free products from commercial vendors

As I’ve already mentioned, these tend to be products with limited functionality, which will ultimately encourage migration to a commercial solution.

Coviant Diplomat OpenPGP Community Edition – This is a free OpenPGP tool to automate PGP encryption and decryption.

Coviant Diplomat Cloud Storage Community Edition – A free tool to PGP-encrypt files being transferred to cloud storage sites.

HelpSystems Free FTP Server – this is a free edition of GoAnywhere MFT, with administrator dashboard, extensive security, audit reports and more.

FTP Voyager – A free GUI FTP client, FTPS client and SFTP client software for Windows. The main interface is similar to dozens of other FTP clients, but it also includes powerful scheduling utilities and synchronisation utilities for free.

Some open source MFT that is no more

These are some of the open source projects that I have been following, which are no longer available:

Policy Patrol by OPSWAT – has now become Metadefender Email Security. The Managed File Transfer element of the open source project appears to have been shut down.

ShieldShare by BlockMaster – now appears to be part of the DataLocker stable but their focus is on encrypted storage. It’s unclear whether the product was acquired for the encryption capabilities. Project shut down.

Appterra – their open source supply chain integration platform with Managed File Transfer capabilities was acquired by Descartes. The open source project has been shut down.

DivConq MFT – a SourceForge project that looked promising but the developers seem to have ended the project and the associated website has closed down.

Karonte – positioned as an open source Managed File Transfer solution but it doesn’t have the basic functionality we consider critical for MFT.

In conclusion

Whilst the open source marketplace can be a fantastic resource for some business applications, Managed File Transfer isn’t currently one of them. If your business is in the tech space or you’ve got an extensive development and technical team, then open source Managed File Transfer may be a viable option for you.

However, Managed File Transfer is mission critical for almost all of our customers. Many come to us looking to mitigate the security and compliance risks associated with supporting a bespoke or homegrown solution. Unmanaged in-house scripts in particular are one of the biggest risks to an organisation’s GDPR compliance. Additionally, as developers and contractors move on, companies get stuck with a solution without documentation, no training and no one to make changes or fix faults.

If you’d like to discuss your Managed File Transfer requirements and the impact of GDPR, I’d be pleased to talk them through with you. You may be surprised at how much bang you get for your buck these days in commercially available products. You can contact us via the web form, or call 0333 123 1240.

Need further expertise? Download our FREE resources

Choosing the right Managed File Transfer protocol

This blog post answers your questions about Managed File Transfer protocols. Which are the most widely used file transfer delivery protocols? Which should you be using and how do you identify which solution uses which protocols?

A protocol is the set of rules that determines how files are transferred from one computer to another, through a network. That might be an internal network (from one computer to another within the same network) or more commonly a Wide Area Network such as the internet.

The nature of your data and its destination will determine the right protocol for the transfer. For example, personally identifiable data and credit card information will need a secure protocol.

BASIC PROTOCOLS

FTP
(File transfer protocol)

How can it be used?

Upload/download files
Rename and delete files
Create/delete folders
Execute custom commands on server
Check integrity of files

When can it not be used?

X Secure data at rest
X Secure data in transit (FTPS can)
X Work over just one firewall port
X Provide strong authentication

FTPS
(“FTP Secured” using SSL)

 Secure data in transit
Upload/download files
Rename and delete files
Create/delete folders
Execute custom commands on server
Check integrity of files
 Provide strong authentication

X Secure data at rest
X Work over just one firewall port

SFTP
(“Secure FTP” using SSH)

Secure data in transit
Upload/download files
Rename and delete files
Create/delete folders
 Provide strong authentication
Work over just one firewall port (22)

X Secure data at rest
X Check integrity of files
X Execute custom commands on server

SCP
(“Secure CoPy”)

Secure data in transit
Upload/download files
Work over just one firewall port (22)
 Provide strong authentication

X Rename and delete files
X Create/delete folders
X Check integrity of files
X Execute custom commands on server

ADVANCED PROTOCOLS

HTTP
(HyperText Transfer Protocol)

HTTP CAN ALWAYS
Download files
Work over one firewall port (80)

HTTP CAN SOMETIMES
Upload files
Rename and delete files
Create/delete folders
Execute custom commands on server
Check integrity of files

Secure data at rest
X Secure data in transit (HTTPS can)
X Provide strong authentication

HTTPS
(HTTP Secured with SSL)

HTTPS CAN ALWAYS
✓ Download files
✓ Work over one firewall port (443)
✓ Secure data in transit

HTTPS CAN SOMETIMES
– Upload files
– Rename and delete files
– Create/delete folders
– Execute custom commands on server
– Check integrity of files
Provide strong authentication

Secure data at rest

WebDav

WEBDAV CAN ALWAYS
Download/upload files
✓ Rename and delete files
✓ Create/delete folders
✓ Work over one firewall port (443)
✓ Secure data in transit

WEBDAV CAN SOMETIMES
 Provide strong authentication

Secure data at rest
X Execute custom commands on server
X Check integrity of files

EMAIL PROTOCOLS

SMTP
To send mail

Push files as attachments
✓ Be secured with SSL/TLS
✓ Often uses ports 25, 465 or 587

Pull files from other servers

POP3
To get mail

Pull files from servers as attachments
Delete original email from servers
Be secured with SSL/TLS
Often uses port 995

X Push files as attachments
X Synchronize email folder contents
X Not supported in all email environments
POP3 is becoming obsolete

IMAP
To get mail and sync mail folders

Pull files from servers as attachments
✓ Delete original email from servers
✓ Synchronize email folder contents
✓ Be secured with SSL/TLS
Often uses port 993

Push files as attachments
X Be trusted if its key mailbox is also accessed interactively

Which Managed File Transfer protocol?

Guidance on what constitutes a secure protocol will change, adapting to stay one step ahead of cybercrime. That’s why it’s important to choose a vendor that releases regular product updates. With Pro2col’s free Managed File Transfer comparison service, you submit your requirements via a questionnaire. Our experts compare them against the different solutions and recommend the right product for you. Our experts consistently review the marketplace and only select credible solutions from credible vendors, who provide excellent support and regular software updates.

Interested in a file transfer solution?

Managed File Transfer software comparison

Managed File Transfer software comparison 

[Updated – September 2018]

Are you doing a Managed File Transfer software comparison? With over forty products on the market, where do you start?

Globalscape EFT, HelpSystems GoAnywhere, Cornerstone from SRT, Ipswitch MOVEit and Cleo Harmony are all excellent Managed File Transfer products that we recommend to customers. But even from this shortlist, which is the right one for your organisation?

Every business has a unique set of requirements and each solution delivers its feature-set differently. There are so many factors that will determine if your implementation is a success. Sourcing the wrong product will cost you more in the long run.

Pro2col’s free comparison service identifies the right solution for your needs and budget. You complete a series of questions about your current and future business requirements, and receive a bespoke report from our technical consultants, recommending the best solution for you.


WHY REQUEST A MANAGED FILE TRANSFER SOFTWARE COMPARISON?

SAVE TIME

You will save weeks of research time by completing this questionnaire, making use of our technical consultants’ knowledge and expertise.

EXPERTISE

Pro2col experts have been delivering secure file transfer solutions since 2004 across 30 countries. Each technical consultant has a minimum of seven years’ experience working with this niche technology. We are also providers and developers of the only file transfer certification, CFTP.

INDEPENDENT

Software vendors will want to sell you their product, but our technical experts independently analyse the best solution for your unique requirements. They consistently review the marketplace and only select credible solutions from credible vendors, who provide excellent support and regular software updates.

FULLY SCOPE YOUR REQUIREMENTS

Your internal processes and current / future business requirements will determine which solution is the best fit. That is because different software differs in how it delivers the same set of features – the level of detail makes all the difference. Our free Managed File Transfer comparison service asks you the right questions to recommend an exact fit for your organisation, making sure your implementation is a success.

The questionnaire prompts you to consider the following criteria: Key infrastructure questions you need to think about when comparing MFT software; how your solution will be impacted by other policies within the business; requirements for automated transfers
Transfer protocols, which will depend on the security requirements for the data (eg: personally identifiable data, credit card information);
which cloud services you need to connect to; which standards you need to comply with (eg: GDPR, PCI DSS).

How can I find out if a file transfer failed?

How can I find out if a file transfer failed?

Data transfers underpin many operations within an organisation, so it’s important to know if a file is not delivered. Your customer SLAs may depend on it. Read on to find out how an MFT solution can alert you when a transfer fails and identify the reasons why.

Organisations transfer data all the time. That might be financial information in the form of invoices, orders and BACs files, or other operational transfers received through a website or shared between internal offices.

If an automated transfer fails it can disrupt business operations and risk breaching service level agreements (SLA) you have in place for that activity. Unfortunately, for many organisations, the first indication that a transfer has not happened, is a call from a user missing the file. By then, it’s usually too late.

That’s why you need good visibility of all your transfers and
alerts set up to notify you if they fail.

Visibility

Managed File Transfer solutions – or MFT – provide excellent visibility of transfers. At a basic level, that might include email message alerts when a file has been delivered. BUT, this relies on you noticing you haven’t received the email.

These systems record events between the server and client, so – with the right module or add ons – you can usually get a detailed level of reporting. This real-time transfer monitoring allows you to keep an eye on the most important transfers as they happen.

A good MFT system will provide the following:

  • Real time status of your servers and sites
  • Views of transfers in and out of your system
  • A dashboard giving key system statistics
  • The run history of event rules configured on your system

With many systems you can design customisable reports showing transfers, which you can then export to save or share. And – because prevention is better than cure – the IT department can uncover factors which may lead to future errors, such as connection failures, firewall misconfigurations, and data corruption.

How you achieve this will depend on your MFT solution and other monitoring systems in your environment. Ideally they will interact, but if this isn’t possible, you could use SNMP traps, or write to a Syslog server. Many monitoring systems read Windows event logs to detect errors, and happily most MFT systems allow this directly. Alternatively you could use a database as an intermediary location for storing monitoring events. Our technical consultants provide professional services to help you if you need it.

Never miss an SLA again

With a good MFT system, you can build a rule to check if a particular file has been transferred by a certain time. The system will generate an email, alerting the administrator or another specified user, if the transfer has failed.

Setting the rule to check before the file is needed gives you advanced warning. So if – for example – an order isn’t sent or payment not made, you have time to contact the sender and address any difficulties before the SLA is breached.

These rules can be set for file names, file sizes or specific senders. You can also track the number of files sent. For example – you can check that at least three files of 100KB or more were uploaded from a specific group of users, with a file name starting ‘finance’.

Why did the transfer fail?

The more sophisticated MFT solutions on the market will allow you to pinpoint which of several steps in an event rule has failed. This means you can quickly identify the problem, without the cumbersome process of analysing logs. Some solutions will display high-level information in a dashboard view, which can be customised to highlight errors, such as failed logins or PCI compliance. You may be able to display other KPIs as well.

Next steps for your organisation

  • You already use an MFT solution
    Our technical consultants can advise on how to achieve this level of visibility.
    Get in touch
  • You don’t have an MFT solution
    Our free download, Do I need MFT?, will identify if this is the right choice of technology for your organisation.
    Do I need MFT? DOWNLOAD
  • You are ready to compare solutions on the market
    If you know MFT is right for your organisation, our free comparison service will identify the best solution for your needs and budget.
    Request MFT comparison

Interested in a file transfer solution?

What do the new SSL and early TLS requirements mean for my file transfer solution?

What do the new SSL and early TLS requirements mean for my file transfer solution?

PCI DSS is the security standard for processing and storing credit card information. From 30th June 2018, organisations can no longer use SSL and early TLS to meet the PCI DSS standard. This blog post will remind you of the requirements and what this means for your file transfer solution.

Earlier this year we reminded you that Secure Sockets Layer (SSL) and early Transport Layer Security (TLS) are no longer considered secure protocols. It’s because of the growing number of attacks and vulnerabilities, with online and e-commerce the area most at risk.

From 30th June 2018 organisations will need a more secure encryption protocol in order to safeguard payment data and meet the PCI DSS standard. With just over a week to go, we wanted to share these key points, so you can check you have everything in place and understand what it means for your file transfer solution.

What do I need to put in place?

Essentially, you need to have a secure alternative – both at the network layer and at the data protection layer – and disable any fallback to SSL or early TLS. Your two options are as follows:

1. Migrate to TLS 1.2

The PCI council make a clear recommendation that you transition to TLS 1.2:

“TLS 1.2 is considered secure and is the recommended option from the council.”

SSL and Early TLS Migration webinar, Feb 2018.

TLS 1.1 is a more complicated option because it is possible to meet the requirements for strong cryptography, but it depends on the configuration, algorithms, strength of keys and other aspects of the implementation.

2. Compensatory controls

SSL and early TLS are not considered strong cryptography so they cannot be used as a security control for PCI DSS. You could add alternative security controls that remove the reliance on SSL and early TLS. Encryption would need to be in place to secure the transmission before it is sent using SSL or early TLS. Eg: at the application layer.

Exception for POI devices

This exception is in place because Point-of-Interaction (POI) terminals are not as susceptible to the vulnerabilities as browser based systems. If the device is built and configured in a way that’s not susceptible to the known vulnerabilities, it is possible to keep using it. You need to contact the vendor or support provider for that terminal, who can evidence this.

The device will still need up to date patches, must not use weak cipher suites or unapproved algorithms (eg: RC4 or MD5) and you must continually check that it hasn’t become susceptible to any new vulnerabilities. You should also have a migration plan in place that you can execute at short notice, should the device become susceptible. Any new devices should be configured to TLS 1.2.

You can find out more information on all the topics covered in this blog by watching the video from the PCI Security Standards Council.

What does this mean for my file transfer solution?

If you are running a file transfer solution and have kept it up to date, there is a good chance you won’t need any major changes. All the current versions from the main MFT vendors support TLS V1.2 and many default to only have TLS enabled.

Some products have PCI compliance scans built in, which will warn you if you are running SSL v3.0. It may not differentiate between TLS V1.0 and V1.2 though, so you will need to do a manual check. If you have a support contract with Pro2col, raise a support ticket and one of our technical consultants will find out if your solution is configured for TLS V1.2 or not.

If you are running an older version of your file transfer solution, you may need to upgrade. Again, Pro2col can advise on the process and our professional services team have experience getting out of date software up to the latest version.

If you are running an older SSL certificate built using 512-bit or 1024-bit key sizes, it is worth renewing it. The recommendation is now to use 2048-bit or greater.

To compare Managed File Transfer (MFT) solutions with PCI DSS compliant features, complete the Managed File Transfer Comparison Report. This will recommend and compare solutions meeting your specific requirements.

Interested in a file transfer solution?

Are your GDPR impact assessments all in place?

Are your GDPR impact assessments all in place?

By 25th May 2018 you will need to have carried out GDPR impact assessments for all processing activities involving personal data of EU citizens. This is really important, even if you are confident your systems, processes and security is in line with the regulation.

This blog takes you through the reasons why, plus exactly what an impact assessment is in the context of data transfer and file sharing.

What is a DPIA?

The Data Protection Impact Assessment – or DPIA – is described in article 35 of the GDPR and then detailed in article 30. DPIAs need to be created for all new workflows and retrospectively applied to all existing workflows.

A DPIA performs several functions at the same time:

  • It is a risk and impact analysis
  • A description of a workflow including its purpose
  • A breakdown of security settings
  • It forms the basis for subsequent audit reviews.

The reality is that every workflow handling personal data must be fully documented. There is no official template for this document, but it makes sense to adopt a standard form as early in the process as possible. Consider creating two assessments – one top level and one containing the technical detail.

Think of the top level DPIA as an executive summary providing a description of the processing options, and the reason for doing it. This can be as simple as ‘Sending employee Records from HR system to Accounts SFTP server’ and ‘Required for loading monthly payroll’. It should also contain the perceived risks to the process, and any mitigations applied. The DPIA needs to be signed off by the Data Protection Officer.

The more technical DPIA should contain the detail of the workflow. We recommend the following as a minimum:

  • The software in use
  • The Data Controller Name
  • The categories of data being transferred
  • The recipient category (EU/3rd Country/International Organisation)
  • The data retention period
  • The security methods being employed (Encryption, High Availability, recovery options, penetration tests, etc.)

DPIA as a record of processing

Completing a DPIA meets the requirements of GDPR article 30. It means that during an audit, you will be able to provide a document to fully explain each workflow handling personal data.

It also means that you will need to periodically review each DPIA and ensure that they still match the reality of the situation – sometimes every organisation will have an update to a workflow (planned or otherwise) which does not get adequately documented. Similarly, if something outside the actual workflow itself changes (for example, retention periods), then the DPIA needs updating. It makes sense to attach the results of continuity or penetration tests directly to the DPIA as evidence, if ever required.

 

White Paper and supporting resources

You can find out more about this and other aspects of the GDPR for data transfer and file sharing in our White Paper. When you download this you will also receive an example GDPR impact assessment template and interface mapping template. These resources are designed as a guide to support you meeting the requirements of GDPR article 30 and 35.

Interested in a file transfer solution?