Open Source Managed File Transfer Software: Current & Past Options

Open Source Managed File Transfer Software

Current & Past Options [Updated January 2019]

 

If you are looking for an open source Managed File Transfer solution, this blog is for you. It’s written by Pro2col Managing Director James Lewis, who is a self-confessed file transfer geek with over two decades’ experience working with file transfer technologies and vendors. He’s been following open source projects for many years and regularly updates this blog as new options become available.

 

I originally wrote this post back in July 2012, after a number of requests for open source Managed File Transfer from potential customers. They’d found us via our website, which clearly promoted a wide variety of commercial products, with no reference to open source, however they were only interested in open source options.

Free clearly doesn’t pay the bills, but being a bit of an industry geek, I decided to do the research and find out what was available. I identified a couple of SourceForge projects, which I’ve been following over the years.

For the record, I’m a fan of open source. Our previous Technical Director was a thrifty Northerner, who converted me. We ran various elements of the business very successfully on open source projects. Our support ticketing system was based on OTRS and our monitoring system used Nagios. Both were mature applications, widely used in the open source community and provided considerable functionality at a price that suited!

 

open_source

When it comes to open source Managed File Transfer though, the landscape is patchy at best. Open source FTP servers exist in abundance and can provide the landing point for incoming and outgoing files, but open source Managed File Transfer projects appear to be scarce. I suspect that this is because Managed File Transfer hasn’t been one of those technologies that every company deploys.

Recent changes in EU regulation in the form of GDPR, also impacts this space. GDPR legislation requires increased levels of security, audit trails and reporting on any transfers involving personal data. (See our blog posts Encryption at rest for GDPR and Where is your data going and why?)

Open source Managed File Transfer trends

My research over the past 6-7 years has highlighted a few trends that don’t bode well for open source Managed File Transfer projects, and may be a reflection of the wider open source landscape. In general, they have slotted into the following categories:

  • The company is acquired and the free option is removed. Some or all of the functionality is incorporated into a commercial offering.
  • The part-time developer gets a contract or new job and the project gets shelved.
  • The project is labeled as Managed File Transfer, but doesn’t contain the key functions of all commercial offerings in the market.
  • Commercial vendors offer a pseudo open source product with limited functionality and encourage migration to their commercial solution.

Genuine open source projects

There are just two genuine projects that I’ve found so far. I couldn’t recommend either, as I’ve not invested any time personally or asked our technical consultants to review them, however they have ongoing development and support packages.

Yade – an open source project, previously going under the name of SOSFTP. This project has been around since at least 2012. SOS Berlin lists a number of customers on their website and provides support and consulting packages. Currently my favourite option, based upon the length of time they’ve been around.

WAARP – a relative newcomer to the market but it looks to have all the basics covered. It also provides commercial support options and their website provides visibility of who is involved in the project. This is certainly one to watch.

If you’re a user of either Yade or WAARP, I’d be interested in hearing from you. I’m keen to understand how complete the project is, how responsive the development team are and what your experience of their support offering has been like.

Free products from commercial vendors

As I’ve already mentioned, these tend to be products with limited functionality, which will ultimately encourage migration to a commercial solution.

Coviant Diplomat OpenPGP Community Edition – This is a free OpenPGP tool to automate PGP encryption and decryption.

Coviant Diplomat Cloud Storage Community Edition – A free tool to PGP-encrypt files being transferred to cloud storage sites.

HelpSystems Free FTP Server – this is a free edition of GoAnywhere MFT, with administrator dashboard, extensive security, audit reports and more.

FTP Voyager – A free GUI FTP client, FTPS client and SFTP client software for Windows. The main interface is similar to dozens of other FTP clients, but it also includes powerful scheduling utilities and synchronisation utilities for free.

Some open source MFT that is no more

These are some of the open source projects that I have been following, which are no longer available:

Policy Patrol by OPSWAT – has now become Metadefender Email Security. The Managed File Transfer element of the open source project appears to have been shut down.

ShieldShare by BlockMaster – now appears to be part of the DataLocker stable but their focus is on encrypted storage. It’s unclear whether the product was acquired for the encryption capabilities. Project shut down.

Appterra – their open source supply chain integration platform with Managed File Transfer capabilities was acquired by Descartes. The open source project has been shut down.

DivConq MFT – a SourceForge project that looked promising but the developers seem to have ended the project and the associated website has closed down.

Karonte – positioned as an open source Managed File Transfer solution but it doesn’t have the basic functionality we consider critical for MFT.

In conclusion

Whilst the open source marketplace can be a fantastic resource for some business applications, Managed File Transfer isn’t currently one of them. If your business is in the tech space or you’ve got an extensive development and technical team, then open source Managed File Transfer may be a viable option for you.

However, Managed File Transfer is mission critical for almost all of our customers. Many come to us looking to mitigate the security and compliance risks associated with supporting a bespoke or homegrown solution. Unmanaged in-house scripts in particular are one of the biggest risks to an organisation’s GDPR compliance. Additionally, as developers and contractors move on, companies get stuck with a solution without documentation, no training and no one to make changes or fix faults.

If you’d like to discuss your Managed File Transfer requirements and the impact of GDPR, I’d be pleased to talk them through with you. You may be surprised at how much bang you get for your buck these days in commercially available products. You can contact us via the web form, or call 0333 123 1240.

Need further expertise? Download our FREE resources

PGP encryption

PGP encryption

To PGP or not to PGP? That is the question several customers have asked us recently. This blog post summarises everything you need to know about PGP encryption, so you can make an informed decision about whether it is right for you.

PGP stands for ‘Pretty Good Privacy’. It is an asymmetric encryption, which means it uses public and private keys to encrypt and then decrypt cipher text. It requires more work than symmetric encryption, which uses a shared key, but is generally considered better security.

PGP provides end-to-end encryption, integrity checking and authentication. It is commonly used for encrypting and decrypting texts, files, directories and whole disk partitions.

 

PGP Encryption: How does it work?

Asymmetric encryption uses two different keys to encrypt and decrypt each file, then two more keys to sign and verify each file. Both parties – sender and recipient – need to exchange their public keys before any transfer can take place.

The sender encrypts the file using the recipient’s public key. The recipient decrypts the file using their private key.

For integrity checking – to make sure the content hasn’t been tampered with – the sender uses their private key to ‘sign’ the encrypted file. For authentication – to check the sender is the sender you think it is – the recipient uses the sender’s public key to verify/validate the sender.

 

PGP and your file transfer solution

PGP Clients will manage the encryption/decryption automatically and are often implemented in FTP servers or as email client add-ons to secure the communication. The exchange of the public keys, however, will always be a manual process.

Any security is only as strong as its weakest point. Security-conscious organisations will usually physically exchange keys via a courier service, and set keys to expire (this is a bit like a password which expires and needs to be reset by the security team). But – as you will have gathered – the process of exchanging keys is time consuming. Most applications provide advance notice about expiring keys, so administrators can plan for the exchange to take place in advance.

Some applications allow you to create sub-keys with pre-configured expiry dates, so that you can plan ahead and have several years of automatic key replacement, avoiding potential outages. We know of some Managed File Transfer solutions that manage this process very effectively.

 

When to use PGP

PGP provides encryption at rest or can be used to protect a file at a particular stage in an otherwise non-encrypted workflow.

Let’s look at a recent example we discussed with a customer who had a PGP requirement for an accounts process. They needed to put files into a specific folder, where they would be PGP encrypted, then moved to another folder to be collected by the bank. This would by-pass a charge that the bank would otherwise make for the processes.

This requirement was driven by the fact that the bank used PGP, and the businesses needed to comply in order to save money.

The advantages of PGP:

  • Security is the big plus. PGP is generally considered more secure than symmetric encryption.
  • Even if the channel transmitting the files becomes compromised, the private keys and files remain safe. Similarly, they are safe if the channel used to share public keys is compromised.
  • Signing files is a built-in procedure, automatically authenticating the sender’s identity.

 

Disadvantages:

  • End users need to exchange keys and use their encryption technology correctly. They often accidentally send their PRIVATE keys to each other.
  • Slower performance than symmetric encryption.

Next steps

If you need to know more about secure file transfer protocols, encryption, or any other aspects of working with a Managed File Transfer (MFT) solution, take the Certified File Transfer Professional (CFTP). It is the only vendor-independent file transfer certification, equipping you with the knowledge you need to implement secure file transfer in your organisation.

Alternatively, if you are investigating which solutions have PGP capabilities, opt for our free MFT Comparison Service. Answer a series of questions about your requirements and our experts will recommend the best solution.

Need further expertise? Download our FREE resources

Supply chain disaster: Do you need an MFT dev environment?

The reasons why you need an MFT dev environment

MFT dev environment - lorries in supply chain disaster

 

In all the years we’ve been working in file transfer, there have been a few occasions when we’ve witnessed the financial impact and reputation damage a system failure can have. This article looks at:

 

  • Why you should think twice before testing in a live environment;
  • When you need to consider a development (dev) environment for your Managed File Transfer (MFT) solution;
  • Details of the six stages for testing and development.

“A few years ago, one organisation was developing workflows in a live environment, and broke other automated processes. The system was down for just a few hours, but the impact was huge. This business supplied products to retailers across the country, but were unable to access the order information. The lorries couldn’t leave the factory and delivery drivers had to be paid overtime. Worse still, the retailers were left out of stock, consumers bought other brands and some ended up staying with that brand. The impact on the business’ finances and reputation were catastrophic.”

 

Richard Auger, Pro2col technical consultant

This particular example could have been prevented if the IT team were developing in a test environment, instead of a live environment. But so many organisations only have a live MFT production licence. That might be to save money, or because decision makers just don’t think a file transfer server needs a test licence. But we know an MFT system is doing so much more than transferring files, so if you have any workflows involved, you need to reconsider.

Is a dev environment business critical?

This will depend on the value of the data your system is handling. Is it critical to business processes? Do you risk breaching service level agreements (SLAs)? Or will you simply not be able to operate, like the example above? While you may be able to send files by some other method for a few hours, it isn’t viable for a sustained period.

You also need a change control policy to meet ISO27001 requirements. While it is down to you to determine the right policy for your unique set of circumstances, example ISO best practice advocates testing in an isolated, controlled and representative environment. Similarly ITIL requires an organisation to follow both ‘change management’ and ‘release and deployment management’ processes from non-production to production systems. It’s an old IT joke that in weaker, less secure environments TIP doesn’t mean ‘Transfer into Production’ – it ends up being ‘Test in Production’ instead.

So to avoid disrupting your system when deploying new releases, building workflows or making other changes, you should follow these six stages for testing, developing and transfer into production:

  1. Sandbox, or experimental environment: This is a local environment no one else can access, where the developer has a working copy of the code. Here they can try it out and change it without putting it live. This environment will typically be an individual developer’s workstation. Once they are happy with it the developer would submit the code to the repository for the next stage of development. Most MFT solutions by default don’t have a sandbox but you can sometimes set it up by installing the software onto a private virtual machine.
  2. Development or integration environment: This is a clean environment where you test how your code is interacting with all the other bits of code associated with the system. The code itself doesn’t get changed in this environment – updates are made to the working copy back in the sandbox and resubmitted. When ready, the developer accepts the code and it is moved to the test environment.
  3. Testing: This is the environment to test the new or changed code, either manually or using automated techniques. You may have different test environments to focus on different types of testing. The developer looks at how it interacts with and impacts other systems and tests performance and availability. If you are upgrading, for example, this will show how your system will behave once the upgrade is in place. From here, the code can be promoted to the next deployment environment.
  4. User acceptance testing (UAT) or quality assurance (QA): In this stage users will trial the software, making sure it can deliver against requirements. Stress testing is also carried out in this stage.
  5. Pre-production, or staging environment: This final stage tests in conjunction with all the other applications in the infrastructure. The aim here is to test all installation, configuration and migration scripts and procedures. For example, load testing happens here. It’s really important that this environment is completely identical to the production (live) environment. All systems should, for example, be the same version.
  6. Production or live environment: Transfer into production – or TIP – is the final stage, bringing the updates live. This is the environment that users actually interact with. This can be done by deploying new code and overwriting the old code, or by deploying a configuration change. Some organisations choose to deploy in phases, in case of any last minute problems.

If you follow these steps you can be confident that any upgrades to the production environment will be completed reliably and efficiently. But if your budget or internal policy won’t allow you to invest in all of these, we would recommend at least a test environment, which should be an exact copy of the production environment.

All our vendors offer test licences at reduced rates. If it’s time to get this set up for your MFT solution, get in touch now. You can contact us via the website or by emailing your account manager.

Interested in a file transfer solution?

Choosing the right Managed File Transfer protocol

This blog post answers your questions about Managed File Transfer protocols. Which are the most widely used file transfer delivery protocols? Which should you be using and how do you identify which solution uses which protocols?

A protocol is the set of rules that determines how files are transferred from one computer to another, through a network. That might be an internal network (from one computer to another within the same network) or more commonly a Wide Area Network such as the internet.

The nature of your data and its destination will determine the right protocol for the transfer. For example, personally identifiable data and credit card information will need a secure protocol.

BASIC PROTOCOLS

FTP
(File transfer protocol)

How can it be used?

Upload/download files
Rename and delete files
Create/delete folders
Execute custom commands on server
Check integrity of files

When can it not be used?

X Secure data at rest
X Secure data in transit (FTPS can)
X Work over just one firewall port
X Provide strong authentication

FTPS
(“FTP Secured” using SSL)

 Secure data in transit
Upload/download files
Rename and delete files
Create/delete folders
Execute custom commands on server
Check integrity of files
 Provide strong authentication

X Secure data at rest
X Work over just one firewall port

SFTP
(“Secure FTP” using SSH)

Secure data in transit
Upload/download files
Rename and delete files
Create/delete folders
 Provide strong authentication
Work over just one firewall port (22)

X Secure data at rest
X Check integrity of files
X Execute custom commands on server

SCP
(“Secure CoPy”)

Secure data in transit
Upload/download files
Work over just one firewall port (22)
 Provide strong authentication

X Rename and delete files
X Create/delete folders
X Check integrity of files
X Execute custom commands on server

ADVANCED PROTOCOLS

HTTP
(HyperText Transfer Protocol)

HTTP CAN ALWAYS
Download files
Work over one firewall port (80)

HTTP CAN SOMETIMES
Upload files
Rename and delete files
Create/delete folders
Execute custom commands on server
Check integrity of files

Secure data at rest
X Secure data in transit (HTTPS can)
X Provide strong authentication

HTTPS
(HTTP Secured with SSL)

HTTPS CAN ALWAYS
✓ Download files
✓ Work over one firewall port (443)
✓ Secure data in transit

HTTPS CAN SOMETIMES
– Upload files
– Rename and delete files
– Create/delete folders
– Execute custom commands on server
– Check integrity of files
Provide strong authentication

Secure data at rest

WebDav

WEBDAV CAN ALWAYS
Download/upload files
✓ Rename and delete files
✓ Create/delete folders
✓ Work over one firewall port (443)
✓ Secure data in transit

WEBDAV CAN SOMETIMES
 Provide strong authentication

Secure data at rest
X Execute custom commands on server
X Check integrity of files

EMAIL PROTOCOLS

SMTP
To send mail

Push files as attachments
✓ Be secured with SSL/TLS
✓ Often uses ports 25, 465 or 587

Pull files from other servers

POP3
To get mail

Pull files from servers as attachments
Delete original email from servers
Be secured with SSL/TLS
Often uses port 995

X Push files as attachments
X Synchronize email folder contents
X Not supported in all email environments
POP3 is becoming obsolete

IMAP
To get mail and sync mail folders

Pull files from servers as attachments
✓ Delete original email from servers
✓ Synchronize email folder contents
✓ Be secured with SSL/TLS
Often uses port 993

Push files as attachments
X Be trusted if its key mailbox is also accessed interactively

Which Managed File Transfer protocol?

Guidance on what constitutes a secure protocol will change, adapting to stay one step ahead of cybercrime. That’s why it’s important to choose a vendor that releases regular product updates. With Pro2col’s free Managed File Transfer comparison service, you submit your requirements via a questionnaire. Our experts compare them against the different solutions and recommend the right product for you. Our experts consistently review the marketplace and only select credible solutions from credible vendors, who provide excellent support and regular software updates.

Interested in a file transfer solution?

Managed File Transfer software comparison

Managed File Transfer software comparison 

[Updated – September 2018]

Are you doing a Managed File Transfer software comparison? With over forty products on the market, where do you start?

Globalscape EFT, HelpSystems GoAnywhere, Cornerstone from SRT, Ipswitch MOVEit and Cleo Harmony are all excellent Managed File Transfer products that we recommend to customers. But even from this shortlist, which is the right one for your organisation?

Every business has a unique set of requirements and each solution delivers its feature-set differently. There are so many factors that will determine if your implementation is a success. Sourcing the wrong product will cost you more in the long run.

Pro2col’s free comparison service identifies the right solution for your needs and budget. You complete a series of questions about your current and future business requirements, and receive a bespoke report from our technical consultants, recommending the best solution for you.


WHY REQUEST A MANAGED FILE TRANSFER SOFTWARE COMPARISON?

SAVE TIME

You will save weeks of research time by completing this questionnaire, making use of our technical consultants’ knowledge and expertise.

EXPERTISE

Pro2col experts have been delivering secure file transfer solutions since 2004 across 30 countries. Each technical consultant has a minimum of seven years’ experience working with this niche technology. We are also providers and developers of the only file transfer certification, CFTP.

INDEPENDENT

Software vendors will want to sell you their product, but our technical experts independently analyse the best solution for your unique requirements. They consistently review the marketplace and only select credible solutions from credible vendors, who provide excellent support and regular software updates.

FULLY SCOPE YOUR REQUIREMENTS

Your internal processes and current / future business requirements will determine which solution is the best fit. That is because different software differs in how it delivers the same set of features – the level of detail makes all the difference. Our free Managed File Transfer comparison service asks you the right questions to recommend an exact fit for your organisation, making sure your implementation is a success.

The questionnaire prompts you to consider the following criteria: Key infrastructure questions you need to think about when comparing MFT software; how your solution will be impacted by other policies within the business; requirements for automated transfers
Transfer protocols, which will depend on the security requirements for the data (eg: personally identifiable data, credit card information);
which cloud services you need to connect to; which standards you need to comply with (eg: GDPR, PCI DSS).

Pro2col Cyber Essentials certification

Pro2col growth continues with achievement of Cyber Essentials certification

Pro2col Ltd have received Cyber Essentials certification, recognising our ongoing commitment to security and data protection.

“As Pro2col becomes more widely recognised for our niche expertise in data transfer and file sharing technology, we are servicing larger organisations. It is important that our already rigorous security standards are formally recognised, and we are delighted to have achieved this certification.”

James Lewis, Pro2col Managing Director

This badge reassures customers that an organisation takes cyber security seriously and will protect their data. It is issued by Bureau Veritas who are a trusted testing, inspection and certification body. You can see Pro2col’s Cyber Essentials certificate of assurance online.

This achievement comes just a few months after Pro2col secured ISO 9001:2015 certification, recognising our commitment to customer service and quality.

About Pro2col

Pro2col are independent consultants, specialising in hybrid integration platforms and secure data transfer and collaboration technology, support and professional services since 2004. We have deployed over 750 solutions for a range of different industries across 30 countries. Companies use this technology to automate regular transfers, send large files, secure data, replace home-grown scripts, move data to and from the cloud and integrate with other systems.

Could your organisation benefit from this technology? Find out more about Pro2col’s products and services. Alternatively complete the Managed File Transfer comparison and receive a free software recommendation based on your organisations unique requirements.

Interested in a file transfer solution?