Open Source Managed File Transfer Software: Is it Really an Option?

Open Source Managed File Transfer Software

Is it Really an Option? [Updated – 2017]

I originally wrote this post back in July 2012, after a number of requests for open source Managed File Transfer from potential customers. They’d found us via our web site, which clearly promoted a wide variety of commercial products, with no reference to open source, however they were only interested in an open source options.

Free clearly doesn’t pay the bills, however being a bit of an industry geek, I decided to do the research and find out what was available. I identified a couple of SourceForge projects, which I’ve been following over the past 4-5 years.

For the record, I’m a fan of open source. Our previous Technical Director was a thrifty Northerner, who converted me. We ran various elements of the business very successfully on open source projects. Our support ticketing system was based on OTRS and our monitoring system used Nagios. Both were mature applications, widely used in the open source community and provided considerable functionality at a price that suited!

 

open_source

When it comes to open source Managed File Transfer though, the landscape is patchy at best. Open source FTP servers exist in abundance and can provide the landing point for incoming and outgoing files, but open source Managed File Transfer projects appear to be scarce. I suspect that this is because Managed File Transfer hasn’t been one of those technologies that every company deploys.

Open source Managed File Transfer trends

My research over the past 4-5 years has highlighted a few trends that don’t bode well for open source Managed File Transfer projects, and may be a reflection of the wider open source landscape. In general, they have slotted into the following categories:

  • The company is acquired and the free option is removed. Some or all of the functionality is incorporated into a commercial offering.
  • The part-time developer gets a contract or new job and the project gets shelved.
  • The project is labeled as Managed File Transfer, but doesn’t contain the key functions of all commercial offerings in the market.
  • Commercial vendors offer a pseudo open source product with limited functionality and encourage migration to their commercial solution.

Some open source MFT that is no more

These are some of the open source projects that I have been following:

Policy Patrol by OPSWAT – has now become Metadefender Email Security. The Managed File Transfer element of the open source project appears to have been shut down.

ShieldShare by BlockMaster – now appears to be part of the DataLocker stable but their focus is on encrypted storage. It’s unclear whether the product was acquired for the encryption capabilities. Project shut down.

Appterra – their open source supply chain integration platform with Managed File Transfer capabilities was acquired by Descartes. The open source project has been shut down.

DivConq MFT – a SourceForge project that looked promising but the developers seem to have ended the project and the associated web site has closed down.

Karonte – positioned as an open source Managed File Transfer solution but it doesn’t have the basic functionality we consider critical for MFT.

Genuine open source projects

There are just two genuine projects that I’ve found so far. I couldn’t recommend either, as I’ve not invested any time personally or asked our technical consultants to review them, however they have ongoing development and support packages.

Yade – an open source project, previously going under the name of SOSFTP. This project has been around since at least 2012. SOS Berlin lists a number of customers on their web site and provides support and consulting packages. Currently my favourite option, based upon the length of time they’ve been around.

WAARP – a relative newcomer to the market but it looks to have all the basics covered. It also provides commercial support options and their web site provides visibility of who is involved in the project. This is certainly one to watch.

If you’re a user of either Yade or WAARP, I’d be interested in hearing from you. I’m keen to understand how complete the project is, how responsive the development team are and what your experience of their support offering has been like.

In conclusion

Whilst the open source marketplace can be a fantastic resource for some business applications, Managed File Transfer isn’t currently one of them. If your business is in the tech space or you’ve an extensive development and technical team then open source Managed File Transfer may be a viable option for you.

However, Managed File Transfer is mission critical for almost all of our customers and many come to us looking to mitigate the risks associated with supporting a bespoke or homegrown solution. As developers and contractors move on, many companies we speak to are stuck with a solution that has minimal documentation and the knowledge required to make changes or fix faults has been lost.

If you’d like to discuss your Managed File Transfer requirements, I’d be pleased to talk them through with you. You may be surprised at how much bang you get for your buck these days in commercially available products.

Get Your Free Copy Of Our Brand New Book,

The Expert Guide to Managed File Transfer[transparent background] Small

 

“THE EXPERT GUIDE TO MANAGED FILE TRANSFER”

 
 

The Expert Guide distills our knowledge from more than 700 Managed File Transfer
projects into a resource to get your project started.

It includes;

  • What is Managed File Transfer, EFSS, Big Data, High Availability etc?
  • What problems Managed File Transfer can solve? With real life use cases.
  • 40 key considerations for a successful Managed File Transfer project.
  • A side by side comparison of eight leading Managed File Transfer solutions.


Click Here Free Copy_Orange_#db5500

5 Steps for Writing a Successful Business Case for MFT

5 Steps for Writing a Successful Business Case for MFT

One area that lots of customers need assistance with is building their business case for Managed File Transfer. Whilst each organisation’s approvals process is unique, we see some common themes. Below are five key steps or suggestions to how to build a successful business case for Managed File Transfer.

Consider all the stakeholders

It goes without saying that the needs of each Line of Business are different and are likely to be distinct from the project drivers from IT. Marketing need to be able to send large files simply with the correct corporate branding. Legal require all files to be signed for. Procurement want date stamping to ensure all tenders were received within specified deadlines. End users would prefer to use consumer grade solutions that they are familiar with. IT are generally looking for scalability, manageability, speed of install, security and visibility. The business case for MFT can address all of these issues and more.

Executive Sponsor

Managed File Transfer is often the glue between corporate systems and there is often no natural owner within an IT structure. An executive sponsor can help to overcome any objections to change from end users and ensure cross team support for a project that will impact networks, database teams, IT security and more.

Business Benefits

All our vendors have white papers designed to address specific business issues such as PCI Compliance or GDPR. There are also reports on the benefits gained by existing users of MFT, which can be used to frame your business case. The Ipswitch report for example suggests that the key reason for implementing MFT is to improve productivity and this is supported by a Globalscape report that suggests MFT implementation can increase your file transfer capability ten fold without increasing your IT support.

Cost Vs Benefit Vs Risk

Whilst the costs are generally easy to identify, benefits can be harder to quantify. Key areas to investigate include savings associated with scripts that can be quantified. For example, experienced staff built the legacy scripts and are often involved with troubleshooting any issues as a result. MFT simplifies not just the creation of scripts, but also on-going management. MFT can also reduce the number of trouble tickets relating to the movement of files and the time it takes to locate the issue.

As well as direct cost savings, there are risks associated with data breaches or failures on customer SLAs. Research based on existing MFT users, show that implementing solutions can improve the satisfaction of internal and external partners and reduce their reliance on IT, thus improving productivity.

Selection Criteria

Lots of customers start with a very high level set of criteria that can be met by almost any MFT solution. To ensure you get the right solution, you need a more comprehensive set of criteria. Our Needs Analysis services can help to clarify functional requirements. It is also important to assess the total cost of ownership including support costs, training, documentation and upgrades. As the use case for MFT frequently expands post-installation, the scalability of the solution and options for additional features now and in the future are also important components of your business case.

There are some more detailed notes in our Expert Guide to Managed File Transfer and we have a library of white papers from research companies and vendors available upon request.

Get Your Free Copy Of Our Brand New Book,

The Expert Guide to Managed File Transfer[transparent background] Small

 

“THE EXPERT GUIDE TO MANAGED FILE TRANSFER”

 
 

The Expert Guide distills our knowledge from more than 700 Managed File Transfer
projects into a resource to get your project started.

It includes;

  • What is Managed File Transfer, EFSS, Big Data, High Availability etc?
  • What problems Managed File Transfer can solve? With real life use cases.
  • 40 key considerations for a successful Managed File Transfer project.
  • A side by side comparison of eight leading Managed File Transfer solutions.


Click Here Free Copy_Orange_#db5500

Implementing MFT? Who you gonna call?

Implementing MFT? Who you gonna call?

Unfortunately, the answer is never going to be as simple as “Ghostbusters!” – but if you don’t select the right people at every stage you could easily be in for some nasty surprises.

As with any implementation project, there are several stages that any organisation will have to go through, each potentially requiring a different team to perform the various activities depending upon the size of your organisation. In smaller organisations, it’s quite normal for one person to perform several roles or functions, but it’s important to know what particular functions are being covered in order to ensure that nothing gets missed. According to ITIL, there are three key phases in the lifecycle of any software – these are Design, Transition and Operations (Service Strategy and continuous Improvement don’t apply at this point).

itil-methodology

Let’s look at the phases in order:

Service Design

During the Service Design phase, the Managed File Transfer (MFT) software is selected and the service is designed.  At this time, most of the work comes from the project leader; however input is also required from other functions.

Function Role
Enterprise Architect Specialises in handling how applications interact.
Information Security Manager Assesses and approves security requirements.
Compliance Manager Ensures that standards are met.
Service Owner Manages the service after go-live date.
Technical Analyst There is normally a separate Technical Analyst or team for each specific aspect of the environment, for example – Server Build & Support, Network Support, Monitoring, Scheduling…
Applications Analyst Manages the MFT application, like Technical Analysts there will often be a separate analyst or team for each Application.
Continuity Manager Validates that the new MFT application meets the existing continuity criteria.
There are of course other key ITIL functions during the Service design phase, however these do not all necessarily need to be involved and may be happy simply to receive meeting minutes and key updates.

Service Transition

Once testing has been completed, it is time to move into the Service Transition phase. This is the crucial time between testing and production where so many projects run into problems; consequently, it is important to get it exactly right before moving forward. At this time, Change Management becomes vital (in some organisations, every step of the Design phase will also have been subject to Change Management).

The functions that provide input at this stage must include as a minimum:

Function Role
Change Manager As the Change Manager approves all changes, he will have final say on whether an application moves into production.
Change Advisory Board The CAB advises the Change Manager on whether to allow a change to go ahead.
Knowledge Manager The Knowledge Manager will ensure that Operations, Help Desk and end-user guides are available.
Test Manager The Test Manager not only ensures that the application is fit for service, but also that the operations team is able to support it.
Again, other ITIL functions exist but are not included here. Pay special attention to Change Management as this is often a point overlooked until the day arrives to install the new MFT system, causing unnecessary delays.

Service Operation 

Once the new system goes live, a whole new set of functions are required to support it.

Function Role
Help Desk The Help Desk team performs first level support and handle all new incidents. Incidents that they cannot resolve are escalated to other teams as necessary.
Access Manager The Access Manager grants or denies access to the MFT application.
Operations Team Perform the day-to-day activities to ensure the application runs successfully.

In Summary

The diagram below should help to show which roles are involved at which points in the lifecycle. As mentioned at the start of this article, these phases are taken from ITIL methodology, however they still apply even if you are not practicing ITIL. You should not for example enter into the design phase without including the Security or Compliance Managers.

key-roles-in-the-project

Get Your Free Copy Of Our Brand New Book,

The Expert Guide to Managed File Transfer[transparent background] Small

 

“THE EXPERT GUIDE TO MANAGED FILE TRANSFER”

 
 

The Expert Guide distills our knowledge from more than 700 Managed File Transfer
projects into a resource to get your project started.

It includes;

  • What is Managed File Transfer, EFSS, Big Data, High Availability etc?
  • What problems Managed File Transfer can solve? With real life use cases.
  • 40 key considerations for a successful Managed File Transfer project.
  • A side by side comparison of eight leading Managed File Transfer solutions.


Click Here Free Copy_Orange_#db5500

Managed File Transfer Automates Cloud Workflows

Managed File Transfer Automates Cloud Workflows

Companies are facing an opportunity to redefine their business and operational processes. Cloud technology has been proven in terms of scale, reach, reliability and security – and businesses are consuming an ever-increasing number of public and private cloud services as they strive to reduce CAPEX and operational overheads.

A recent Ipswitch survey of 500 IT professionals showed that a significant majority of businesses, across almost every industry sector, have adopted cloud file share services to increase the productivity of individuals, teams and workgroups.

 

cloud-workflows-yes-or-no

Source – Ipswitch Report Data Security Compliance, 2016

The next stage in this transition is to use cloud storage sites to automate business workflows. Using cloud storage in this context though presents a significant number of business challenges. Business data must be secured to the appropriate level.   And, data must be available to business critical workflows when needed. In general, cloud services, take data security extremely seriously and provide adequate protection. But how to automate files moving to and from the cloud is typically left for the customer or a technology partner to figure out.

Step forward Managed File Transfer!

Managed File Transfer (MFT) lends itself to resolving the ‘ground to cloud’ and back again conundrum. Built to interface with various endpoints, supporting open standard protocols, MFT solutions are built to automate business processes and an increasing number of MFT vendors offer interfaces to major cloud storage sites.

Cloud storage vendors have various ways that enable technology partners to interface with their services. Automated processes primarily use secure FTP or a public API provided by the cloud vendor.

A number of Managed File Transfer solutions now allow the delivery to and collection of files from an increasing number of cloud storage sites including:

  • Amazon S3
  • Citrix ShareFile
  • Dropbox
  • Google Cloud
  • Google Drive
  • Microsoft Azure
  • Box
  • OneDrive
  • Oracle Cloud

The benefits of automating the integration of cloud storage into business processes are clear. Reduced manual intervention, near to real-time file based workflows, notification and audit trails are some of the most obvious, reducing processing time and therefore providing significant ROI. Whilst the same could be achieved by orchestration, Managed File Transfer provides a competitive, feature rich, competitively priced option for file-based workflows.

As demand increases, the level of complexity and depth of integration is extended. The ability to write files directly using vendor-provided APIs(e.g., directly to Amazon S3 buckets or Microsoft Azure blobs) is becoming more widely used.

Now that the cloud is a mainstay of a business’ extended infrastructure, companies are looking to consolidate their services with a cloud storage provider requiring cloud-to-cloud migration. Where consolidation isn’t possible Managed File Transfer solutions are facilitating cloud-to-cloud workflows, so that businesses can link together best in class cloud technologies.

The cloud is here to stay and will become an increasingly important piece of the overall infrastructure landscape. Moving data to it, from it and between them will be as important as sending and receiving data directly with trading partners. Fortunately, a consolidated view of all business data transfers including the cloud is possible – once again Managed File Transfer is leading the way.

If you are currently using cloud storage or are considering adding it to your business workflow, drop us an email to [email protected] or call on 020 7118 9640. We can recommend which MFT solutions will best meet your needs.

Get Your Free Copy Of Our Brand New Book,

The Expert Guide to Managed File Transfer[transparent background] Small

 

“THE EXPERT GUIDE TO MANAGED FILE TRANSFER”

 
 

The Expert Guide distills our knowledge from more than 700 Managed File Transfer
projects into a resource to get your project started.

It includes;

  • What is Managed File Transfer, EFSS, Big Data, High Availability etc?
  • What problems Managed File Transfer can solve? With real life use cases.
  • 40 key considerations for a successful Managed File Transfer project.
  • A side by side comparison of eight leading Managed File Transfer solutions.


Click Here Free Copy_Orange_#db5500

Top 5 Tips for Reducing your Managed File Transfer costs

Top 5 Tips for Reducing your Managed File Transfer costs

Secure, automated, robust b2b workflows are key to businesses, frequently a vital system in the creation of customer value. However, Managed File Transfer solutions can be expensive, very expensive. If business continuity is critical, solutions may include high availability within their primary business locations and remote disaster recovery.  Coupled with pre-production and test environments, the Managed File Transfer landscape can become considerable with software investments reaching in excess of £100k in the first year.

Driving Cost Down and Functionality Up

There are ways to keep the cost down, providing you with the functionality required on a smaller budget. Here is a list of considerations to make your budget go further.

 

Do Your Research To Find The Right solution

Many IT professionals have a go-to technology that they’ve used before, or that has been recommended to them. Typically, it will be one of the better-known brands. These are proven systems, but it is wise to conduct thorough research to ensure you’re not backing the wrong technology from the outset. With such a large number of vendors and functionality largely the same, its not what they do but how they do it that’s important.

Upgrade What You Already Have In Place

It’s very likely that your organisation has an FTP server already.   In fact, if you dig around there’s a good chance you’ve got a lot more than just one. Many of today’s Managed File Transfer vendors started out by developing FTP server software. As business requirements have changed further protocols, enhanced reporting, analytics, automation engines, to name just a few have been added. It is definitely worth considering what upgrade paths exist to get you to your goal.

Only License What You Need

This might seem obvious but trust me, it’s not. Many vendors have developed further capabilities around mobile, ad-hoc transfer, API’s, dashboards, which you may not need right now. The bundled offerings provided by vendors look like a good deal, promoted as saving you ‘x’ percent over purchasing the modules separately. You may end up licensing software that you are unlikely to use, resulting in a wasted investment.

Another aspect is how many trading partners you are likely to communicate with. An unlimited licensing model isn’t appropriate for all companies and in fact some vendors license to a granular level by the number of connections you require. If for example you’ve only got 25 trading partners, you can get a really feature rich solution at a fraction of the cost. A comprehensive understanding of how different vendors price up their solutions can save you many thousands of pounds over the lifetime of the solution.

The Cloud Has Changed Everything

The rise in the development and adoption of cloud/SaaS solutions has been meteoric in recent years. Newer software companies are typically agile and have programmed their systems using newer languages, allowing much more rapid delivery. By developing for a single, hosted environment, new features can be added quickly in response to demand, and the QA process is much quicker. However, what is key here is the payment model. Subscription services tend to fall under OPEX expenditure, softening the blow of a significant CAPEX investment up front. Some on-premise software vendors have responded by offering subscription models for their software.

Reduce Your IT Support/Management Overheads

A recent study by the Aberdeen Group showed that 65% of companies purchase Managed File Transfer to improve productivity. Like any system though, these savings can only be achieved from a complete, correctly scoped and maintained solution. Time and time again, we see responsibility for MFT solutions falling between networking and security teams and the many lines of business.  The result is no-one takes ownership of the solution, businesses fail to take advantage of its capabilities and the net result is that it doesn’t deliver the potential ROI.

In this instance, I’d recommend a managed service, providing you with expert resource available at short notice to help onboard trading partners, configure interfaces and much more. Of course (shameless plug coming up) this is something our Managed File Transfer support team can help you with.

 

In addition to the key points above, there are a variety of other ways in which working with a Managed File Transfer consultant, could ensure you get the best solution for your needs, whilst maximising the value.

 

  • Complete a requirements analysis – for example our project guide has over 230 scoping questions
  • Design your ideal solution and use this to form the basis of your vendor selection
  • Minimise your professional services costs by working with a team, like us, that have been through the process hundreds of times before
  • Don’t recreate the wheel, make use of existing documentation and processes out there and available to you
  • Verify solution capabilities and pricing with independent experts

Get Your Free Copy Of Our Brand New Book,

The Expert Guide to Managed File Transfer[transparent background] Small

 

“THE EXPERT GUIDE TO MANAGED FILE TRANSFER”

 
 

The Expert Guide distills our knowledge from more than 700 Managed File Transfer
projects into a resource to get your project started.

It includes;

  • What is Managed File Transfer, EFSS, Big Data, High Availability etc?
  • What problems Managed File Transfer can solve? With real life use cases.
  • 40 key considerations for a successful Managed File Transfer project.
  • A side by side comparison of eight leading Managed File Transfer solutions.


Click Here Free Copy_Orange_#db5500

The Growth of Mobile Devices in the MFT Workspace

The Growth of Mobile Devices in the MFT Workspace

Background

android-and-iphone-iconsIn the last ten years, there has of course been a massive growth in the realm of mobile devices, specifically with regards to smart phones and tablets. It now seems strange to think of life without them, but iPhones only appeared in 2007, and Android in 2009.
Of course, before then we had PDA devices that allowed us to sync with our desktop computers and tap away at a tiny Word document while sitting on a train, but the whole idea of transferring data while on the move was very much in its infancy still.

Smartphones

We all know, as the market grew for mobile devices, so did the range of applications available for them. One common factor for all mobile devices is the ability to browse the internet; it is this perhaps that first made people stop and think about how they could use a mobile device to access their company Managed File Transfer (MFT) site. In my case, I would download a list of support phone numbers from our MFT system whenever I needed to contact a technician from another team.

MFT specific clients

Before long however, specific mobile device client software from the MFT vendors began to appear, simplifying the process of downloading files. As is often the case, where one vendor releases a new product, it is soon available from all of the vendors; now it seems that every product has a mobile device client available.

Much like the MFT systems that they support, not all clients are equal. If you are planning on selecting an MFT product on the basis of the strength of the mobile client, here are some thoughts for you to consider:

  • Does the client open files in the right application automatically? Or do you need to save them first?
  • Can the MFT system enforce a PIN on the device? Remember that confidential files could be in the wrong hands if the device is stolen.
  • Can you remotely wipe files on the device if required?

Being able to remotely download a file and work on it is certainly very useful, however equally important is the ability to upload files; this is easiest to see when considering taking photos. All smartphones come with pretty good cameras; some MFT clients allow you to navigate to a folder, then activate the camera and store all photos into the folder automatically. Imagine a roving insurance agent or removals planner being able to have the images available in his/her office before they return; the possibilities for workflow management are endless.

Network speeds

However, we do need to remember the limitation that some networks bring; A report by Ofcom in late 2014 found that the average UK download speed for 4G was 15.1 Mbit/s (6.1 Mbit/s for 3G) and the average upload speed for 4G was 12.4 Mbit/s (1.6 Mbit/s for 3G). At that time, 98% of the UK was covered by 3G, but only 70% was covered by 4G.

Source: https://www.ofcom.org.uk/about-ofcom/latest/media/media-releases/2014/3g-4g-bb-speeds

Source: https://www.ofcom.org.uk/about-ofcom/latest/media/media-releases/2014/3g-4g-bb-speeds

To put that into context, that’s nearly a minute to download a 100MB file on 4G and just over a minute to upload it again; if you drop to 3G coverage you would need around nine uninterrupted minutes to upload the file…

It happened to me…

I remember a project I worked on where the salesforce would upload orders into MFT while visiting customers; inside the cities it worked fine, but once out into the countryside there simply wasn’t sufficient coverage. No one had considered the coverage during the planning phase.

Of course there’s always the wifi alternative; but somehow I would feel uncomfortable if I knew end-users were rushing to their local fast food cafes or shopping malls and using unsecured wifi networks.

Summary

So in summary, if you intend to use Managed File Transfer primarily for mobile devices, pay special attention to exactly what the client application provides you. However; pay even more attention to whether the mobile networks in the area you will use the devices can deliver the speeds you need to succeed.

Get Your Free Copy Of Our Brand New Book,

The Expert Guide to Managed File Transfer[transparent background] Small

 

“THE EXPERT GUIDE TO MANAGED FILE TRANSFER”

 
 

The Expert Guide distills our knowledge from more than 700 Managed File Transfer
projects into a resource to get your project started.

It includes;

  • What is Managed File Transfer, EFSS, Big Data, High Availability etc?
  • What problems Managed File Transfer can solve? With real life use cases.
  • 40 key considerations for a successful Managed File Transfer project.
  • A side by side comparison of eight leading Managed File Transfer solutions.


Click Here Free Copy_Orange_#db5500