Managed File Transfer Versus Middleware

Managed File Transfer Versus Middleware

Managed File Transfer and Middleware have both undergone a period of evolution in the past few years. Historically speaking, the early days of both can be easily traced back to the need to move data between various parts of a computer network, generally over simple protocols like FTP or RCP. As a consequence and especially as organisations began to move away from legacy environments, many networks contained an inordinate number of FTP servers, frequently with an unknown array of FTP clients pushing and pulling data in an often uncontrolled fashion.

Middleware stands up…

This became a standard argument for switching to using a middleware product – taking back control of your network and the data that crossed it. Most early middleware systems used a hub and spoke affair and provided a central point where all data would arrive and depart from. Additionally, the notion of data transformation during transit became popular, rather than the more traditional manipulation during processing at source or target system. A ‘code once use many times’ approach appeared for interfaces, allowing for a reduction in development costs, and the only limitation appeared to be the ever-growing range of available connectors.

The beginning of MFT…

FTP servers didn’t go away however; instead organisations began to centralise their FTP sites and a newer smarter generation of FTP server software began to appear. These early versions of Managed File Transfer quickly developed a common set of standard features – encryption, automation, protocol support and user management.

Which is which?

As both middleware and Managed file transfer systems matured, the boundaries between them began to diminish somewhat, with Managed file transfer performing some middleware functions and vice versa. Now we have reached a point where the practical differences have become a little fuzzy, however it shouldn’t be impossible to follow some simple guidelines to decide upon whether an architect should be following a middleware or a managed file transfer approach.

A good starting point is data transformation. Traditionally this falls squarely within the realms of middleware; however there are Managed file transfer solutions which can offer this feature well enough to be considered. In contrast, most middleware does not provide an FTP interface for end-users, relying instead on web services for input or FTP clients for output. An organisation therefore has to review its requirements – do they need an Managed file transfer solution with some middleware functionality, or middleware with some Managed file transfer?

Managed File Transfer Functionality

Middleware Functionality

While trying to avoid generalisations, here are some things to consider that Managed File Transfer solutions provide and middleware ones generally don’t (or at least not well):

  • Enterprise File Sync and Share – the process of sharing data by sending a hyperlink via email is not well supported by middleware
  • Large File Transfer – Very large files are not suitable for transformation and therefore are not often considered by middleware vendors
  • File repository – Managed file transfer systems normally provide a repository of data for download, often encrypted
  • Home folder management – mostly, if a middleware system permits users to have home folders, these have to be manually created
  • Development and Deployment – on the whole, managed file transfer allows for faster design and rollout of interfaces than middleware, which often requires full development teams

Conversely, Middleware can provide functionality that Managed file transfer often struggles with, for example:

  • Mapping, database lookups and transformation – middleware supports complex mapping operations, either custom or using internationally recognised templates.
  • Customisable interfaces – middleware provides a framework for development, meaning bespoke designs can be implemented.
  • Peer-to-peer relationships – generally only available in specialised Managed file transfer products (using agents for example), peer-to- peer interfaces are becoming more popular, especially when making use of cloud technology.
  • Adapter support – most middleware products provide adapters which allow connections to just about any kind of system. Managed file transfer systems are generally limited to a handful of transfer protocols
  • Realtime support – with the exception of AS2 transfers, most MFT transfer products are not well suited to synchronous transfers, whereas middleware will generally handle synchronous transfers without problem

In Summary

When considering simple automation, large file transfers or user initiated transfer, Managed File Transfer is better suited than middleware. When looking to introduce complicated interfaces, message transformations or realtime processing, consider using middleware.

The best solution however must come when there is a symbiosis of the two; traffic passes through a Managed file transfer system and is handled by the middleware product. From an automation perspective the flexibility of Managed File Transfer represents a tactical solution, whilst more persistent interfaces are developed using middleware.

If your company is considering implementing a system for securely exchanging data and integrating it into your internal network, you’ll need to know whether the features you require are provided by the leading managed file transfer solutions, or middleware systems. Download our free managed file transfer comparison guide, which provides an ‘at a glance’ list of features and much more:

Download a Comparison of 8 Leading Managed File Transfer Solutions!

 

MFT_Comparison Guide Img

In this essential pack you’ll also find…

 

  • Key features and frequently asked questions

  • Other business policies that will need to be considered

  • Access to additional resources

  • Side by side comprehensive comparison

    * Updated to include new vendors (October 2015)

Manchester United and the Exploding Mobile Phone

Manchester United and the Exploding Mobile Phone

It hit me at 3.20pm on Sunday, as fans started to pour out of the stadium that the game had been cancelled. The end of season party was over and the 3,500 travelling Bournemouth fans fell silent, the inflatable beach balls were popped. This wasn’t how it was meant to end.

At 8.00 a.m. that morning we’d left Bournemouth in eager anticipation at seeing the mighty Cherries playing at the Theatre of Dreams. We’d a five-hour road trip ahead of us but it didn’t matter, Bournemouth were safe from relegation and we were going for an end of season party. Nothing could spoil our day, or so we thought!

The sun was shining in Manchester as we approached the ground just before 3.00 pm. From outside the ground all we could hear was the Bournemouth fans in full song, the stadium was ringing with, “We’ve got more fans than you”. It should have registered then that something was afoot as our fans should have been, outnumbered 20 to 1.

secure-old-trafford-pic-1

We passed through the gates with security guards searching us, our tickets checked and only then were we permitted into the ground. After a quick pint we made our way out from the bowels of the stadium to our seats. Nothing could prepare us for the sight before us. The pitch was empty, no ground staff or player’s warming up, the North and East stands in front of us were completely empty, with not a soul in sight. Only the West stand contained any United fans and of course the 3,500 noisy Bournemouth fans in the away end. Something was wrong, very wrong!

secure-old-trafford-pic-2After a couple of minutes at our seats we noticed the arrival of sniffer dogs and security personnel scouring the other side of the stadium, which suggested something serious. However the regular communications over the tannoy system and with stewards close by keeping us updated as they got news, the fans felt assured they were in good hands.

When the announcement finally came that the game had been abandoned due to security concerns fans were efficiently ushered through the nearest exit and away from the stadium by the security personnel outside.

What Manchester United demonstrated on Sunday, was meticulous planning that had gone into dealing with a security breach. They had the reporting procedures in place, lines of communication open to ensure constant updates were available, resources in situ to manage developments at a local level and if needed, they knew how to quickly close down operations to avert a disaster. Whilst I was irritated spending nearly 11 hours in total travelling and not getting to watch the football, you had to wonder at the efficiency shown by Manchester United.

It begs the question, what processes, procedures, lines of communication and disaster planning does your business have in place to cater for a security breach. Can your company demonstrate Premier League standards to your customers? Does your organisation have the visibility it needs to know when there’s a problem and are your processes robust enough.

If you’re not sure, then the answer is probably no. This week we’re offering 10 free audits to assess whether your current file transfer strategies and technology is up to Premier League standards. So if you’d like to stay in the game, when all around you are trying to knock you off top spot, speak to one of our data security consultants today on 020 7118 9640.

Secure File Sharing at the Local Government Strategy Forum

Secure File Sharing at the Local Government Strategy Forum

Heythrop Park, April 12th – 13th

This month I attended my second Local Government Strategy Forum, at the beautiful Heythrop Park Resort in Oxfordshire. Invited by our partner Maytech, I was the ‘independent industry expert’ and had the pleasure of spending two days in this lovely environment, talking with senior management and C-suite executives from councils all around the UK.

heythrop-park

Before attending these events I had, what I believe to be a commonly held opinion, that council workers were underworked and overpaid. I’d read all the stories in the local press about the six-figure salaries and the cancellation of services to ensure their lavish lifestyle. However I’d never stopped to think what they actually did. Listening intently at these events has given me a small insight into the workings of councils, and whilst I’m sure there is still more efficiencies to be realised, I couldn’t have more admiration for the wide range of services they provide and the challenges they have prioritising them to balance the books.

The financial challenges being faced by councils has lead to them adopting a more business-like approach. They are looking at every aspect of their business to drive out wastage and streamline operations, and that’s where my expertise came in.

John Lynch, CEO, Maytech – presenting Quatrix on day one

Over the duration of two days I spoke with in excess of 50 delegates about their data sharing, collaboration, secure file transfer and business process automation challenges. Our experience in this area, working with council’s such as Cambridgeshire County Council, North East Lincolnshire Council and most recently Mid-Sussex Council, ensured we already had a view on some of the challenges being faced for data sharing in the public sector.

As ever there’s not one technology, which addresses the wide range of data sharing requirements of councils, our council customers are using solutions from five of our suppliers. The service we provide is to help them to fully understand their requirements and then choose the right solutions for their needs and budget.

If your council or company needs to address its file sharing, collaboration and secure file transfer requirements why not download one of our free resources below:

What is Managed File Transfer?

Managed File Transfer Starter Pack

Comparison Guide

Building a Business Case for MFT

First European CFTP course held in London

First European CFTP course held in London

This Certified File Transfer Professional (CFTP) training program prepares IT staff to work with managed file transfer (MFT) technology and operations. The CFTP classroom experience consists of presentations, practical labs and discussions which invite an open exchange of ideas and opinions around many areas of file transfer.

On February 2nd and 3rd 2016, Pro2col Limited hosted the classroom based CFTP course in it’s London offices – the first time it has been held outside of America. The course attendees came from both the UK and continental Europe, and all had experience from different File Transfer products which resulted in a blend of skills and experiences between them.

Day 1

CFTP Training Course_SmlOn the first day the class looked at the different concepts used in file transfer and the methods used to encrypt data or perform integrity checking. The practical labs allowed the class to get a real hands-on feel for encryption by practising symmetric and asymmetric Public Key Infrastructure exchanges; hashing, salt and rainbow tables were also explored in depth.

The various protocols used to transfer files were next on the agenda and included everything from basic FTP to the more advanced topic of accelerated data transfers. Combining the learning from protocols and encryption helped to facilitate understanding of how to perform AS1,2 & 3 transfers and de-mystify it for some of the attendees.

Day 2

Following a review of the first day’s activities, on the second day of the course the class looked at the concepts around the “synchronisation” or “Ad-Hoc” transfers, an increasingly important aspect with modern MFT tools. Drawing again on the encryption lessons from day one, the class went on to look into encryption at rest (PGP etc.) before moving on to file transfer operations. This included troubleshooting of transfers and a look at how Service Level Agreements are applied and managed in File Transfer.

CFTP Small LogoFinally, the only thing left to do was to take the exam. Delegates were given ninety minutes to complete a random computer generated selection of questions and they needed to correctly answer a pre-set percentage to pass.

So, how did they do?…………..All Delegates Passed!

 

The European CFTP programme continues with the next London based 2-day accreditation course scheduled for May 2016 and online training courses available anytime.


CFTP Certificate

“This was a very good training event and the trainer, Richard, was extremely knowledgeable. I would definitely recommend CFTP to IT professionals working or considering a project with MFT Software.”

Matt Scotney-Jones
Regional Channel Manager – UK & Ireland at Ipswitch, Inc.

Our USA trip to visit file transfer vendors: Cleo & Ipswitch

Our USA trip to visit file transfer vendors: Cleo & Ipswitch

An insight into more than just product roadmaps…

USA FlagLast week was my first trip to the USA, which is all the more surprising as almost all of our suppliers are based there.  The weather reports leading up to the trip had forecast a snow storm making its way up the East Coast, dumping three foot of snow on New York and closing down the city.  Fortunately, we flew into Chicago around the storm. It was still bitterly cold though; it didn’t get much above freezing whilst we were there and don’t get me started on the windchill factor!!

CHICAGO, IL - JANUARY 28

I was joined by my colleague Mark Allen, one of our senior technical consultants, as we visited two of our strategic vendors. Thankfully, Mark provided the technical insight and expertise leaving me to cover the subjects I’m more comfortable with; industry knowledge and business functions.

We met quite a lot of people in our week long trip. Literally everyone I spoke to was happy to take the time to talk to you, was interested and engaging. The customer service I experienced was outstanding, everywhere.  One hotel worker even gave us a lift to a local steakhouse; not the ‘hotel recommended’ option but a little joint down the road. Needless to say the ribeye was excellent. I think the UK could learn a thing or two here, and it’s certainly made me think differently about our own services.

Cleo File TransferThe week started at Cleo.  We’ve only been working with them for a year or two, but what a company. Their Cleo Integration Suite, provides companies with a common platform for all of their data integration requirements. Going further than just pure MFT, Cleo provides A2A, B2B, EDI, Secure Email, Collaboration and a High Speed Protocol for Big Data. The single instance aspect shouldn’t be underestimated and is a true differentiator for Cleo. Being able to audit and report on all data transfers into and out of the business through a single point is powerful and I’d have to say that Cleo is probably one of the strongest Enterprise offerings within our portfolio.

We spent some time at Cleo’s Sales Kick Off, learning about the upcoming product roadmap and gaining further insight into the current product offerings. Since Cleo was acquired in 2012, the company has made considerable progress. Not only has the software platform evolved from being predominately focused on EDI, but it is now a strong contender in many other areas. They’ve add to their development and support capabilities by opening up an office in Bangalore, enabling them to provide true follow the sun support. Cleo is going to be a vendor to watch in 2016 and beyond. No longer a follower but a serious player with an excellent product and increasingly strong offering.

Chicago winter stormWe left Chicago for Boston to visit Ipswitch, already a major player in the MFT space and a key vendor for Pro2col. We’d already met most of Ipswitch’s Executive team on their trips to Europe, but this was a chance to meet them on their own turf and get some deep insight into the product roadmaps for 2016 and beyond. The sessions proved to be very insightful.  A lot of what we saw was shared under Non-Disclosure, but I’d say the coming year is looking very interesting for Ipswitch. The addition of a further product to the file transfer portfolio will enhance their MOVEit offering further still and we also spent some time revisiting MessageWay, their Enterprise offering.

Ipswitch_Offices_Snow_CroppedWhilst in Boston, Ipswitch took the opportunity to draft me in to make a brief video for their Virtual Summit in March. Dial-in to find out what’s great and good about Ipswitch in 2016 and you may be lucky enough to see a video of yours truly telling potential partners how great Ipswitch are to work with.   There again, that might be reason enough not to check in!

My first trip to the USA was a resounding success, even though I suffered man-flu (a very serious strain) and it was so cold. I thoroughly enjoyed it. There’s no doubt we’ll be back later in the year to check in with a couple of our other strategic vendors, I just hope it’s a little warmer next time.

A Fantastic Year & A Mystery Christmas Hamper

This year has been one of significant change at Pro2col, I’m glad to say all for the better.

We’ve added three people to the team this year; two senior technical consultants, Mark Allen and Richard Auger, both of whom have considerable experience with managed file transfer (MFT) expertise. Mark joined us having spent time working with Ipswitch and spent the last 18 months consulting on MFT systems and Richard came all the way from Australia to join us having spent the last 12 years managing MFT systems for Philip Morris! Chantelle joined us to become our Office Manager and bring greater efficiency to the team.

 

mystery christmas hamper office

Richard, Michelle & Ailsa with the hamper outside the office.

After six long years in an office that wasn’t big enough we moved into a brand new, purpose built unit, two doors away in December. The team couldn’t be happier and it now means we’ve got room to accommodate customers and suppliers for meetings. Although it’s quite likely we’ll still head to the Captain’s Club overlooking the River Stour at Christchurch on a nice day.

It’s been a great year for business too. By adding to the team we’ve increased our capabilities exponentially, which has enabled us to work on some extremely interesting opportunities with some fantastic companies. All in all, as the owner of the company I couldn’t have asked for much more (except a Lotto win perhaps).

I’d like to put on record my thanks to my exceptional team who’ve helped to make this year such a special one in the evolution of the business. In addition I’d like to thank our suppliers who’ve helped us to drive significant growth this year. However, I have a question to my suppliers. Which one of you very generous people sent us a hamper with the following message?

“Thank you for all your support. We appreciate your business. Have a great holiday season and our best wishes for the New Year!”

After trying the tracking number and scouring the packaging, we’re still none the wiser who sent it to us. Whoever you are, thank you we’ll enjoy all the treats.

hamper

To all our customers, thank you for your business in 2015. We’re striving to improve our services again to you next year, and have some exciting announcements to make in the coming weeks.

I hope you all have a Merry Christmas and prosperous New Year.

James