Managed File Transfer Automates Cloud Workflows

Managed File Transfer Automates Cloud Workflows

Companies are facing an opportunity to redefine their business and operational processes. Cloud technology has been proven in terms of scale, reach, reliability and security – and businesses are consuming an ever-increasing number of public and private cloud services as they strive to reduce CAPEX and operational overheads.

A recent Ipswitch survey of 500 IT professionals showed that a significant majority of businesses, across almost every industry sector, have adopted cloud file share services to increase the productivity of individuals, teams and workgroups.

 

cloud-workflows-yes-or-no

Source – Ipswitch Report Data Security Compliance, 2016

The next stage in this transition is to use cloud storage sites to automate business workflows. Using cloud storage in this context though presents a significant number of business challenges. Business data must be secured to the appropriate level.   And, data must be available to business critical workflows when needed. In general, cloud services, take data security extremely seriously and provide adequate protection. But how to automate files moving to and from the cloud is typically left for the customer or a technology partner to figure out.

Step forward Managed File Transfer!

Managed File Transfer (MFT) lends itself to resolving the ‘ground to cloud’ and back again conundrum. Built to interface with various endpoints, supporting open standard protocols, MFT solutions are built to automate business processes and an increasing number of MFT vendors offer interfaces to major cloud storage sites.

Cloud storage vendors have various ways that enable technology partners to interface with their services. Automated processes primarily use secure FTP or a public API provided by the cloud vendor.

A number of Managed File Transfer solutions now allow the delivery to and collection of files from an increasing number of cloud storage sites including:

  • Amazon S3
  • Citrix ShareFile
  • Dropbox
  • Google Cloud
  • Google Drive
  • Microsoft Azure
  • Box
  • OneDrive
  • Oracle Cloud

The benefits of automating the integration of cloud storage into business processes are clear. Reduced manual intervention, near to real-time file based workflows, notification and audit trails are some of the most obvious, reducing processing time and therefore providing significant ROI. Whilst the same could be achieved by orchestration, Managed File Transfer provides a competitive, feature rich, competitively priced option for file-based workflows.

As demand increases, the level of complexity and depth of integration is extended. The ability to write files directly using vendor-provided APIs(e.g., directly to Amazon S3 buckets or Microsoft Azure blobs) is becoming more widely used.

Now that the cloud is a mainstay of a business’ extended infrastructure, companies are looking to consolidate their services with a cloud storage provider requiring cloud-to-cloud migration. Where consolidation isn’t possible Managed File Transfer solutions are facilitating cloud-to-cloud workflows, so that businesses can link together best in class cloud technologies.

The cloud is here to stay and will become an increasingly important piece of the overall infrastructure landscape. Moving data to it, from it and between them will be as important as sending and receiving data directly with trading partners. Fortunately, a consolidated view of all business data transfers including the cloud is possible – once again Managed File Transfer is leading the way.

If you are currently using cloud storage or are considering adding it to your business workflow, drop us an email to [email protected] or call on 020 7118 9640. We can recommend which MFT solutions will best meet your needs.

Get Your Free Copy Of Our Brand New Book,

The Expert Guide to Managed File Transfer[transparent background] Small

 

“THE EXPERT GUIDE TO MANAGED FILE TRANSFER”

 
 

The Expert Guide distills our knowledge from more than 700 Managed File Transfer
projects into a resource to get your project started.

It includes;

  • What is Managed File Transfer, EFSS, Big Data, High Availability etc?
  • What problems Managed File Transfer can solve? With real life use cases.
  • 40 key considerations for a successful Managed File Transfer project.
  • A side by side comparison of eight leading Managed File Transfer solutions.


Click Here Free Copy_Orange_#db5500

Top 5 Tips for Reducing your Managed File Transfer costs

Top 5 Tips for Reducing your Managed File Transfer costs

Secure, automated, robust b2b workflows are key to businesses, frequently a vital system in the creation of customer value. However, Managed File Transfer solutions can be expensive, very expensive. If business continuity is critical, solutions may include high availability within their primary business locations and remote disaster recovery.  Coupled with pre-production and test environments, the Managed File Transfer landscape can become considerable with software investments reaching in excess of £100k in the first year.

Driving Cost Down and Functionality Up

There are ways to keep the cost down, providing you with the functionality required on a smaller budget. Here is a list of considerations to make your budget go further.

 

Do Your Research To Find The Right solution

Many IT professionals have a go-to technology that they’ve used before, or that has been recommended to them. Typically, it will be one of the better-known brands. These are proven systems, but it is wise to conduct thorough research to ensure you’re not backing the wrong technology from the outset. With such a large number of vendors and functionality largely the same, its not what they do but how they do it that’s important.

Upgrade What You Already Have In Place

It’s very likely that your organisation has an FTP server already.   In fact, if you dig around there’s a good chance you’ve got a lot more than just one. Many of today’s Managed File Transfer vendors started out by developing FTP server software. As business requirements have changed further protocols, enhanced reporting, analytics, automation engines, to name just a few have been added. It is definitely worth considering what upgrade paths exist to get you to your goal.

Only License What You Need

This might seem obvious but trust me, it’s not. Many vendors have developed further capabilities around mobile, ad-hoc transfer, API’s, dashboards, which you may not need right now. The bundled offerings provided by vendors look like a good deal, promoted as saving you ‘x’ percent over purchasing the modules separately. You may end up licensing software that you are unlikely to use, resulting in a wasted investment.

Another aspect is how many trading partners you are likely to communicate with. An unlimited licensing model isn’t appropriate for all companies and in fact some vendors license to a granular level by the number of connections you require. If for example you’ve only got 25 trading partners, you can get a really feature rich solution at a fraction of the cost. A comprehensive understanding of how different vendors price up their solutions can save you many thousands of pounds over the lifetime of the solution.

The Cloud Has Changed Everything

The rise in the development and adoption of cloud/SaaS solutions has been meteoric in recent years. Newer software companies are typically agile and have programmed their systems using newer languages, allowing much more rapid delivery. By developing for a single, hosted environment, new features can be added quickly in response to demand, and the QA process is much quicker. However, what is key here is the payment model. Subscription services tend to fall under OPEX expenditure, softening the blow of a significant CAPEX investment up front. Some on-premise software vendors have responded by offering subscription models for their software.

Reduce Your IT Support/Management Overheads

A recent study by the Aberdeen Group showed that 65% of companies purchase Managed File Transfer to improve productivity. Like any system though, these savings can only be achieved from a complete, correctly scoped and maintained solution. Time and time again, we see responsibility for MFT solutions falling between networking and security teams and the many lines of business.  The result is no-one takes ownership of the solution, businesses fail to take advantage of its capabilities and the net result is that it doesn’t deliver the potential ROI.

In this instance, I’d recommend a managed service, providing you with expert resource available at short notice to help onboard trading partners, configure interfaces and much more. Of course (shameless plug coming up) this is something our Managed File Transfer support team can help you with.

 

In addition to the key points above, there are a variety of other ways in which working with a Managed File Transfer consultant, could ensure you get the best solution for your needs, whilst maximising the value.

 

  • Complete a requirements analysis – for example our project guide has over 230 scoping questions
  • Design your ideal solution and use this to form the basis of your vendor selection
  • Minimise your professional services costs by working with a team, like us, that have been through the process hundreds of times before
  • Don’t recreate the wheel, make use of existing documentation and processes out there and available to you
  • Verify solution capabilities and pricing with independent experts

Get Your Free Copy Of Our Brand New Book,

The Expert Guide to Managed File Transfer[transparent background] Small

 

“THE EXPERT GUIDE TO MANAGED FILE TRANSFER”

 
 

The Expert Guide distills our knowledge from more than 700 Managed File Transfer
projects into a resource to get your project started.

It includes;

  • What is Managed File Transfer, EFSS, Big Data, High Availability etc?
  • What problems Managed File Transfer can solve? With real life use cases.
  • 40 key considerations for a successful Managed File Transfer project.
  • A side by side comparison of eight leading Managed File Transfer solutions.


Click Here Free Copy_Orange_#db5500

The Growth of Mobile Devices in the MFT Workspace

The Growth of Mobile Devices in the MFT Workspace

Background

android-and-iphone-iconsIn the last ten years, there has of course been a massive growth in the realm of mobile devices, specifically with regards to smart phones and tablets. It now seems strange to think of life without them, but iPhones only appeared in 2007, and Android in 2009.
Of course, before then we had PDA devices that allowed us to sync with our desktop computers and tap away at a tiny Word document while sitting on a train, but the whole idea of transferring data while on the move was very much in its infancy still.

Smartphones

We all know, as the market grew for mobile devices, so did the range of applications available for them. One common factor for all mobile devices is the ability to browse the internet; it is this perhaps that first made people stop and think about how they could use a mobile device to access their company Managed File Transfer (MFT) site. In my case, I would download a list of support phone numbers from our MFT system whenever I needed to contact a technician from another team.

MFT specific clients

Before long however, specific mobile device client software from the MFT vendors began to appear, simplifying the process of downloading files. As is often the case, where one vendor releases a new product, it is soon available from all of the vendors; now it seems that every product has a mobile device client available.

Much like the MFT systems that they support, not all clients are equal. If you are planning on selecting an MFT product on the basis of the strength of the mobile client, here are some thoughts for you to consider:

  • Does the client open files in the right application automatically? Or do you need to save them first?
  • Can the MFT system enforce a PIN on the device? Remember that confidential files could be in the wrong hands if the device is stolen.
  • Can you remotely wipe files on the device if required?

Being able to remotely download a file and work on it is certainly very useful, however equally important is the ability to upload files; this is easiest to see when considering taking photos. All smartphones come with pretty good cameras; some MFT clients allow you to navigate to a folder, then activate the camera and store all photos into the folder automatically. Imagine a roving insurance agent or removals planner being able to have the images available in his/her office before they return; the possibilities for workflow management are endless.

Network speeds

However, we do need to remember the limitation that some networks bring; A report by Ofcom in late 2014 found that the average UK download speed for 4G was 15.1 Mbit/s (6.1 Mbit/s for 3G) and the average upload speed for 4G was 12.4 Mbit/s (1.6 Mbit/s for 3G). At that time, 98% of the UK was covered by 3G, but only 70% was covered by 4G.

Source: https://www.ofcom.org.uk/about-ofcom/latest/media/media-releases/2014/3g-4g-bb-speeds

Source: https://www.ofcom.org.uk/about-ofcom/latest/media/media-releases/2014/3g-4g-bb-speeds

To put that into context, that’s nearly a minute to download a 100MB file on 4G and just over a minute to upload it again; if you drop to 3G coverage you would need around nine uninterrupted minutes to upload the file…

It happened to me…

I remember a project I worked on where the salesforce would upload orders into MFT while visiting customers; inside the cities it worked fine, but once out into the countryside there simply wasn’t sufficient coverage. No one had considered the coverage during the planning phase.

Of course there’s always the wifi alternative; but somehow I would feel uncomfortable if I knew end-users were rushing to their local fast food cafes or shopping malls and using unsecured wifi networks.

Summary

So in summary, if you intend to use Managed File Transfer primarily for mobile devices, pay special attention to exactly what the client application provides you. However; pay even more attention to whether the mobile networks in the area you will use the devices can deliver the speeds you need to succeed.

Get Your Free Copy Of Our Brand New Book,

The Expert Guide to Managed File Transfer[transparent background] Small

 

“THE EXPERT GUIDE TO MANAGED FILE TRANSFER”

 
 

The Expert Guide distills our knowledge from more than 700 Managed File Transfer
projects into a resource to get your project started.

It includes;

  • What is Managed File Transfer, EFSS, Big Data, High Availability etc?
  • What problems Managed File Transfer can solve? With real life use cases.
  • 40 key considerations for a successful Managed File Transfer project.
  • A side by side comparison of eight leading Managed File Transfer solutions.


Click Here Free Copy_Orange_#db5500

Why Do File Transfer Systems Need a Health Check?

Why Do File Transfer Systems Need a Health Check?

A Managed File Transfer solution like many other IT systems, requires regular system maintenance to keep it performing at optimum efficiency. Luckily, most back end house-keeping tasks can be automated, but how the tool is configured may mean it’s not reaching optimal performance.inr_diagnosing_com

A health check can uncover bottle necks, security issues, performance gaps and any inconsistencies with best practice.

Configuration

Many MFT vendors pride themselves on the fact that their solution can be installed by any IT professional. However, this is only half the story. There can be hundreds of security settings inside the applications and knowing the implications of turning a particular feature on or off is not always obvious. Many of the systems have a set of default settings, which are a vendor’s view of compliance. These may not align with an organisations security policies, local legislation or particular client SLA’s.

For example, checking the environment an MFT solution is running on may lead into the following questions being raised; “Is the server being backed up?” or “Is the server running under the correct account credentials?” You might be surprised at the number of systems which are not.

Performance

MFT solutions in larger organisations may handle many thousands of transfers every hour in extreme cases; as a result ensuring the system is as lean as it can be is vital to meet increased demand. Many of the systems we look at have old, out of date, or even test processes configured which, while they may not move data, clutter up the systems or constantly fail, which can skew KPI reports.

In addition, removing unused or expired accounts and data can help free up capacity on the server. Workflows may be created when a system is installed, altered over time or duplicated. It may be possible to rebuild workflows to reduce complexity making them easier to troubleshoot. Other settings that frequently need changing are security protocols such as TLS or authentication sources. For initial implementation, the system may not have been connected to the organisation’s Active Directory for example, but now the number of users has grown, this would improve security.

Software version

MFT systems are typically much more reliable than legacy file transfer systems built into operating systems. File transfers only tend to fail, when the remote server has an issue or if someone changes the network. Combine this with the “if it’s not broken, don’t fix it” school of thought and the systems are often not looked at or kept up to date with the latest versions and patches. As vendors develop their solutions, new features are introduced and issues addressed. Sometimes these can then be leveraged to significantly improve either the performance of the system, or can add functionality offloaded elsewhere. A vendor maintenance agreement, is only valid if you are running a supported version of software, which can leave an organisation exposed if they have a major issue.

Keeping the MFT system software up to date means that the system can benefit from all the latest features and security patches.

Knowledge transfer

System Managers move on to new roles and are replaced. Often there is some handover of what the system will do and how to perform common operations on it. However, this is not always a complete training course and can be prone to mis-perceptions being propagated.

At installation time, several features may not have been activated as they were not required for initial deployment; however it may transpire that these features address needs that have subsequently appeared. Utilising features the MFT solution has can help the system to perform better and may allow the organisation to get more return on investment.

Is it future proof?

As an organisations’ requirements change over time, the initial installation and configuration can become inappropriate to fulfil the organisation’s needs. Preparing your system for either High Availability or Disaster Recovery implementations may not be as straight forward as installing new nodes. A health check at this point may throw up issues such as database locations or firewall problems, which if dealt with before any configuration changes, can make the whole process go much smoother.

 

A good regular health check ensures that your system is running efficiently and effectively. As Managed File Transfer becomes mission critical, a health check ensures an organisation isn’t taking any unnecessary or unknown risks.

Get Your Free Copy Of Our Brand New Book,

The Expert Guide to Managed File Transfer[transparent background] Small

 

“THE EXPERT GUIDE TO MANAGED FILE TRANSFER”

 
 

The Expert Guide distills our knowledge from more than 700 Managed File Transfer
projects into a resource to get your project started.

It includes;

  • What is Managed File Transfer, EFSS, Big Data, High Availability etc?
  • What problems Managed File Transfer can solve? With real life use cases.
  • 40 key considerations for a successful Managed File Transfer project.
  • A side by side comparison of eight leading Managed File Transfer solutions.


Click Here Free Copy_Orange_#db5500

How do I monitor my Managed File Transfer system?

How do I monitor my Managed File Transfer system?

As we all know, any computer system tends to merge together multiple products, often from different vendors. These products may encompass scheduling, user authentication, monitoring, database and a whole host of others. Finding one unified product can sometimes be problematic, as very few vendors offer a lock, stock and barrel solution.

monitoring_a_managed_file_transfer_system

With regards to a Managed File Transfer (MFT) solution, most products contain a dedicated reporting component, available as either an integral part of the solution or as an additional module which can be purchased and installed separately.

However, when it comes to event monitoring, the majority of solutions have kicked the ball over the fence and instead have taken the approach of providing some simple methods to alert operators or administrators of potential problems. In this article, I’ll explore some of the ways that you can use these interfaces to best suit your needs.

Before you even consider how you want to interface your MFT to your monitoring, you need to take a long look at whether in fact something is worth monitoring. For example, would you want to be alerted when someone fails to login to your FTP server? If it’s a wannabe hacker and their IP address gets automatically locked out, then probably not. If it’s a production batch account, then probably yes. Think about your MFT system in component pieces and judge each part on its own merit. Just because you monitor some of it, you don’t have to do it all.

The Problem with Email

One of the easiest monitoring methods is to generate an email when something goes wrong; unfortunately this is also one of the biggest areas for monitoring failures, due to a couple of reasons.

First, relying on email does not preclude a failure or delays in your mail system. Emails can potentially get lost or marked as spam by the mail server if enough are generated. Secondly, if you are only notified of failure, but you don’t receive any emails, is your system working?

Simple Network Management Protocol

SNMP is a protocol designed for monitoring a network and its various devices. There are several monitoring solutions commercially available, however you need to check your MFT system to determine if it is able to create an SNMP trap; if not, you are limited to just monitoring the MFT server(s).

Log Watcher

Most monitoring tools contain a log watcher of some description. The monitoring solution can be set to read your log files on a regular basis and will generally remember which parts of the log have already been read. An alert is raised when a certain regular expression is encountered in the log file.

Be careful when using this approach that you do not inadvertently change the log levels of the MFT solution and that error text does not change with software upgrades.

Event Log

Some MFT solutions allow writing to Windows event logs, which you can then monitor with any commercial monitoring solution. On a Linux or Unix system, you would perhaps be checking the /var/log directory (system logs are written to /var/log/messages).

Database

If your MFT solution writes log records to a database, use a query launched from the monitoring solution to routinely extract error events. Depending upon the frequency of execution, this can give near real-time results.

And Finally…Scripting

If your MFT product provides an API, why not use some scripting to generate events? A Cron or Windows scheduled task can routinely check directly into your system for noteworthy events.

Past the Interface

Now that you’ve worked out a way to get the events from your MFT system into your Monitoring solution, you need to consider how you want to be alerted. Of course, this is the responsibility of the monitoring solution, but consider how you would like to grade the events that you receive. Do they all require your immediate attention, or can you apply a priority to some, while others can wait. In practice, it makes sense to prioritise events before passing them to the monitoring solution.

Reactive recovery

Whichever method you use to pass events to the Monitoring tool, you may find that you also have the opportunity to execute certain activities when you detect an issue. Many monitoring tools possess this functionality (If you script your monitoring interface, this can be used too). A good example of this may be to restart a failed interface, or enable an alternate workflow.

Find Out the Top Ten Signs your MFT Software Needs a Health Check!

 

Top 10 Signs

Like any computer system, your managed file transfer solution needs to be kept well tuned in order to provide the best performance.

Learn to recognise the top ten signs that your Managed File Transfer solution needs a health check.

Download our guide now; ‘10 Common Signs your Managed File Transfer Solution Needs a Health Check‘.

Pro2col’s engineers are trained and accredited to provide a comprehensive solution health check of your managed file transfer software and environment. For more information get in contact today by calling our team on 020 7118 9640.

Do you need Managed File Transfer or just FTP?

Do you need Managed File Transfer or just FTP?

Secure File Transfer

FTP and SFTP servers perform 2 basic tasks:  “Put” and “Get.” You can put files on the FTP Server or get files from the FTP Server. If security is not a concern, FTP Server software is an easy and inexpensive way to accomplish this.

Typical use cases include backing up your Cisco Unified Call Manager (CUCM) or other applications, receiving non-sensitive data from business partners or enabling remote employees to upload non-confidential reports.

However, file transfer requirements have changed considerably since FTP was released back in 1971. End users are more aware of the risks of identity fraud, business partners require at least the base security levels outlined in ISO 27001, file sizes and volumes have grown exponentially and business workflows are ever more complex.   Add in the legal requirements of GDPR and UK Data Protection Act, the fines that can be levied by the ICO and the economic impact of a data breach and an unsecured “put/get” system fast becomes a liability.

 

Managed File Transfer (MFT)

Research conducted by Vanson Bourne, on behalf of IBM last year, identified the top three concerns that drove the implementation of Managed File Transfer as being security, complexity and growth.

The top three drivers for choosing MFT over FTP

Top three drivers for choosing MFT

Security

When security and compliance are high priorities, MFT solutions do more than simply secure files in transit. Standard security features in MFT solutions include:

  • Support for secure protocols and refusal of unsecure connections
  • Encryption for stored data and the assurance that unencrypted versions of the file are never written to the server
  • Perimeter security, such as a reverse proxy that operates as a pass through and does not temporarily store data
  • Support for the current versions of privacy standards, such as PCI v3.1 and HIPAA
  • The ability to support security policies, such as complex/expiring passwords
  • Hacking detection with automated shut down of offending users or domains

MFT solutions can enable PCI compliance for the management of credit card data, secure workflows for the sharing of confidential data from patient records to patents and present a secure interface to your customers and partners for system to system or person to person transfers.

 

Complexity

Files are transferred between applications, systems and end-users. The receipt of a file may trigger an entire workflow with complex what if routing and varied notifications. Automated “Push” and “Pull” technology, as well as the ability to automatically sort data and send to pre and post processing applications, is a key driver in the need for MFT.

  • Event-driven commands and notifications/alerts (e.g., “on file upload, do…”)
  • API or command-line tools
  • Integration with key systems such as Active Directory, Outlook, Sharepoint, Salesforce, anti virus and Data Loss Prevention solutions
  • Flexibility to handle differing customer requirements such as protocol, password protection and PGP.
  • Full reporting and notification on failures enabling prompt action to meet customer SLAs.

MFT is often described as the glue between different systems and elements of the business, linking processes and people, enabling information flows but confirming security standards are met. Most can be administered via a web based GUI, removing the need for scripting expertise.

 

Growth in File Size and Volume

Thanks to new software applications, increased storage capacity and a fundamental shift to online working, both the volume and size of files has grown exponentially. And the growth rate is just getting faster. It took 51 years for hard discs to reach 1TB and only a further 2 to reach 2TB. Our customers now regularly need to transfer files of over 5GB and frequently much more. With its lack of compression and checkpoint restart, FTP is not designed to manage these large file sizes. Lots of our customers are also struggling with the degree of manual intervention required by IT to set up new transfers via their FTP server and the lack of self service options.

  • EFSS options for end-users to transfer files under policy controlled conditions
  • Modules for file acceleration or UDP based protocols now available for many MFT solutions
  • Self service options for end-users
  • Web clients to simplify the movement of files over 2Gb via HTTP(S)
  • Auditing to ensure compliance and a full view of who is sharing what.
  • Automation of workflows to reduce the need for manual interventions.

 

MFT v FTP

So do you need Managed File Transfer? If you handle sensitive data, need to be PCI compliant, work with partners who demand security, have more transfers than you have man hours to handle, are juggling varied demands from different parts of the business or simply have to move increasingly large files, I would suggest MFT would be of benefit. As not all MFT solutions offer the same functionality, it’s important to determine the goals for your implementation and understand how the MFT functionality will be used before you start your product research.

Get Your Free Copy Of Our Brand New Book,

The Expert Guide to Managed File Transfer[transparent background] Small

 

“THE EXPERT GUIDE TO MANAGED FILE TRANSFER”

 
 

The Expert Guide distills our knowledge from more than 700 Managed File Transfer
projects into a resource to get your project started.

It includes;

  • What is Managed File Transfer, EFSS, Big Data, High Availability etc?
  • What problems Managed File Transfer can solve? With real life use cases.
  • 40 key considerations for a successful Managed File Transfer project.
  • A side by side comparison of eight leading Managed File Transfer solutions.


Click Here Free Copy_Orange_#db5500