Here at Pro2col we’re increasingly being asked by our clients to help them move large data sets. The amount of data as everyone knows is increasing in size, as are file sizes. It is now common in our discussions to talk about files of many Gigabytes in size. The challenge this presents however is how to move the data from point A to B as invariably we’re finding that companies are needing to move these volumes of data halfway around the world. Welcome to Extreme File Transfer! Extreme file transfer is an expression which has become more widely adopted in recent times, but what does it mean? IDC describes it as;

“Extreme file transfer requirements come from the need to solve problems around file size. In this case, the file may simply be too big and the target too far away to reliably deliver over TCP/IP because of shortcomings in this networking protocol. In other cases, there is a problem delivering a file within an allowed time window, and therefore, there is a need to find an alternative approach.” [“IDC competitive review of MFT Software” October 2010]

IDC pretty much hits the nail on the head here, although I’d place a little more emphasis on the infrastructure over which extreme file transfer takes place. The efficiency of TCP based file transfer protocols such as FTP reduce dramatically when the round trip time between client and server increases due to latency. The result is an increase in packet loss and rapidly decreasing throughput. For file based workflows there are some great solutions available which addresses these issues. Vendors have taken various approaches such as breaking files into smaller bits, using hardware acceleration, compression, synchronisation, shipping a HDD but the most effective are those that utilise a variation of the UDP protocol. UDP itself isn’t a very successful protocol but when some controls are added it becomes the most effective way of moving extremely large files over extremely challenging connections. These re-engineered protocols form the cornerstone of the Extreme File Transfer solutions. The vendors in this space have designed their solutions to enable the use of the extreme file transfer protocol for these typical business requirements:

  • Disaster recovery and business continuity
  • Content distribution and collection, e.g., software or source code updates, or CDN scenarios
  • Continuous sync – near real time syncing for ‘active-active’ style HA
  • Supports master slave basic replication, but also more complex bi-directional sync and mesh scenarios
  • Person to person distribution of digital assets
  • Collaboration and exchange for geographically-distributed teams
  • File based review, approval and quality assurance workflows

If your business needs to transfer extremely large files or volumes of data speak to our team of expert consultants. As independent file transfer specialists since 2003 we’re able to provide an objective vendor agnostic view to finding the right solution for your file transfer needs. Speak to one of our consultants now on 0333 123 1240.