FileCatalyst FAQ - managed file transfer for automation, acceleration and workflow

Frequently Asked Questions

1. What is FileCatalyst
2. How does FileCatalyst accelerate file transfers?
3. How does On-the-Fly compression help FileCatalyst?
4. Which protocols are used by FileCatalyst to transfer files?
5. What is the UDP-based protocol used in FileCatalyst?
6. Why is the FileCatalyst UDP protocol better than regular FTP, HTTP, or other TCP based transfers?
7. How does FileCatalyst use multiple TCP streams to transfer files?
8. What is delta transfer and how does it help?
9. How does FileCatalyst compare to a WAN accelerator appliance?
10. Can FileCatalyst be used in conjunction with WAN accelerators?
11. Why is FileCatalyst better than other UDP-based file transfer solutions?

1. What is FileCatalyst?
FileCatalyst is a software solution that supports a client/server architecture to transfer large files. It is a software-only solution which circumvents the need to add special hardware or make bandwidth upgrades. FileCatalyst uses advanced technology to yield exceptionally high throughput, regardless of network conditions. In the end, this results in faster file transfers between remote locations even when there is high network latency or packet loss.

2. How does FileCatalyst accelerate file transfers?
FileCatalyst uses multiple techniques to yield exceptional results; in many cases, transfers are much faster than actual line speed. These techniques include:

FileCatalyst UDP-based protocol: FileCatalyst uses a patent-pending UDP-based protocol that is faster in transferring large data sets when there is high latency or packet loss over the network
On-the-fly Compression: FileCatalyst compresses the data it sends over the network in real time, meaning there is no time-consuming compression or decompression at the beginning or end of each transfer. File sizes may be reduced by 50% or more, creating the illusion of transfers that exceed line speed.
Multiple TCP Streaming: In situations where UDP is not possible, FileCatalyst can achieve some measure of acceleration by running transfers through multiple concurrent TCP streams, along with on-the-fly compression.
Delta Transfers: If a file has changed since its last transfer, FileCatalyst can send only the “deltas” (incremental differences) of the file, rather than resending it in its entirety.

3. How does On-the-Fly compression help FileCatalyst?
On-the-fly compression allows digital files to be reduced in size as they are sent. It uses the same principles as WinZip, Gzip and other compression utilities. What differentiates FileCatalyst is that compression occurs as the file is being transferred, saving preparation time. As the files reach the recipient, they are decompressed and automatically stored in their original formats.

Less setup and teardown is required with on-the-fly compression, which becomes important when transferring a large number of files. Imagine the overhead involved in trying to send 1,000 files, as each file is individually created and closed by the server. Standard compression techniques interrupt the data flow, adding to the total time and making the file transfer appear slower. When sending one large archive, there is only one setup and teardown involved, which greatly speeds up the overall transfer process.

4. Which protocols are used by FileCatalyst to transfer files?
FileCatalyst uses the UDP protocol for data transport and TCP for control commands and retransmission requests. FileCatalyst also provides a secondary firewall-friendly transfer method that enhances the performance of TCP by opening up multiple concurrent streams of data.

5. What is the UDP-based protocol used in FileCatalyst?
The UDP-based protocol used in FileCatalyst is proprietary. It is a highly efficient, patent-pending, retransmission and congestion control mechanism that adds a reliability layer to UDP. The flow of data can achieve full line speed with an amazingly low 0.25% overhead.

6. Why is the FileCatalyst UDP protocol better than regular FTP, HTTP, or other TCP based transfers?
Both FTP and HTTP use TCP as the transport protocol. The inherent characteristics of TCP make it highly susceptible to network latency and packet loss. Even on a relatively stable network, TCP goodput is always lower than the actual available line speed. For example, on a T3 network (45 Mbps) with packet loss of 0.1% and a delay of 10 ms, FTP transfers can peak at only 30 Mbps. In sharp contrast, FileCatalyst yields goodput of 44 Mbps—only slightly less than maximum available line speed. When network conditions deteriorate to 2% packet loss and a delay of 150 ms, FTP transfers can be expected to perform at 450 Kbps, or 1% of the actual available bandwidth. FileCatalyst maintains its 44Mbps goodput. To view other comparable metrics of FTP vs FileCatalyst click here.

7. How does FileCatalyst use multiple TCP streams to transfer files?
Rather than reading and sending the file sequentially, multiple threads read from the file and transfer pieces over their own TCP streams. These pieces are received and reassembled on the fly by an equivalent number of receiver threads. There is no latency during reconstruction as the pieces are written using random access. The number of streams can be tuned to achieve the desired throughput. This method of transferring files is effective when network degradation is at a reasonable level. Due to the large number of concurrent threads that must run in order to sustain high speeds, this method is not as scalable as FileCatalyst UDP-based transfers.

8. What is delta transfer and how does it help?
FileCatalyst supports an advanced “delta” transfer algorithm. Once a file is transferred in full, any new revisions will only require these incremental changes to be sent rather than the whole new file. Imagine, for example, a large database file. Sometimes only small portions of the database are changed, such as a single name or location field. FileCatalyst calculates these modifications as "deltas" and transfers only the new data. At the destination, FileCatalyst automatically amends the changes, bringing the file up to speed with the source. This ensures that bandwidth usage is kept to a minimum and results in a very high effective goodput.

9. How does FileCatalyst compare to a WAN accelerator appliance?
When all data must be accelerated between all of an organization’s WAN nodes, a WAN accelerator appliance may be a good solution. Such devices do a great job speeding up CIFS, Email, and Web, as well as a reasonable job improving FTP performance—providing network conditions are not too severe. If the problem is not limited only to file transfer, a WAN accelerator may be an important part of your data infrastructure.

In contrast, FileCatalyst only accelerates file transfers; but it does this better than any other product. Being software-based, FileCatalyst is very easy to deploy, and requires no physical changes to your network. FileCatalyst can guarantee full line speed or higher, regardless of link conditions. The FileCatalyst Server-based pricing model makes it much more cost effective when the transfer scenario is many to one, or one to many. You can deploy one server and support thousands of end users with only one license.

In addition, the FileCatalyst software suite offers a host of application-level features that are simply not present when using other 3rd party file transfer applications along with a WAN appliance. Features like task and bandwidth scheduling, automatic reconnection and resume, on-the-fly compression and delta transfer. No other file transfer software does a better job of maximizing actual data throughput than FileCatalyst.

10. Can FileCatalyst be used in conjunction with WAN accelerators?
Yes. FileCatalyst can add to the value of WAN accelerators. WAN accelerators streamline all network traffic across a large area using techniques appropriate for a broad set of tasks. FileCatalyst on the other hand best optimizes file transfer, using proprietary technology built for the job. When used in conjunction, the two can actually complement one another.

11. Why is FileCatalyst better than other UDP-based file transfer solutions?
File transfer solutions that use UDP for transport lack the speed and scalability of FileCatalyst. Due to the nature of UDP itself, packet transmission is not inherently guaranteed. All UDP-based solutions must track individual data packets to ensure delivery. If a packet is lost, a message must be sent back to the sender to request retransmission. If this process is not handled efficiently, the performance will degrade substantially when packet loss is high. FileCatalyst’s patent-pending technology ensures a constant flow of data, never waiting for acknowledgement before proceeding, and always retransmitting concurrently with new data. The flow of data is constant—at the speed of your link.

In addition, FileCatalyst’s extensive list of applications blows away the competition. No other application combines UDP transfers with on-the-fly compression and delta transfer. FileCatalyst is the only application with Java applets for web-based transfer with no installation required. It is the only application that can run on every platform with a Java VM installed, without the need for special builds and without all the “gotchas” of other applications. FileCatalyst is the only application that can fill your pipe out of the box with no extensive tweaking required.