Photo: Jonel Hanopol License: Creative Commons
With shrinking production budgets and tighter deadlines, your time is more valuable than ever. When you’re moving large files through the client review process, you can’t afford to restart failed uploads or wait around for consumer services like Vimeo, Dropbox and Hightail.
We’re always looking for ways to refine Screenlight to better support the needs of video professionals. That’s why we’ve introduced a new uploader that makes large file transfers a breeze. With our new uploader you’ll enjoy more reliable uploads, faster transfer speeds and the ability to upload files of any size.
Accelerated transfer tools were once the rarified domain or large production companies with big budgets and custom workflows. These benefits are now available to all Screenlight customers at freelance friendly rates.
Want to know more? Read on.
More Reliable Uploads with Auto-Resume.
Our new uploader is super resilient. If you encounter a problem partway through uploading a file, we’ll resume the upload where it left off rather than forcing you to restart at the beginning.
The common approach to uploading files is to send each one in its entirety. This works fine for small files. However, uploading a large video file can take hours. During this time, the upload is susceptible to all kinds of interruptions. A prolonged network interruption can cause the entire upload to fail. Few things are more frustrating than having an upload abort at the end of a 5 GB transfer.
Our new uploader breaks each file into multiple parts and uploads several of these parts at once. If a connection problem causes one of the parts to fail, we only have to retry that particular part rather than restarting the upload from the beginning.
The practical benefit of this is that our uploader can automatically resume uploads. If there’s a network problem like your WiFi cutting out or your ISP running into problems, the upload will continue as soon as the connection is restored.
If your computer crashes or you inadvertently close the tab where an upload is in progress, you can go back into Screenlight and restart the upload by dragging the files back into the uploader. Once you do this, you’ll see the progress bar quickly jump to where the upload was left off.
Accelerated Uploads Mean Fewer Delays.
Super speedy video uploads are an important part of the video review and approval process. You don’t want a slow upload or encoding job getting between you and an eager client.
Our new uploader uses a cool bag of tricks to speed up large file transfers. We think you’ll be very happy with this.
Uploading parts in parallel speeds file transfers
Most uploaders, including our old one, send one stream of data to the destination server. If you’ve got a very good connection and are transferring files over a short distance you can get pretty good speeds. Unfortunately this isn’t always the case.
There are some issues with the TCP/IP protocol that create inefficiencies when transferring files. I’ve discussed these issues at great length in this ProVideo Coalition blog post.
“Basically, TCP/IP sends packets of data serially (one after the other). A new packet of data won’t be sent until the receipt of the previous one has been confirmed. If delivery isn’t acknowledged after a certain period of time, the packet will be resent. In the meantime, the other packets will essentially be queued up waiting their turn.
Further, the amount of data that can be sent in a packet varies based on the quality of the connection. The more congestion or data loss, the smaller the packets of data the protocol will send. This means a greater percentage of the transfer time is spent waiting when there are poor network connections.
The bottom line is that the greater the distance, the more congested the network and the more lossy the connection, the lower the effective transfer rate. ”
Our uploader gets around this by breaking each file up into lots of parts and then uploading several parts in parallel. If a part in one of the streams is delayed, the other four streams will continue uploading data while the packets in the disrupted stream are resent. An analogy is moving from a single lane highway to a multi-lane highway.
Transfer acceleration decreases the distance between your location and our servers
In general, the greater the distance, the slower the data transfer speed. When connecting over larger geographic distances, traffic makes more hops between different connections. Each hop increases the round trip time, or latency.
Example of network hops - non-accelerated
Most upload tools send files to a single central location. This is great for people that live nearby (usually in the US), but it isn’t the best experience for customers like ours who are distributed around the globe.
For several years, we’ve decreased the distance by looking up the IP address of each person that is uploading and then routing their upload to our storage location nearest them. For example, uploaders in Australia would transfer files to a storage location in Australia. With 8 locations worldwide, we were able to decrease distances pretty significantly for most uploaders. It’s been a pretty great approach, but we’re always looking for ways to improve the customer experience.
Example of network hops - transfer acceleration
Our new accelerated uploader takes advantage of a new service from Amazon Web Services. Uploads are now automatically routed to one of over 54 global edge locations. From there, files are sent over a private managed network to our central storage location. This private network is run by Amazon and is designed to manage throughput.
It looks like an extra step compared to our previous solution where files were stored regionally, but this transfer back to our home base is over highly optimized connections. Simply put, your data travels a short distance over the public Internet and then it jumps into the AWS fast lane.
Amazon reports that results can be 50 - 400% faster than using just the public Internet. The larger the file, the faster the connection and the longer the distance, the greater the benefit. Want to see a simple example based on your location? Check out how much faster transfers are to Virginia (our storage location) from your location.
Here are my results from Berlin to Virginia. My results in this test aren’t particularly dramatic, but I suspect the limiting factor is the speed of my connection and the small size of the test file.
Accelerated transfer from Berlin to Virginia
Support for Very Large File Uploads.
With higher resolution cameras and 4K workflows becoming more common, files seem to be growing by the day.
Our old uploader capped files at 5GB. Our regular plans now support files up to 10 GB. This can be increased to as much as 5 TB on our enterprise plans. You’ll obviously need a very fast connection to upload a multi-terabyte file, but if you’re lucky enough to have that, we’re not going to stand in your way.
Simple and Inexpensive Accelerated Uploads.
We want to keep things as simple as possible for our customers. A great advantage of this approach to acceleration versus using something like Aspera or Signiant is that customers don’t need to install any client software or extensions to initiate accelerated uploads. Accelerated uploads work in every browser that we support.
A further benefit of this approach is that it’s designed to easily pass through firewalls. We’re sending everything as regular https traffic rather than using different protocols like UDP which may require special firewall rule configuration in a corporate environment. Screenlight is drop dead easy to use and doesn’t require any special IT involvement.
Bottom Line Benefits.
With transfer acceleration you’ll see upload speeds comparable to Aspera, Signiant and File Catalyst without signing a contract or paying a premium price per GB uploaded.
You’ll also see significantly faster uploads than you do with Box, Dropbox, Hightail, Vimeo and other services that aren’t optimized for quick turn-around media projects.
If you have ideas on other ways to improve Screenlight, we’d love to hear them.
