Don’t let Large Files Slow You Down
White Paper for LucidLink by Jason Bloomberg, Managing Partner, Intellyx
Content creators face numerous challenges, from the inexorable advancement of technology to the post-pandemic reality of work-from-anywhere, hybrid work.
Technology is supposed to enable creative work, but it too often impedes it. Large files, in particular, can slow down collaboration while increasing costs, especially when creative teams are geographically distributed.
To address the challenges of working remotely with large files, LucidLink has produced an innovative approach to collaborating in the cloud without the laborious task of copying files to the local computer.
Content Creation in the Future of Work
Little did we know when the digital revolution upturned the world of creative content over thirty years ago that the computer-driven transformation of creative work would continue for decades with no end in sight.
Nowadays, systems are blisteringly fast. The emergence of cloud technology has made it possible for teams to collaborate at a distance and leverage the scalability of cloud storage. Creative applications from Adobe, Avid, and Autodesk turn imagination into reality with the mere click of a mouse.
The role of the content creator has also evolved. Present-day creatives challenge brands by making consumers more critical of content quality, appeal, and diversity of representation.
Today’s creative content is remarkably diverse. It cuts across many industries and departments, including media and entertainment (M&E) endeavors like film, television, streaming content, social media, multiplayer gaming, live sports, and event productions.
Creative Work beyond Media and Entertainment
Beyond M&E, creative pursuits are alive and well within marketing departments across multiple industries ranging from advertising to retail to architecture and engineering, as well as the full spectrum of fine and performing arts.
Independent creators, influencers, and small business owners are using creative content to expand their audience reach and promote their ideas and products.
The payment company Stripe has its finger on the growth of creative content as one of the most popular monetization platforms for new and upcoming creators. According to Stripe, the number of content creators is growing at an annual rate of 48% during 2020 – 2021, with no signs of slowing down.
Regardless of the industry or type of project, modern content production enjoys a combination of new technologies and the opportunities they present. From augmented and virtual reality to artificial intelligence to the still-maturing metaverse, creative work has never been as technology-empowered as it is today.
The Human Experience of the Work-from-Home Era
The dramatic changes impacting creatives and their work go beyond the relentless advance of technology.
Covid shifted where creative efforts took place as offices shut down, driving people to work from home. Everyone had to change their lifestyles. Creatives had to adapt to this work-from-home era, leaving the office behind and their computers home and easily connect to home networks.
The physical distance of remote work allows employees to cultivate space for creative time during their day by putting ‘do not disturb’ on their calendars and switching off notifications. Working remotely also eliminates the need for round trip commuting time going to an office.
Collaboration, however, suffered during the pandemic. According to research by Lucidspark, 75% of remote employees who expressed concerns about working from home ranked team collaboration as the aspect of their work that suffered the most.
The Post-Pandemic Reality
Now that we’re largely past the pandemic, most creatives are not content to go back to the way things were before. Work cannot be stuck in the past – today, the focus is on the future of work and accommodating a work-from-anywhere lifestyle.
This future includes hybrid working conditions, mixing in-office time with work from home as well as other locations of choice, from coffee shops to local shared offices. Within this hybrid scenario, creatives don’t want to shortchange collaboration opportunities. If anything, they want to improve collaboration as well as geographic flexibility.
The move to post-pandemic hybrid work, in turn, takes place within the context of improved work-life balance. Hybrid is not just about geographic independence; it’s about meeting creatives where they are, having working hours and the balance of personal time in mind.
This new balance requires more than flexible schedules. It also requires improved productivity. Creatives want to accomplish more in less time, not by feeling stretched to the point of burnout, but by leveraging better tools and technologies to streamline the busywork and other tedious time sinks.
The future of work, therefore, is more about the human experience than it is about technology – but make no mistake, we must get the technology right to make the vision of hybrid work a reality.
Why Creative Productivity Continues to Suffer in the Future of Work
Creatives are spontaneous, collaborative, inventive, and imaginative. It shouldn’t matter how distributed a team is or what technology they use. Creative work proceeds, nevertheless.
While distributed teams that leverage global talent are now the norm, in some industries, they have been the norm for years. For example, the film production industry has been particularly resilient by leveraging digital technology in new ways to solve the issues of remote production.
For example, studios are using remote production technology to rethink the notion of a studio as more of a mindset than a physical location.
Video creation leverages smartphones and other common technologies to film at actors’ homes and other unconventional locations. The shift to streaming video has also reworked the technology requirements and expectations for theatrical distribution.
The Challenge of Virtual Private Network Bottlenecks
Today, distributed work has become the norm for modern creative teams, who are now running into bottlenecks as they become hybrid workforces.
One of the primary bottlenecks is the central role of on-premises technology. ‘On-premises’ refers to technology within corporate offices, either on desks or in server rooms within the building, as opposed to somewhere in the cloud.
Virtual private networks (VPNs) connected remote workers to on-premises technology before the pandemic. At the onset of the pandemic, however, scalability, performance, and security challenges limited the practical use of VPNs to access corporate resources.
VPNs’ bandwidth constraints prevented everyone from connecting at once. The distance between the user and the server impacted VPN speed. Security also became an issue as VPNs provide network access to every entity – making it easier for malicious users to access and exploit network resources and applications.
In addition, VPNs were simply too expensive to support more than occasional remote connections and can be difficult and frustrating to use, especially for long periods of time.
Was Migrating to the Cloud the Answer?
Shifting on-premises applications and creative resources to the cloud was the obvious answer.
Creative organizations’ move to the cloud had been taking place for several years, but the pandemic suddenly added new urgency to the process, as a remote workforce could no longer take advantage of on-premises technology as they had before.
Remote work, previously ranking around 70 out of 100 in terms of priorities, suddenly became one of the top priorities for organizations.
Soon, creative files, including large ones, moved to the cloud as well. Cloud file sync and share services like Box and Dropbox helped to fill some of the need for both storage and limited collaboration for files of all sizes.
Such services, however, are better suited to smaller files than the giant ones that creatives often work with and fail to deliver on the seamless collaboration that creatives require.
These services work by synchronizing files on the cloud and on the local system. As file sizes and other data sets grow, these synchronization-based approaches don’t scale well, limiting their efficacy for production use.
Other integration-based technologies can pull data from storage services as needed, but these approaches also have severe limitations.
The lack of control over how and where such tools store data is also a significant challenge. They offer a ‘one size fits all’ storage back-end that doesn’t fit many organizations’ requirements.
Synchronization takes time with large files, slowing down collaboration. Worst of all, such services are quite expensive when many large files are involved. As a result, productivity suffers, as does the budget.
Perhaps the biggest challenge with file sync and share solutions, however, is the time it takes to access large files. Creatives want to work with files instantly regardless of their size, rather than spend time downloading them.
Getting a Handle on the Modern Content Creation Problem
In the future of work, creative content producers require several core technologies to get their jobs done efficiently. They need superior collaborative applications, fast connectivity to the internet, the ability to work with and collaborate on large files – and of course, everything they do must be secure.
Today, these capabilities rely upon the cloud. The cloud alone, however, doesn’t address all of creatives’ technical requirements.
For many creatives, the challenges of working with large files introduce a bottleneck that interferes with their desired pace and increases frustration among the team. Cloud-based storage with its file sync and share capabilities is part of the answer but doesn’t go far enough.
In many cases, the problem is data gravity: moving data from one place to another is slow, onerous, and expensive – and the larger the files, the worse the problem.
Popular file sync and share solutions, cloud connectors, and cloud gateways help, but synchronization requires multiple copies of each file that can lead to version conflicts and slowdowns.
How LucidLink Solves the Problem
The solution to this problem is for creatives to be able to work both collaboratively and locally on large files without having to move them out of the cloud. This seeming contradiction may seem paradoxical, but cloud storage solution vendor LucidLink has figured out how to have the best of both these worlds.
LucidLink Filespaces is an advanced, general-purpose distributed file system that the company architected for the cloud. Filespaces can use any Amazon S3 or Microsoft Azure Blob Storage compatible object storage as a back-end repository, giving customers a choice of where they store large files.
LucidLink’s secret sauce: instead of moving entire files between the cloud to the local system the way file sync and share solutions do, the platform streams small blocks on-demand as the application needs them, managing corresponding metadata updates in real-time separately from the file contents in each block.
Streaming these blocks in parallel allows LucidLink to download only the portions of a file the application requires. When creatives modify the files, they only need to upload those blocks that they changed, rather than the entire file.
LucidLink compresses and encrypts these blocks at the client, improving the efficiency and performance of Internet connectivity and cloud storage while ensuring the security of client data.
Once LucidLink has the file metadata on the local machine, it uses a sophisticated ‘prefetching’ technique to download only those bits of the large files that the creative in front of the computer will require to do their work.
LucidLink then uses a similar technique to write changes back to the files in the cloud that avoids losing work even if the network goes down. And of course, LucidLink fully secures all data and metadata as they move up and down from the cloud.
From the creatives’ perspective, they are working locally with the file that works with whatever applications they choose to run – and furthermore, they can collaborate with other team members working on the same files, wherever in the world they happen to be located.
Behind the scenes, LucidLink leverages encryption to keep data secure, compression to further reduce the size of files and other information moving between local systems and the cloud, and garbage collection – a technical term for freeing up memory and local storage when the user doesn’t need them anymore.
Meanwhile, the files live on in the cloud or in on-premises object storage, wherever the organization wants them to. Decisions about where to store files are now independent of decisions about where to access and use them, supporting the full vision of hybrid work.
The Intellyx Take
The computer revolution hit the creative world over three decades ago, as computers quickly replaced paper-based processes and tools for creative work.
Nevertheless, content creation is not now, nor has it ever been about technology. Technology serves as tools in the creative’s hands, just as paintbrushes and T-squares did back in the day.
Creatives don’t care how the technology works – only that it does work. They don’t care about files or applications or metadata or the cloud. They care about the work and the value it provides to their clients and their audience.
The demand for creative content, however, continues to increase at a rapid pace, raising the bar on the volume and velocity of data as well as the quality of execution – all within constrained budgets and timelines.
Given these constraints, improving collaboration to increase productivity without adversely impacting creativity is an absolute necessity.
The only way M&E companies – or any other content-producing company – can keep up with the market demand for creative content is by adapting to the future of work and having the right tools to support it.
Copyright © Intellyx LLC. LucidLink is an Intellyx customer, and Adobe and Microsoft are former Intellyx customers. None of the other organizations mentioned in this article is an Intellyx customer. Intellyx retains final editorial control of this article. AI was not used in the production of this article. Image credit: LucidLink.
