dailymotion puts Big Data at the heart of their vision

Mar 23, 2015
Vote on Hacker News

When people think of dailymotion, many things spring to mind. Online of videos of course. Or, even, the on-going buzz around their future ownership, including the recent news of a new Hong Kong-based investor who’s participation is expected to help them accelerate their growth in Asia. However, what truly defines dailymotion’s present and future is, rather, Big Data.

Delivering 3 billion videos to users each month coming from individuals and up to 50k content partners (including the likes of Warner Music, the NBA, and Deutsche Welle), means that they have a massive amount of data to manage and mine.  This offers both a huge challenge and opportunity for the #2 video site. They even announced a deal last year with Scality (another sponsor of our upcoming Paris Startup Job Fair) to complement their existing data storage solution.

Big Data = audacious vision and super skilled teams

Any Big Data startups or companies know well the opportunities that Big Data offer. However, the big challenge lies in how to interpret and act on all this data. Figuring out how to do this is what drives and defines the focus of the dailymotion team. Of course, doing this well requires a deeply skilled team that also work near seamlessly together.

The tech team described the challenge to us, underscoring that:

The Big Data challenges at dailymotion are constantly changing. This is where you ask yourself the question ‘Yeah, that works for 500 MB of data; what about 500 TB?’.

The scale and variety of data to be absorbed is overwhelming and we are constantly seeking high performing and highly scalable solutions; using bleeding edge technology, to capture and process up to 15 billion events per day. And capturing the data is just the beginning. How do you leverage the data? How do you join and synchronise the data sources? How do you design for real-time usage? And how can the data be presented in usable way for everyone?

At dailymotion we have are working across teams and skills, from data scientists and developers, to UX architects and business analysts, to design and implement the best ways of capturing and using our data. It’s a crucial part of our business, with a remarkable value, and a technology space that is constantly changing.

Sponsoring the Big Data zone 

We’re excited to have dailymotion joining us again this year at the Paris Startup Job Fair, this time as a sponsor of our Big Data zone. If you’re driven by both the challenges and opportunities Big Data offers, make sure to stop by dailymotion’s stand to check-out the opportunities they have to offer.