How to Solve Video Color Inconsistencies Across Browsers

A technique utilizing the canvas element to play videos in their intended color space.

💡 Our Bright Idea

While creating the new website for Swivel, we decided early on that we wanted to tell the story of what Swivel is and how it helps remote teams be awesome through the medium of videos. And that these videos would be embedded right into the page itself using native HTML 5 video elements, they would autoplay (no controls shown), and they would appear seamlessly integrated into the rest of the page design. Primarily that the background color of each video would perfectly match the background color of its containing element on the page.

Early page design concept created in Sketch.

⚔️ The Great Mint War of 2018

For the most part, this worked out okay in practice, with the bulk of our video content being juxtaposed against the main dark-gray background of the site. However, there was one place where this design goal quickly broke down into the hot mess now known as the Great Mint War of 2018.

The utter tragedy of mismatched colors!

When we placed our main header animation video into its minty green sea of tranquility, it revolted violently in Chrome and Safari, yet remained quite at peace in Firefox. We were understandably perplexed.

🎨 Web Browser Color Profiles

After some initial digging around in the caverns of the Interwebs, we found some information leading us to believe that perhaps it was the color profile that was used to export the video from Adobe Premiere or Adobe After Effects that was the culprit. But then why was Firefox okay? After some more digging, it seemed that some browsers utilize their own color profiles and forcibly thrust them upon whatever MP4 file which dares wander into their dominion.

🗜 Video File Downsizing

Not being well-versed in the realm of video codecs, color profiles, and the like, we thought that perhaps we could simply remove whatever wayward data was causing such bad behavior by simply downsizing the video file — this, of course, was a good idea to do anyway as we quickly discovered that each video file shaved off a good 80–90% of its file size! The service that we ended up using, and which I recommend, is called VideoSmaller.

Downsize your video files!

With our now 195 KB, 20-second video in hand (Yes! 195 KB video. Did I mention how awesome VideoSmaller is?), we tried our header video setup once more. But, alas, the colors still did not match in Chrome or Safari (yet they remained ever-correct in Firefox).

💡 A Newer Brighter Idea 💡

Completely bewildered and frustrated, we regrouped and came up with a new plan. If the colors are just going to be off in those browsers, we could at least set the background color of the parent element to the same background color from the video… right? I mean, how hard could it be?

Immediately, a couple of us had the same bright idea: let’s just grab a frame from the video, paint it to a canvas element, then grab the pixel data from a spot we know will have the background color, then apply that color as the background color of the containing element. Easy peasy!

To test this idea out, I set up a CodePen with the expected background color and the video embedded in the page. Then I searched around for some quick ideas on how to get images from the video painted to a canvas element. After a few minutes of sleuthing and coding, I had the beginnings of a solution. Except, I couldn’t get the pixel color information to show up in my console log.

🛑 Another Set of Hurdles 🚫

I asked around on the team to see if anybody could help me solve the mystery of why I wasn’t able to get my seemingly simple experiment to yield any fruit. Turns out there was a error being logged to the real browser console (not the simple CodePen implementation I was staring so longingly at):

Unable to get image data from canvas because the canvas has been tainted by cross-origin data.

Gah-what!? A tainted canvas preventing me from using getImageData? Another round of Google-sleuthing clued me into what the problem was, and why it was, but it didn’t really help me figure out how to solve the problem. I had no idea how to properly ask for the video source from the CodePen side, nor did I find any simple-to-understand means of adding the necessary CORS header on the Amazon S3 or CloudFront side.

Something I did notice, however, is that when we painted a frame of the video to the canvas element, the colors were suddenly and inexplicably correct. I had a new hunch that even if we figured out the CORS issue and were able to successfully grab a pixel’s color data, we’d still get the wrong (correct) color.

🌈 The “Aha!” Moment, and a Scheme 🤔

We did, however, focus on this quite possibly wrong solution for longer than we may care to admit. Until at last we had a breakthrough, “Aha!” moment:

When we paint the video frames to the canvas, the colors match … in every browser!

“Could we just re-paint the entire video to the canvas element?” I pondered. Yes, yes we can! It didn’t take me very long to expand the simple proof-of-concept in CodePen to repaint every frame of the video to the canvas element in near-real-time. And just like that, we had video colors perfectly matching background colors in every browser! 🎉

Yay! The colors are back!

📈 But What About Performance?

There is definitely a hit to the CPU while it constantly paints frames of video into a canvas element. So much so, in fact, that we’re already looking into other alternatives (my alternative of choice would be to export a few simple PNG images and animate the whole thing using pure CSS).

Currently we are using a setTimeout of about 30ms between calls to drawFrame (roughly 33.333 FPS). We tried using requestAnimationFrame, thinking that it would potentially be smoother and kinder to the CPU than constant timeouts, but it actually ended up being a lot choppier and didn’t seem to alleviate the CPU enough to continue using it.

We also tried using the same technique on all of the videos on our page, since it really does fix the color inconsistency issue, but that proved to be way too heavy-handed. The colors were fixed (which made the pixel-focused designer in me so happy 😃) but the cost of our relatively simple page on the CPU was unacceptable (and made the user-focused designer in me so sad 😢).

💭 Your Thoughts

So, what do you think of our solution? Have you struggled with color inconsistencies across browsers? What solutions have you come up with in the past to deal with problems like this?

---

About

Swivel is a digital product aimed at fostering incredible remote team culture, collaboration, and inclusivity by means of a shared audio/visual virtual office. Swivel, Inc. is the newly-formed company behind Swivel the product and an off-shoot of the K3 software development consulting firm.

🧔🏻 Dallas does UX and UI design things, some full-stack development, a wee bit of video scripting, acting, & editing, a smidgen of customer service, and a slice of marketing. He lives in College Place, WA, where he and his wife, 👩🏻 Nicole, also run an escape room business called 🗝 16 Keys. They also enjoy taking walks with their dog, 🐶 Ruby. Dallas is very keen to make your acquaintance and promises to be less and less conversationally awkward every time he talks with you. 😄👋🏼

...
Originally posted on
Medium
.