Slow HTML Canvas Performance? Understanding Chrome's `willReadFrequently` Attribute

August 2, 2024

At work, I build web-based video annotation tools. We want annotators to quickly draw on high-res video frames to mask important objects.

A while ago, we had a bug report at work that our HTML canvas drawing tools were sometimes laggy. Every developer loves 'sometimes' bugs!

I tried to reproduce the issue, but failed. Even directly on the user's machine, the problem came and went. A while later, after partially refactoring of the rendering code, the lag suddenly appeared consistently. It always started after a few brushstrokes.

It took me ages to track it down to a newly added getImageData() that was now called after each stroke. Once I commented out the getImageData() call, the lag disappeared. I was super confused why reading data would cause drawing to slow down.

The final clue came from a Chrome warning I had seen previously. This warning appeared around the same times as the lag:

Read on to learn how this warning is related to the lag and how you can control this behavior with the willReadFrequently attribute.

TLDR; Chrome sometimes disables GPU acceleration for a canvas element using a heuristic, leading to slower writes and higher CPU usage. You can control this behavior by setting willReadFrequently. More details, plus performance numbers, below.

GPU vs. CPU: Reading and Writing

The browser uses the GPU for rendering whenever possible. The GPU is optimized for parallel pixel processing. Thus, writing a lot of pixel data onto a canvas is much faster when done by the GPU.

When rendering on the GPU, pixel data resides in the GPU memory. The browser gets direct write access to that memory. This avoids the performance cost of transferring data between CPU and GPU. Because of this direct access to the GPU, drawing paths and shapes on a canvas is lightning fast, even for high-resolution images.

But what if we want to read each pixel? For this, we can use getImageData() on the rendering context. To make the resulting ImageData available to our JavaScript code, the browser must copy all pixel data from GPU memory to system RAM.

Reading pixel data from a canvas

This process is "slow" (several ms) and can add up to a performance issue when done frequently enough. That's why the browser sometimes suggests enabling the willReadFrequently attribute.

The willReadFrequently Attribute

The willReadFrequently warning in Chrome

> Link from the warning.

This warning means that Chrome has noticed how often you have read from a canvas. Frequent calls to getImageData(), toDataURL(), or toBlob() will trigger this.

The warning suggests that we can improve canvas read performance by setting willReadFrequently to true. We can set the attribute when creating a 2D canvas context, like so:

const canvas = document.createElement("canvas");
const ctx = canvas.getContext('2d', { willReadFrequently: true });

Tip: Set this attribute on the first call to getContext('2d'). Setting it later won’t have any effect because all following calls to getContext('2d') will always return the same instance.

The spec for willReadFrequently says that the flag tells the browser to prioritize read performance on this canvas. Because reading directly from the CPU's RAM is faster, many browsers will then disable GPU acceleration for the canvas. All rendering will then be done by the CPU.

This is a trade-off between read and write performance. Let's see how this affects real-world performance.

Impact on READ Performance

I conducted a quick tests to observe the real performance difference. I set willReadFrequently to true or false and measured the time taken for a single call to getImageData() on a 4K canvas.

Disclaimer: These are rough measurements with performance.now() to give you an idea of the magnitude. Actual numbers will vary based on hardware, canvas size, and CPU load.

Try it yourself:

Reads < 25ms are rendered in green, while reads > 50ms are rendered in red.

Impact on WRITE Performance

Setting willReadFrequently to true disables GPU acceleration for the canvas, meaning we lose the parallel processing power of the GPU for writes.

This is especially noticeable when using filters on the 2D rendering context. At work, we use SVG filters to achieve various effects. An important filter for bitmasking prevents anti-aliasing. We need this for maximum pixel precision.

Write performance is a bit harder to measure. Drawing calls from JavaScript return immediately, but the actual rendering happens asynchronously. The time it takes for the browser to render the canvas is called 'commit time'. This is the time between the JavaScript call and the actual rendering on the screen.

We can see the commit time in the Chrome DevTools' performance tab. CPU and GPU usage are also visible there.

The following screenshots show the difference in commit time and CPU/GPU usage when setting willReadFrequently to true or false.

The measured JavaScript task is: "Stroke a path onto the canvas during a mouse move event."

A) willReadFrequently: false

Low CPU usage with willReadFrequently === false. Visible GPU usage.

willReadFrequently === false with roughly 0.1ms commit time + GPU utilization.

< 1ms GPU time

Here we see a commit time of 0.1ms, which is negligible. The GPU is used for rendering, as indicated by the GPU usage in the performance tab.

B) willReadFrequently: true

High CPU usage with willReadFrequently === true. (Almost) no GPU usage.

willReadFrequently === true with 47ms commit time.

Now we see a commit time of 47ms. This is the time it takes for the browser to render the canvas after the JavaScript call. Even on a 30fps screen, the 47ms duration causes missed frames and a laggy UI.

Try it yourself:

The impact is most noticeable when trying to quickly draw smooth circles/loops.

The tool detects frames < 16ms (60fps) using Chrome's performance API. Note, the dropped frame counter doesn't track every dropped frame, but it gives you an idea of how many frames are missed due to extensive commit times.

Summary for WRITE Performance

In short, setting willReadFrequently to true improves read performance slightly but significantly increases CPU usage, negatively impacting write performance.

Chrome's willReadFrequently Behavior

We’ve learned that setting willReadFrequently to true disables GPU acceleration, while setting it to false forces Chrome to use the GPU, potentially slowing down read performance.

But what happens if you don’t set willReadFrequently at all?

Drumroll... 🥁

Chrome uses a (poorly documented) heuristic to decide when to disable GPU acceleration!

This heuristic is at least partially based on how often you read from the canvas. If you read "too often," Chrome disables GPU acceleration. According to a response I got from the Chromium team, "too often" is:

Twice!

Wait, what?

Yes, you read that right. Reading from the canvas twice can disable GPU acceleration. This is why the Chrome warning appears after two strokes in my video annotation tool.

Don't trust me. Here's the link to the ticket: https://issues.chromium.org/issues/349853784

Looking at the impact this has for any interactive drawing tool, I found this behavior to be quite surprising. It's especially frustrating because the warning doesn't tell you what the heuristic is. You have to dig into the Chromium source code to find out.

Note: Safari doesn’t have this heuristic, so it’s not an issue there.

Try it yourself:

Set willReadFrequently to undefined and draw a few strokes. Read times should go down and drawing becomes laggier. Press the 'Reset' button to reset the canvas. 'Reset' with mount a new HTML canvas.

Conclusion

Here are the key takeaways:

1. Multiple calls to getImageData(), toDataURL(), or toBlob() can trigger a performance warning in Chrome.
2. Setting willReadFrequently to true disables GPU acceleration for the canvas.
3. This can lead to significantly slower writes, high CPU usage, and laggy interactions.
4. The performance difference shows up as 'commit time' and higher CPU usage in DevTools.
5. The impact is especially noticeable when using filters on the 2D rendering context.
6. While read performance improves slightly, write performance gets significantly worse with willReadFrequently (at least in my tests).

My personal takeaway? Force willReadFrequently to false in your application. This way, you can leverage the GPU for rendering and keep the UI snappy. Reading performance remains acceptable, as tested on various devices.

Feedback?

Have you encountered this issue before? Did I miss something? Let me know on X, LinkedIn, or in the comments below!