Only once for the containing canvas. A direct access to the canvas pixel data and webGL offscreen rendering is all you need. With a few hacks, we're already 90% there. I expect by next year to see an explosion of this. There already are projects in Github that translated SDL and OpenGL to build web widgets.