In an ideal world, where bandwidth wasn’t a factor, content of all sizes would play in its intended form. Until then, David Meyer looks at how to achieve the perfect picture.

In a world of rainbows and unicorns and unlimited bandwidth, ultra-high video resolutions and high frame rates could co-exist. They will someday (rainbows and unicorns notwithstanding). But in the meantime, bandwidth is limited and that means some compromise — higher resolution with lower frame rate, or vice versa. For example, 8K/30 or 4K/120, both of which occupy the same nominal bandwidth. But which actually looks better? I’ll cut to the chase and say that my pick, depending on the content type, is high frame rate. Let me explain.

But first, why does it matter? Well, resolution is typically associated with detail, assuming the more, the better. This is the promise of 8K, but when it first entered our consciousness a few years ago 8K piqued my inner— or not so inner— geek. As an SMPTE member and CEDIA standards contributor I turned to the science of visual acuity and wondered whether we’d even be able to see the difference unless sitting two metres (or less) away from a 75” (or bigger) display.

As I write this, the joint CEDIA/CTA R10 Residential Systems Committee of industry volunteers has launched a new review of the CEB23 recommended practice for video system design. This will inevitably spark a debate about relative viewing distances, among other things. The last 2017 review of that document already references 8K and allows for viewer preferences with screen size, but in practice it commonly results in closer seating distances (wider angle of view) than SMPTE and THX recommendations. Importantly, sitting too close can lead to viewer fatigue. There’s a number of reasons for this, one of which is a challenge I’ve identified: that there’s more to resolution than just the number of pixels.

That’s where frame rate comes into the picture (excusing the pun), for which there are several considerations. They’re not in any particular order as they’re not mutually exclusive, but let’s start with motion blur.

Consideration #1: Motion Blur

A single frame is otherwise known as a photo. A still image photographer will use a very slow shutter speed as an artistic choice to make a waterfall look silky, or a sports photographer will use very fast shutter speeds to freeze the action in exquisite detail. But video (and film) is a sequence of frames played in quick succession to give the illusion of motion. Motion blur is an essential feature of this to ensure that one frame rolls into the next and the resulting movement looks smooth. Conversely, a sequence with at least moderate movement or panning and without any motion blur would look jittery and unnatural. The lower the frame rate, the higher the relative motion blur needs to be, which in turn reduces detail. After all, blur is blur, regardless of the resolution.

Now consider sitting close enough to a very large display to capitalise on visual acuity and actually be able to discern the extra detail that 8K affords. Have you ever noticed that demos tend to be of tranquil scenes with modest movement, slow flyovers and the like? They can be quite immersive. But dial up the movement and its requisite motion blur, and that’s when being too close can be even more fatiguing. Sitting further back is generally more comfortable, but then much of the value proposition of 8K is lost, at least when compared to 4K for normal home viewing. Different story for other applications like interactive signage and the like, but I won’t get into that here.

Consideration #2: Aliasing

Another, less impactful consideration is that of aliasing. This is the term to describe anomalies in video that aren’t supposed to be there. A leading example is the “wagon wheel” effect, wherein a spoked wheel looks like it’s rotating very slowly backwards instead of the reality of its forward motion. This is caused by a conflict of the spoke positions, the shutter speed, and frame rate, typically at time of capture.

Back in 2008 when 4K (later to be known as UHD) was a new thing, BBC R&D in the UK identified that increasing the resolution could exacerbate aliasing, but that also increasing the frame rate could help to mitigate this. So, they went about conducting some tests to determine at what speed aliasing and motion blur would effectively disappear, arriving at a whopping 300 fps. Try jamming that down an HDMI cable! However, recognising that this was impractical, they suggested 150 fps as a reasonable compromise, or 120 fps as the maximum rate per the still-current ITU and CTA standards.

Consideration #3: Temporal Resolution

As we’ve established, the resolution of a single frame doesn’t really paint the whole picture (there’s that pun again). Also considering frame rate introduces the crucial element of time. Therefore, it is not so much the spatial (width x height) resolution, but the temporal (time-based) resolution— pixels per second— that most influences how we perceive motion video.

Whenever you see a product other than a display being promoted as 8K capable, it can generally be assumed as an indicator of its HDMI bandwidth capacity rather than specific to 8K applications or content availability. That is, it indicates HDMI 2.1 compatibility with up to 48 Gbps data rate support. This can be used for many formats other than 8K, such as 4K with high frame rate (HFR) or variable refresh rates — we’ll come back to the latter shortly.

For example, let’s compare 8K/24 with 4K/120:

• 8K boasts a 7680 x 4320 spatial resolution, totalling just shy of 33.2 million pixels. But at a mere 24 fps, its temporal resolution runs at 796 Mpixels/second.
• 4K/120 features the well-established 3840 x 2160 spatial resolution, or 8.3 million pixels.
  But at the very high frame rate of 120 fps, the temporal resolution is 995 Mpixels/second. Not only is this 20% higher than 8K/24, but it will also have 1/5^th the amount of motion blur in each frame.

In summary, 4K HFR could well be perceived as being more detailed, especially with action scenes. In due course we’ll be able to have the best of both worlds with 8K and high frame rate, but that means tapping into the very top end of HDMI 2.1. We’ll get there in due course, as we eventually did with HDMI 2.0.

Consideration #4: Gaming

I’ve written about gaming previously in this magazine, and while not a gamer myself, I’m constantly in awe as I explore its technical attributes and capabilities. I maintain it’s the most demanding consumer video source there is today.

Gaming is all about speed. Lots of action and phenomenal processing capacity. Like with sports, 60 fps is regarded as the minimum to afford enough detail to properly see what’s going on, but higher again is better, preferably at least 120 fps on a supporting TV. The ultimate is what is known as Variable Refresh Rate (VRR) in the HDMI 2.1 specification, being comparable technology to VESA’s Adaptive-Sync. This can match the speed of frame output from a graphics processing unit (GPU) right through to the display, even if the speed varies from one frame to the next (as it does, depending on scene complexity). It’s the ultimate but needs LOTS of bandwidth.

With the new generation gaming consoles there’s some buzz around 8K gaming, but consistent with the considerations listed above, going to such high resolution would have to be at the expense of the frame rate. And for gaming, frame rate wins absolutely hands down. No contest. Again, until bandwidth is truly no longer an issue (think DisplayPort 2.0 and 80 Gbps, or whatever comes next).

Consideration #5: The “Cinema Look”

All this is for naught when you factor one last, very important thing: artistic/creative intent. In the leadup to the CEB23 review launch I was talking to John Bishop of Mavericks Architectural Cinema in Maine, USA, about frame rates. He said that he simply far prefers 24 fps as the classic “cinema look”, citing that higher rates can result in the cardboard cut-out crispness that’s often associated with TV video. Technically, if the SMPTE rule-of-thumb of double frame rate is used, the cinema look can be maintained with 48 fps and 1/96^th shutter speed, albeit with more detail. But that’s a choice for the cinematographer in balancing technical with art. Our job is to reproduce that on the screen at home.

In conclusion, the bandwidth-limited world in which we live means some compromise is needed. It always has. Bandwidth limitations were the main reason we started with interlaced video; it’s why HDMI 1.3 was limited to 1080p, why HDMI 1.4 was limited to 4K/30, and why HDMI 2.0 was limited to 4K/60. But fortunately, it’s a stretchy band that keeps getting bigger, and as technology evolves we grow into the bandwidth and the capabilities it affords. Some day, 8K/120 may well become normal just as 4K/60 has today. In the meantime, while high resolution is great it’s not always the most important thing.

Lastly, participation in the R10 effort to review the video system design recommended practice, is available and free for CEDIA or CTA members. Contact your association for details.