Except that interpolating 24p content to more than double the frame rate is inherently going to look strange, because the algorithm is inventing information that didn’t originally exist. The Tom Cruise tweet is explicitly against this, because, like colorization, it mucks up the film as intended by the creator.
Native high-framerate film is a related, but distinct, discussion.
Wait WTF? Frame rate interpolation should never be enabled for video games. It introduces significant output lag, and seriously… why?
Just reporting what I had noticed when I tried it – you can call off the dogs. :) I never noticed any output lag, but I also probably wasn’t playing anything where it would have been an issue. I thought it did a fine job of faking 60fps for console games that wouldn’t ever hit it natively. I just couldn’t get past the artifacting.
You and I agree on this. I was just saying that I don’t think a lot of people here would have a different opinion about native high-framerate versus interpolated high-framerate if their objection to the latter is “it looks like a soap opera.”
Well, sort of. If the scene is static enough that it conforms to the sampling limit, then all the information necessary to construct the interpolation is actually there. I suspect most “natural” scenes (unanimated, unmodified, and unprocessed) are static enough. I’m also curious how much the compression algorithms affect the de-sampling process. I think the hitches you see in the background during panning (the difference in foreground and background speeds that someone mentioned above) are due to how H.264 does compression, not to the interpolation.
A couple of friends and I have a regular movie night, and apparently watched several movies on their new 4K set that had interpolation turned on without anyone noticing it enough to say anything (a few times I thought I was catching the telltale alarming sinuous–nay, ophidian–motion of actors in a scene, but hey, it’s their TV, so let 'em be).
Then we popped in the 1971 Fiddler on the Roof–the oldest movie we’d seen thus far by a couple-few decades–with all its long panning shots of countryside and sunsets, and the hitching became immediately interminable for me. So I do wonder if the presence of artifacts does in fact screw with the algorithm somehow to make it work even worse than normal (in this case, artifacts more from the film’s age than especially lossy compression).
There are a lot of things that can mess with the algorithms. White backgrounds, for one. My mom had it on while my niece was watching some kid’s cartoon that was basically just characters on a white background. The characters were standing still, but the algorithm was inserting a bunch of frames that made it look like they had little trails coming off them in one direction.
It’s used alot for demanding VR games as current gpus can’t handle some games (usually the poorly optimized ones) at 90fps. I don’t notice any response lag in the VR headset, and I think you would notice it alot there, but artifacts can be an issue.
On PC all of them: rift, vive, mixed reality. The quality of the algorithm varies by headset, I have mixed reality and it works well there. It’s called motion reprojection or asynchronous reprojection or phrases like that, but reading about the tech it sounds identical to frame interpolation in TVs.
OK, I watched the 5 minute 4K Aquaman trailer on my TV with Motionflow (Sony’s interpolation) on and then off. It definitely looks janky with it on, but it’s mostly the effects that suffer. Scenes with dialogue are fine. Fighting scenes that are mostly animation look like plastic figures. With Motionflow off though, the video is most definitely less crisp; the edges are blurrier and there’s flickering in the deep colors. Much as the effects look like effects with MF on, I kind of prefer the sharpness anyway.
I got some questions, because it honestly don’t understand some of this.
First, i had just assumed that modern movies were recorded digitally. Do they actually still use film? That seems odd to me.
Second, I’m not sure why anyone would prefer the motion blur from film, rather than just a higher framerate (although 60 may not be high enough).
Motion blur with 24hz film is necessary, because it lets your brain fill in the gaps and not look super choppy… But in real life you don’t need blur. Your eyes do that, on their own. If you view a suitably high framerate video, your eyes will vote it like the real world. There’s no need for the blur to try and trick your brain.
Now, with video interpolation, i guess that’s doing some extra crap that might screw up other stuff view some post processing effect… But I’m not understanding why folks would prefer a low framerate video with blur, rather than just a high framerate video.
I also may totally be missing lots of stuff about all this.
If the film is produced at a native 60Hz, it looks great shown at 60Hz. I just watched a 4k 60Hz video someone took in Costa Rica and it looked like I could walk into the screen. Gorgeous. The problem is that (almost) all movies are filmed at 24fps. That’s the native format. When displayed on a 30Hz NTSC TV screen, first repeated frames are used to convert 24fps video to 60Hz (every frame is repeated, every 3rd frame is repeated three times), then that stream is downsampled to 30Hz by eliminating every other frame. (It’s actually slightly more complicated than that, but that’s the gist of it.) The result looks like the 24fps film, with motion blur. What interpolation does is make a “guess” at the contents of the in-between frames, rather than using repeated frames, so that you get sharp, smooth motion characteristic of 60Hz video. And I think in general, for natural images, the “guess” works. (It’s not actually a guess; it’s math.) But with animation or effects that were produced for 24fps video, it looks weird.
On that note, I tried the 4k trailer for Avengers:Endgame on my set and it looks great with Motionflow on. I suspect Disney spent the extra bucks to make the effects look good at 60Hz.
I don’t think what VR does is the same as this interpolation. My understanding of what VR does is it takes the current frame and moves it based on the head movement in the last 1.5ms to simulate 90fps. This is because the eyes are almost always focused on the center of the image and therefore you won’t really notice artifacts at the edge of the screen where it doesn’t have data from. So if you are looking at an image and move your head up, the next frame 1.5ms later is the exact same frame you were looking at but moved downward by a little bit to make it seem like it’s still responsive to your head movement.
At a conceptual level, I can only see one difference between them. A TV knows the frame at t-1 and t+1 when presenting the frame at t, and so the smoothing is an interpolation. A VR headset will not know the frame at t+1 when presenting the frame at t, so it takes the position of the headset at t-1 and guesses where you’ll be at t using the gradients/hessian. So the smoothing is an extrapolation.
I don’t believe it is merely taking the frame and physically moving it a nm left/right/up/down, as these days it also works for 6 dimensions of head movement and does cost some gpu time (it’s just much cheaper). I am not sure though, I can’t find a piece on the internet with a detailed technical write up.
Even if they’re using digital cameras (and mostly they do), film cameras capture images at 24fps. It’s been the standard for 35mm film since about 1930.
