Converting video and software vs. hardware encoding

I have a tv tuner and some DVR recordings I want to bring on a flight with me. When I used Handbrake to convert into 720p h.264, it results in ~1GB per hour of TV, and h.265 ends up at around 600MB. Cool!

But that’s all software encoding, which with my CPU takes about 40 minutes per hour of recording to encode.

I tried NVIDIA hardware encoding with my RTX 2060 and it went 5-10x faster, but the resulting h.265 file was over twice as big for the same quality.

What’s up with this? Anyone have any tips on how to improve the file size for hardware encoding while using NVIDIA hardware encode?

I’m looking forward to being educated here, but my understanding has been that the GPU encoder is totally different software than the CPU encoder, little in common other than that they both make 264/265. The CPU frames are more efficient because the CPU can do the general video analysis better, while the GPU runs a simpler encoding strategy, faster. So you would just need to fiddle with the encoding settings until you find something that gives you the output you want at an acceptable speed. It’s an art…

I’m an old hand with this. What is your CPU? 2-pass, 1-pass turbo with fidgeting with CBR instead of VBR yield best tradeoff for space/quality/transcoding time. Getting better than real-time (less than 2h for a 2h video) is always going to be a challenge. The NVENC hardware transcoding is for faster speed of encoding at cost of both quality and file size.

I have a Core i5-9600K. I’ll probably just stick with the default settings then for CPU encode. Had no idea GPU was so much worse.

Solid CPU. Do 2-pass encoding for smaller file size.

Handbrake loves CPU cores and I believe it scales up to 12? So an upgrade would do wonders for you.

Your 9000 series CPU is more than modern enough. Anything below Skylake I’d consider gimped nowadays. If you want to improve quality/reduce file sizes you will have to experiment a few times with your personal threshold/tolerance for bitrate/encoding time. I go from 1080/4K to 720p and HEVC/H265 with 2-pass and various.

I keep it 768kbps-1MBps bitrate overall with the audio being stereo AAC_LE 48 KHz.

Under 300MB for 45min of 720p video at 25fps. Just set it up overnight.

Okay, Handbrake just added AV1 support and I’m playing around with it. Holy shit, this codec is ridiculous.

Episode of an HBO show at Super High Quality h264, HEVC, and AV1 presets.

h264 5.68 gigabytes
HEVC 2.6 gigabytes
AV1 1.53 gigabytes

That’s insane. AV1 is 26% the size of h264, and 58% of HEVC.

The downside is AV1 it is an absolute bear to transcode. I’ve got a 12-core 5900X and I’m seeing a transcode speed of about 11 fps. Which means it takes a bit more than 2 seconds to transcode a second of the episode. It took two hours to transcode 51 minutes.

In comparison, I can get 30-35 transcoding to HEVC.

I did try the using the GPU *6800XT) to encode to AV1, and it was a lot faster, but you lose efficiency. The file size ended up being 5.81 gigabytes, which is even more than h264.

CPU transcoding remains the most efficient if you’re looking for quality and file size.

All useful info, thanks. What about the load to decode it? I suppose nothing can decode it with HW assist yet?

Difficult for me to tell because my laptop and desktop both have hardware AV1 decode.

Intel added it with 11th gen Core, AMD added it with RDNA2.