Featured image of post Accelerate Video Editing and Transcoding for FFmpeg Using GPU
Software

Accelerate Video Editing and Transcoding for FFmpeg Using GPU

Using FFmpeg for video editing and transcoding, and how to use GPU acceleration

Motivation

FFmpeg is a very powerful audio and video processing tool that can be used for editing, transcoding, merging, splitting, extracting video and audio, etc. FFmpeg is a free and open-source software, initially initiated by French programmer Fabrice Bellard and now maintained by Michael Niedermayer. Many players, video editing software, transcoding software on the market, such as Blender, Kodi, Plex, Shotcut, VLC media player, YouTube, etc., are based on FFmpeg. Of course, there are also many software that use FFmpeg’s code but do not comply with FFmpeg’s open-source agreement, and are nailed to the “FFmpeg Pillory”.

FFmpeg is a command-line tool that is a bit complicated to use, but very powerful. I have only used a few simple functions, such as video clipping, merging, adjusting resolution, transcoding, video acceleration, slow motion, audio and video separation. But I have always used CPU for processing, now I have a graphics card on my computer, I want to use GPU for acceleration. It did take me a little effort, so here is a summary.

Prerequisites

  • NVIDIA graphics card on the computer
  • Linux system

Compile and Install FFmpeg from Source Code

I use Ubuntu 22.04 system, and my FFmpeg was installed through apt before, but this version does not support GPU acceleration. So I need to compile and install from source code. If you have installed FFmpeg through apt before, you need to uninstall it first:

```bash
sudo apt-get remove ffmpeg
```

The compilation and installation process I referred to the official documentation of NVIDIA. Unfortunately, this document seems a bit outdated. If you compile and install directly according to the steps in the document, you will encounter some problems. Here are the steps I successfully compiled and installed.

Install Dependencies

There are three main dependencies:

  1. NVIDIA graphics card driver. Please refer to my previous article Install NVIDIA Graphics Card Driver on Ubuntu 22.04.

  2. NVIDIA encoding interface library. Use the following command to compile and install from source code.

    1
2

    git clone  https://github.com/FFmpeg/nv-codec-headers.git
cd nv-codec-headers && sudo make install && cd –

  3. FFmpeg dependency library. Use the following command to install.

    1

    sudo apt-get install build-essential yasm cmake libtool libc6 libc6-dev unzip wget libnuma1 libnuma-devsudo apt-get install build-essential yasm cmake libtool libc6 libc6-dev unzip wget libnuma1 libnuma-dev

Compile and Install

  1. Download FFmpeg source code.

    1

    git clone https://git.ffmpeg.org/ffmpeg.git

  2. Configure compilation parameters.

    1

    ./configure --enable-nonfree --enable-cuda-nvcc --enable-libnpp --extra-cflags=-I/usr/local/cuda/include --extra-ldflags=-L/usr/local/cuda/lib64 --disable-static --enable-shared --disable-x86asm

    • /usr/local/cuda here is the installation path of NVIDIA graphics card driver. If your installation path is different, please modify it.

    • --enable-nonfree is to support non-free encoders.

    • --enable-cuda-nvcc is to support CUDA acceleration.

    • --enable-libnpp is to support NPP acceleration. NPP is the abbreviation of NVIDIA Performance Primitives, a high-performance image and signal processing function library provided by NVIDIA, which is not supported by FFmpeg by default.

    • --disable-x86asm is to avoid the following error during compilation:

      1

      nasm not found or too old. Please install/update nasm or use --disable-x86asm for a build without hand-optimized assembly.

    If the above command is executed without any problems, you can continue to compile.

  3. Compile.

    1

    make -j8

    -j8 here means using 8 threads for parallel compilation, you can adjust according to the number of CPU cores.

  4. Install.

    1

    sudo make install

    The above command will install FFmpeg in the /usr/local/bin directory.

Problem Solving

After compiling and installing according to the above steps, when I run ffmpeg on the command line, I encountered the following error:

1

ffmpeg: error while loading shared libraries: libavdevice.so.61: cannot open shared object file: No such file or directory

This is because the FFmpeg library file is not correctly linked, and manual linking is required:

1

sudo ldconfig

If the above command does not solve the problem, it is because FFmpeg installed the linked library files in the /usr/local/lib directory during compilation and installation, while the system’s default linked library path is /usr/lib. In this case, you need to add /usr/local/lib to the linked library path:

1
2

sudo echo "/usr/local/lib" > /etc/ld.so.conf.d/ffmpeg.conf
sudo ldconfig

Basic Usage of FFmpeg

Here are a few commands I have used:

  1. Video clipping.

    1

    ffmpeg -i input.mp4 -ss 00:00:00 -t 00:00:10 -c copy output.mp4
    • -i input.mp4: input file.
    • -ss 00:00:00: start time.
    • -t 00:00:10: duration.
    • -c copy: copy encoding.

  2. Video transcoding.

    For example, convert mkv format to mp4 format.

    1

    ffmpeg -i input.mkv -codec copy output.mp4
    • -codec copy: copy encoding.

  3. Video acceleration or slow motion.

    1

    ffmpeg -i input.mp4 -vf "setpts=0.5*PTS" output.mp4
    • -vf "setpts=0.5*PTS": acceleration factor. Less than 1 means acceleration, greater than 1 means deceleration. Here 0.5 means 2 times acceleration.

  4. Video resolution adjustment.

    1

    ffmpeg -i input.mp4 -vf scale=1920:1080 output.mp4
    • -vf scale=1920:1080: target resolution.

  5. Audio and video separation.

    1
2

    ffmpeg -i input.mp4 -vn -acodec copy output.aac
ffmpeg -i input.mp4 -an -vcodec copy output.mp4
    • -vn: no video.
    • -acodec copy: copy audio encoding.
    • -an: no audio.
    • -vcodec copy: copy video encoding.

  6. Video merging.

    If the resolution, frame rate, encoding, etc. of the videos to be merged are the same, and there are only two videos, you can use the following command:

    1

    ffmpeg -i input1.mp4 -i input2.mp4 -filter_complex "[0:v][0:a][1:v][1:a]concat=n=2:v=1:a=1[outv][outa]" -map "[outv]" -map "[outa]" output.mp4
    • -filter_complex "[0:v][0:a][1:v][1:a]concat=n=2:v=1:a=1[outv][outa]": merge video and audio.

    If there are more videos to be merged, it is recommended to write the video list into a text file, and then use the concat protocol to merge.

    1

    ffmpeg -f concat -safe 0 -i list.txt -c copy output.mp4

    • -f concat: specify the protocol.

    • -safe 0: allow reading any file.

    • -i list.txt: video list file. The content is as follows:

      1
2
3

      file 'input1.mp4'
file 'input2.mp4'
file 'input3.mp4'

    • -c copy: copy encoding.

Using GPU Acceleration

To use GPU acceleration, you need to add --enable-cuda-nvcc and --enable-libnpp parameters when compiling. The above compilation and installation steps have added these two parameters.

To use GPU acceleration, you need to specify the -hwaccel cuda parameter. For example:

1

ffmpeg -y -hwaccel cuda -hwaccel_output_format cuda -i input.mkv -c:v h264_nvenc -ss 0:00:00 -to 0:01:23 output.mp4
  • -hwaccel cuda: specify to use CUDA acceleration.
  • -hwaccel_output_format cuda: specify the output format as CUDA.
  • -c:v h264_nvenc: specify to use NVIDIA’s h264 encoder. Of course, you can also use other encoders, such as hevc_nvenc. If you want to see the supported encoders, you can use the ffmpeg -h encoder=nvenc command.

Of course, the commands introduced above can also use GPU acceleration by adding the -hwaccel cuda parameter.

My CPU is i5-9600K, and the graphics card is NVIDIA’s RTX 4060 Ti. When using only the CPU, processing a video with a resolution of 1920x1080, the CPU usage is about 100%, and the processing speed is about 30 frames per second. After using GPU acceleration, the GPU usage is about 33%, and the processing speed is about 500 frames per second. It can be seen that the processing speed is increased by about 16 times after GPU acceleration.

Clarity Issue

After using GPU acceleration, the video clarity may decrease. This is because when using GPU acceleration, FFmpeg will use the NPP library to process images, and the processing accuracy of the NPP library may not be as good as that of the CPU.

If you process the same video separately with CPU and GPU, and then compare the sizes of the two videos, you will find that the video processed by GPU is much smaller. For example, the size of a video clipped by CPU is 300MB, and the size of the same video clipped by GPU is only 50MB.

If you check the bitrates of the two videos, you will find that the bitrate of the video clipped by GPU is much smaller than that of the source video. This is because when processing with GPU, the video will be compressed, resulting in a decrease in video clarity. If you want to maintain the clarity of the video, you can specify the bitrate.

1

ffmpeg -y -hwaccel cuda -hwaccel_output_format cuda -i input.mkv -c:v h264_nvenc -b:v 20M -ss 0:00:00 -to 0:01:23 output.mp4
  • -b:v 20M: specify the bitrate as 20M. Here 20M means 20Mbps, you can adjust according to your needs.

FFmpeg prioritizes speed rather than clarity when using GPU acceleration. If you want to maintain the clarity of the video, you can also use the -preset slow parameter to specify the encoding speed.

1

ffmpeg -y -hwaccel cuda -hwaccel_output_format cuda -i input.mkv -c:v h264_nvenc -preset slow -ss 0:00:00 -to 0:01:23 output.mp4
  • -preset slow: specify the encoding speed as slow. Here slow means slow, the processing speed is slower, but the clarity is higher.

You can also combine the above two parameters.

1

ffmpeg -y -hwaccel cuda -hwaccel_output_format cuda -i input.mkv -c:v h264_nvenc -b:v 20M -preset slow -ss 0:00:00 -to 0:01:23 output.mp4
comments powered by Disqus