ffmpeg 🌐 is a swiss army knife for everything audio/video. It can do practically every task under the sun, and in fact powers most major dedicated “video players” (VLC, MPC-HC, built-in players in Chrome and Firefox…)2
If you’re on Windows, it’s technically possible to install
ffmpeg and use it directly 3, but since the windows Command Prompt sucks ass comfort-wise and scripting-wise, it’s recommended to just install Ubuntu as part of the Windows Subsystem for Linux 🌐, and then
apt-get install ffmpeg.
If you’re on Linux, you already know what to do 😎
(All commands are expected to be ran in
bash or a similar Linux shell.)
ffmpeg is pretty clever, it can correctly guess the codecs and reasonable default settings by the file extension, so all of the following will work as expected (and retain metadata4!):
ffmpeg -i video.avi video.mp4 ffmpeg -i video.mp4 video_sound_only.wav ffmpeg -i video_sound_only.wav video_sound_only.mp3 ffmpeg -i song.flac song.mp3
“Reasonable” might not be what you want though, especially in the case of mp3, where the default bitrate is V4 (!), i.e. 140-185 kbps.
If you want, for example, V, use the
-q:a5 option, like so:
ffmpeg -i song.flac -q:a 0 song.mp3
More info at: https://trac.ffmpeg.org/wiki/Encode/MP3 🌐
Since container/format ≠ codec, you might want to select the codec manually.
While it can reasonably assumed that
avi is a bit more complex. You can list all the supported codecs with
ffmpeg -codecs6, but since there’s several hundreds, you better have an idea of what you want to do in the first place.
For example, if you want an
.avi with xvid codec, you just do:
ffmpeg -i original.mp4 -c:v libxvid output.avi
General codec options
This StackOverflow post explains everything: https://stackoverflow.com/a/20587693/3833159 🌐
video // image files // frames
images -> video
1. creating a list of images
ffmpeg needs a list of images in a text file in a specific format 🌐 in order to convert them to a video. There’s a couple ways to do this:
ls *.jpg | xargs -I xyz echo "file 'xyz'" > list.txt
for f in *.jpg; do echo "file '$f'" >> list.txt; done
It’s up to preference, all end up with a list of all JPGs in current directory, in
2. list to video
ffmpeg -f concat -r 30 -i list.txt out.mp4
-f concat tells
ffmpeg to handle
list.txt as a list.
-r 30 specifies resulting FPS (30 FPS)
out.mp4 is output file - autodetected as h264-encoded. (
out.gif, etc. also work - refer to ffmpeg manual)
video -> images
ffmpeg -i FILE image%05d.png
FILE is the video file, and
image%05d.png is the format string for image filenames; this will create
image00123.png, etc. (
%05d means pad with
%010d for padding with
ffmpeg can also smoothly handle streams, so basic stream capture is pretty trivial, provided you grabbed the playlist/HLS url from somewhere7:
ffmpeg -i "https://example.com/playlist.m3u8" my_stream.mp4
Taking a screenshot of a stream
-vframes 1 is the option that tells
ffmpeg to just capture one (i.e. the first) frame of the video - in the case of streams, this means the latest one anyway.
ffmpeg -i "https://example.com/playlist.m3u8" -vframes 1 capture.jpg
dropped frame re-interpolation
ffmpeg also has a rich set of filters 🌐, two of which are of interest for us now:
- mpdecimate 🌐 - Drop frames that do not differ greatly from the previous frame in order to reduce frame rate.
- minterpolate 🌐 - Convert the video to specified frame rate using motion interpolation.
The idea is that
mpdecimate drops all near-duplicate frames, and
minterpolate re-calculates them using non-duplicate frames that were left.
mpdecimate’s defaults are pretty okay, but the result may not look too good if the frame drops are frequent and long. I’ve had pretty good results using its
max parameter which limits the amount of frames dropped in a single stretch of video, e.g.
-vf mpdecimate=max=15 which drops at most 15 frames (i.e. half a second assuming 30 FPS), meaning interpolation won’t happen everywhere and the video will remain faithfully choppy.
minterpolate, on the other hand, defaults to semi-smart motion compensated interpolation, and that might just be what you want, but it generally gives pretty funky results. Fortunately, it also has a “blend” mode, which just averages the start and end frames and crossfades them, which gives much more agreeable outputs for simple frame drop situations. It is also generally much faster, I was getting near or above real-time speeds using “blend”, whereas motion compensation dropped the processing speed to 0.01x.
TL;DR: Full command(s) including the filter pipeline:
# Fill out all frame drop gaps ffmpeg -i choppy_video.mp4 -vf mpdecimate,minterpolate=mi_mode=blend smoother_video.mp4 # Fill out all frame drop gaps no longer than 10 frames: ffmpeg -i choppy_video.mp4 -vf mpdecimate=max=10,minterpolate=mi_mode=blend smoother_video.mp4 # Motion interpolate the gaps and replicate a bad ketamine trip ffmpeg -i choppy_video.mp4 -vf mpdecimate,minterpolate smoother_video.mp4
h264 also has “profiles”, basically sets of features 🌐 - and it turns out this can make the difference between a file working and not working on some crappy embedded media players, like TVs or pico projectors.
-profile:voption limits the output to a specific H.264 profile. Some devices (mostly very old or obsolete) only support the more limited Constrained Baseline or Main profiles. You can set these profiles with
ffmpeg -i original.mp4 -profile:v baseline output.mp4
And apparently, some players are also sensitive to the pixel format8, i.e. can’t handle anything else than YUV w/ 4:2:0 chroma subsampling, to fix this use the
-pix_fmt option as follows:
ffmpeg -i original.mp4 -pix_fmt yuv420p output.mp4 # or, with the profile settings... ffmpeg -i original.mp4 -profile:v baseline -pix_fmt yuv420p output.mp4
No silver bullet, you’ll just have to try different things for different devices. A database of crappy players and appropriate
ffmpeg settings would be great.
like ID3 tags and their FLAC, OGG, WAV, etc. equivalents ↩
read as -quality:audio ↩
and respectively, formats with
like https://addons.mozilla.org/en-US/firefox/addon/hls-stream-detector/ 🌐; or just press F12, check the Network tab, and look carefully ↩