When I download streams from Twitch, I remux them into MP4 because MPEG-TS doesn't support -movflags faststart
, is not supported by HTML5 players and can't be uploaded directly to YouTube. Sometimes this process changes framerate of the video stream. Incorrect FPS somehow corrupts the video during processing on YouTube, which leads to horrible audio desynchronization on any resolutions below 1080p60.
My working theory is that Twitch doesn’t fill the voids in the stream when frames are lost. This is not critical for MPEG-TS and FLV containers, however, the MP4 container tries to compute the global average FPS. YouTube somehow relies on the global FPS during video processing, which leads to a decrease in the duration of the video stream, and causes its desynchronization with audio.
Good video (60 fps):
# ffmpeg -i Streams/good.mp4
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'Streams/good.mp4':
Duration: 06:04:58.97, start: 0.000000, bitrate: 7672 kb/s
Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, bt709), 1920x1080 [SAR 1:1 DAR 16:9], 7495 kb/s, 60 fps, 60 tbr, 90k tbn, 120 tbc (default)
Bad video (59.99 fps):
# ffmpeg -i Streams/bad.mp4
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'Streams/bad.mp4':
Duration: 05:58:20.47, start: 0.000000, bitrate: 7677 kb/s
Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, bt709), 1920x1080 [SAR 1:1 DAR 16:9], 7499 kb/s, 59.99 fps, 60 tbr, 90k tbn, 120 tbc (default)
function ffprobe_entry { # filename, stream, entry
ffprobe \
-v error \
-select_streams "$2" \
-show_entries "$3" \
-of default=noprint_wrappers=1:nokey=1 \
"$1" | tail -n1
}
function is_correct_framerate { # filename
avg_fps=$(ffprobe_entry "$1" v:0 "stream=avg_frame_rate")
r_fps=$(ffprobe_entry "$1" v:0 "stream=r_frame_rate")
# Allow no more than 0.05 seconds of desynchronization per hour
[ "$(echo "60 * ($r_fps - $avg_fps) < 0.05" | bc -l)" -eq 1 ]
}
is_correct_framerate Streams/good.mp4 && echo YES || echo NO
is_correct_framerate Streams/bad.mp4 && echo YES || echo NO
Output:
YES
NO
The easiest and the fastest way is to remux the video into FLV. But first it needs to be remuxed back into MPEG-TS, otherwise ffmpeg will carry incorrect framerate into the new container.
ffmpeg -v error -i INPUT.mp4 -c copy -f mpegts - | \
ffmpeg -y -hide_banner -i - -c copy -bsf:a aac_adtstoasc OUTPUT.flv
The resulting OUTPUT.flv should have the same framerate as the original source (60 FPS in this example).
ffmpeg -i bad.flv
Input #0, flv, from 'bad.flv':
Duration: 05:58:20.48, start: 0.010000, bitrate: 7675 kb/s
Stream #0:0: Video: h264 (High), yuv420p(tv, bt709, progressive), 1920x1080 [SAR 1:1 DAR 16:9], 60 fps, 60 tbr, 1k tbn, 120 tbc
That means this video can now be uploaded to YouTube.
Feel free to leave comments with your solutions. I still can't figure out how to remux MPEG-TS into MP4 without altering the framerate. I have tried MP4Box, mkvtoolnix, FFmpeg 2.8-4.2, all without success. Full re-encoding is not an option for me because I don't want to stress my hardware this much.