Frameserving from Premiere Pro CC 2019 to FFmpeg

A while ago while working on batches of video edits, I came to the realisation that frameserving is simply the best, most flexible way to encode in some cases. Time marches on, and so did my software - eventually I came to a new machine, new Premiere Pro and - disaster - no apparent support for frameserving. Just when I needed it...

Cleverer people than me have solved the CC2019 problem - for those of you editing in Premiere it's once again fairly easy to frameserve encode. However, it did take a bit of sleuthing to figure out a few things; this tutorial should help you to avoid the same problems I encountered.

November 2019: Vouk's excellent Voukoder plugin for After Effects and Premiere can now accomplish some of what this article covers, and it has an active developer and user community. Vouk includes the FFmpeg/libav filters to enable things like bwdif deinterlacing. There are still some bugs but it's worth a test - it should integrate nicely into an AME or batch workflow. More complicated workflows may still benefit from frameserving, so it's still worthwhile to do.

July 2020: wangqr, the developer of the dfscPremiere.prm patch, informed me by email that he's not maintaining it any more as it's been merged into the main Debugmode Frameserver repository. Download v3.0 for use with CC2019 and above. Thank you for your hard work, Q!

You will need:

1. Premiere Pro CC 2019
2. Debugmode Frameserver v3 for 64-bit Premiere and Vegas
   (v2.14 for 32-bit apps required the dfscPremiere.prm patch)
3. wangqr's dfscPremiere.prm patch (no longer needed)
4. A recent 32-bit FFmpeg static build
5. A recent 32-bit AVISynth install
6. A few minutes of your time

Grab a recent 32-bit static build of FFmpeg and drop it into an easily-typable folder. 64-bit versions of FFmpeg unfortunately don't have the necessary code in them to properly interpret the frameserved AVI files.

Install Debugmode Frameserver v3 without installing any plugin (not necessary unless you also use other editors). 

NB: The older v2.14 build also needed wangqr's dfscPremiere.prm patch (which is no longer being maintained, but I've mirrored it here) which you would drop in to C:\Program Files\Adobe\Adobe Premiere Pro CC 2019\Plug-Ins\Common\ . Version 3 of Debugmode Frameserver has this patch merged already.

July 2020: One caveat is that I haven't actually tried yet with v3 of Debugmode frameserver. It may be that you can now use 64-bit FFmpeg as the plugin is now 64-bit native. 32-bit should still work fine.

---

Let's get rendering!

Open Premiere, open a sequence, and hit Ctrl-M to export. Choose the Debugmode Frameserver render option, specify your Output Name for the 'signpost' AVI as something short and memorable (I use fs-output.avi but you could use frameserve.avi, output.avi, woofyfloof.avi -- whatever).

Most Premiere export settings are greyed out, but check that you're exporting audio at the correct sample rate, and that you're exporting both audio and video. Hit Export...

Time to frameserve

In the Debugmode Frameserver dialog which opens, select Video Output and YUY2 colour space (unless you're doing extensive edits or CG rendering, in which case you may wish to choose RGB24/32) - then hit Next. Most videos will be processed internally by Premiere in YUY2, so going to RGB will require internal YUY2->RGB->YUY2 transforms for generally no benefit.

If you feel it may benefit your encode to export as RGB, do some comparative encoding. Arguably, if you're exporting very high resolution or CG content, you're better off encoding with a 4:4:4 colour space instead of the more common 4:2:0; make sure you set up your encoder or FFmpeg appropriately later on.

You should not need to write audio as PCM samples in the signpost AVI, you'll have to wait for the audio to render out and is generally unnecessary.

The Debugmode Frameserver applet will save out the 'signpost' .avi and you can use that to pass to an encoder. I use FFmpeg. However! Attempting to pass the .avi directly to FFmpeg will almost certainly result in an error. This is an example FFmpeg output using a fairly standard FFmpeg recipe:

C:\av\ffmpeg-20190430-19af948-win32-static\bin>ffmpeg -i "D:\projectfiles\fs-output.avi" -vf "bwdif" -c:v libx264 -preset slow -crf 20 -maxrate 3000k -bufsize 3000k -profile:v high -pix_fmt yuv420p -c:a aac -b:a 320k "D:\projectfiles\test.mp4"

 ffmpeg version N-93719-g19af948e53 Copyright (c) 2000-2019 the FFmpeg developers
   built with gcc 8.3.1 (GCC) 20190414
   configuration: --enable-gpl --enable-version3 --enable-sdl2 --enable-fontconfig --enable-gnutls --enable-iconv --enable-libass --enable-libdav1d --enable-libbluray --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libtheora --enable-libtwolame --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libzimg --enable-lzma --enable-zlib --enable-gmp --enable-libvidstab --enable-libvorbis --enable-libvo-amrwbenc --enable-libmysofa --enable-libspeex --enable-libxvid --enable-libaom --enable-libmfx --enable-amf --enable-ffnvcodec --enable-cuvid --enable-d3d11va --enable-nvenc --enable-nvdec --enable-dxva2 --enable-avisynth --enable-libopenmpt
   libavutil      56. 26.100 / 56. 26.100
   libavcodec     58. 52.100 / 58. 52.100
   libavformat    58. 27.103 / 58. 27.103
   libavdevice    58.  7.100 / 58.  7.100
   libavfilter     7. 50.100 /  7. 50.100
   libswscale      5.  4.100 /  5.  4.100
   libswresample   3.  4.100 /  3.  4.100
   libpostproc    55.  4.100 / 55.  4.100
 [avi @ 00ae3ec0] non-interleaved AVI
 [avi @ 00ae3ec0] Could not find codec parameters for stream 0 (Video: none (DFSC / 0x43534644), none, 1920x1080): unknown codec
 Consider increasing the value for the 'analyzeduration' and 'probesize' options
 [avi @ 00ae3ec0] Could not find codec parameters for stream 1 (Audio: none ([172][223][0][0] / 0xDFAC), 48000 Hz, 2 channels, 6 kb/s): unknown codec
 Consider increasing the value for the 'analyzeduration' and 'probesize' options
 Guessed Channel Layout for Input Stream #0.1 : stereo
 Input #0, avi, from 'D:\projectfiles\fs-output.avi':
   Duration: 00:14:53.80, start: 0.000000, bitrate: 32 kb/s
     Stream #0:0: Video: none (DFSC / 0x43534644), none, 1920x1080, 25 fps, 25 tbr, 25 tbn, 25 tbc
     Stream #0:1: Audio: none ([172][223][0][0] / 0xDFAC), 48000 Hz, stereo, 6 kb/s
 Stream mapping:
   Stream #0:0 -> #0:0 (? (?) -> h264 (libx264))
   Stream #0:1 -> #0:1 (? (?) -> aac (native))
 Decoder (codec none) not found for input stream #0:0

This is due to FFmpeg not properly understanding the DFSC FOURCC in the AVI; you need to use AVISynth to interpret the AVI for FFmpeg. 

---

The AVS workaround for FFmpeg signpost AVIs

In Notepad, save the following as fs-output.avs, altering the path as appropriate:

AviSource("D:\projectfiles\fs-output.avi")

From then on, after starting the Premiere export up and preparing your frameserver, invoke FFmpeg with the path to the .avs file. The .avs must have the correct path to the signpost AVI in it, hence why I recommend something short, as you'll need to re-enter it in Premiere's export dialog for each sequence. FFmpeg will then begin encoding:

Guessed Channel Layout for Input Stream #0.1 : stereo
 Input #0, avisynth, from 'D:\projectfiles\fs-output.avs':
   Duration: 00:13:41.48, start: 0.000000, bitrate: N/A
     Stream #0:0: Video: rawvideo (YUY2 / 0x32595559), yuyv422, 1920x1080, 25 fps, 25 tbr, 25 tbn, 25 tbc
     Stream #0:1: Audio: pcm_s16le, 48000 Hz, stereo, s16, 1536 kb/s
 Stream mapping:
   Stream #0:0 -> #0:0 (rawvideo (native) -> h264 (libx264))
   Stream #0:1 -> #0:1 (pcm_s16le (native) -> aac (native))

If you wish, you can apply further transformations to the incoming streams and resultant encoded file directly within FFmpeg. In the following recipe, I'm doing the following:

ffmpeg -i "D:\projectfiles\fs-output.avs" -vf "bwdif,scale=960:-1:flags=spline,unsharp=3:3:0.5:3:3:0.0" -c:v libx264 -tune film -crf 18 -preset slow -maxrate 3M -bufsize 1.5M -profile:v high -pix_fmt yuv420p -c:a aac -b:a 320k "D:\projectfiles\output-960x540p50-crf18-3mbit.mp4"

FFmpeg input processing first...

-vf "bwdif,scale=960:-1:flags=spline,unsharp=3:3:0.5:3:3:0.0"

There's a few parts to this -vf filtergraph argument...

bwdif: deinterlace using bwdif algorithm
My footage is 1080i25 (interlaced 25fps 1920x1080) ProRes. I'm not deinterlacing inside Premiere - its algorithms are awful and inflexible, as they are in After Effects. If you have to work with mixed interlaced and progressive, seriously consider transcoding your interlaced footage to 50p progressive using ProRes - something you can also do with FFmpeg. This will also avoid you having to 'interpret' footage inside Premiere, where it'll invariably do a subpar job. A shame for such a premier NLE. 

I use FFmpeg's great deinterlacing filters. Blend deinterlacing is for schmucks; I use FFMpeg's 'bwdif' algorithm, aka Bob Weaver Deinterlacing Filter. bwdif employs algorithms from the Weston 3 Fields Deinterlacer method, using methods described by BBC R&D's Martin Weston and expertly authored by BBC R&D's Jim Easterbrook. (And if anyone knows how to process video signals, it's BBC R&D.)

w3fdif has the advantage of being motion adaptive (as is YADIF) but has generally superior behaviour regarding interpolation, motion and edge detection. This means things like laterally scrolling elements render much better (e.g. news tickers, small text or scoreboards on football programmes). w3fdif also preserves fluidity of motion by doubling the frame rate; from 25i footage you get 50p output with no 'oscillating' horizontal edges.

scale=960:-1:flags=spline: scale the output, with additional arguments

You can instruct the FFmpeg's scale command many ways - with percentages, mathematically calculated values based on the source (or one relative to another) or imply a value proportional to the original. There's lots of detail to the scaler. You should read all the sets of scaler documentation and look at the various examples before diving in.

Here I'm telling the scaler to downscale the output to 960 px width, calculate the proportional height to match the aspect ratio of the source (denoted by -1) and use the higher quality 'spline' algorithm. The Lanczos method is the only other one I'd consider for either downscaling or upscaling, the default bicubic method is poor quality by comparison for no gain in speed. 

NB: If you're deinterlacing - or reinterlacing (?!) - footage, your order of processing is crucial. ALWAYS deinterlace before doing anything else. Otherwise, you end up scaling the interlaced source, then forcing the deinterlacing algorithm to try and deal with image lines which are no longer 1 line high. Awful things happen as a result.
ALWAYS DEINTERLACE FIRST.

unsharp=3:3:0.5:3:3:0.0: apply an unsharp mask to the scaled image

FFmpeg's unsharp filter is similarly powerful, and can be used to selectively sharpen the chroma or luma in either width, height or both. The syntax is explained in the documentation, and there's various good examples on the web.

My settings define a gentle sharpening of luma and chroma in the horizontal and vertical, but only gently apply it (0.5) to the luma portion of the input signal.

... Then specify output parameters

These are plentiful, and you should consult the FFmpeg manual for a full explanation (https://ffmpeg.org/ffmpeg.html#Main-options). The H.264 options also merit a read, https://trac.ffmpeg.org/wiki/Encode/H.264.

 -c:v libx264 -tune film -crf 18 -preset slow -maxrate 3M -bufsize 1.5M -profile:v high -pix_fmt yuv420p -c:a aac -b:a 320k "D:\projectfiles\output-960x540p50-crf18-3mbit.mp4" 

• -c:v libx264 - shorthand for -codec:video libx264
• -tune film - use x264's preset 'film' tune which lowers deblocking
• -crf 18 - apply a Constant Rate Factor of 18 to the encode
• -preset slow - use the x264 "slow" preset, for a slightly more efficient encode at the cost of slightly longer encoding time 
• -maxrate 3M - define a 'ceiling' of 3 Mbit/sec for the bit rate
• -bufsize 1.5M - more applicable to streaming, but helps constrain maximum bit rates more tightly. Setting a lower bufsize makes FFmpeg calculate the current bit rate more frequently; in this case, every 0.5 seconds. Most of the time, bufsize is set to 1x or 2x maxrate.
• -pix_fmt yuv420p - defines the output to be YUV colour space with 4:2:0 chroma subscampling and planar colour alignment. It's compatible with just about everything.
  • For the highest possible quality you might use --output-csp i444 instead; this still uses YUV but preserves full 4:4:4 chroma information, important for very high quality encodes or video game captures with minimal colour banding.
• -c:a aac 320k - shorthand for -codec:audio aac at a bit rate of 320 kbps.
• Finally, the full path to the output file, enclosed in "inverted commas" to escape any spaces in the path, which are normally treated as delimiters (without folders, C:\folder name\file name.avi would make FFmpeg break at C:\folder and not understand the subsequent parts of the path). FFmpeg is clever enough to understand what container you want to mux your encode as (MP4, AVI, MOV, etc - just specify it, FFmpeg does it).

---

It's quite a lot to process, but it's worth it.

The results I get from frameserving to FFmpeg far outstrip what I could get from exporting directly in Premiere. I trust FFmpeg to deliver great encodes, and it's got a remarkably flexible set of filtergraphs for managing, processing, transcoding or dubbing multiple audio and video tracks. The -c:v and -c:a parameters in particular are deceptively diminuitive but are remarkably powerful. Read the FFmpeg documentation and be amazed.

But for now, enjoy the wonderful world of flexible encoding direct from Premiere! Debugmode Frameserver will probably work well with whatever you're using to edit, provided it's not bleeding-edge new. Some other sophisticated editors like VirtualDub even allow usage of external encoders directly from within the application, using parameters to pass filenames, video settings and so on. I highly recommend VirtualDub if what you mostly need to do is batch transcodes or transforms on many clips, I've processed thousands of files that way. FFmpeg on the commandline is also immensely powerful, and it's great to be able to use it with Premiere again.

If you're interested in the power of FFmpeg, and handle professional video files, check out my other article about manipulating ProRes with FFmpeg.