This ain’t no Rotoscope

As usual it started with a tweet:

The referenced tweet looked interesting:

I’ve tried Rotoscoping before, but this looked good. I hied over to MacOS/OSX – Paint of Persia community – itch.io, read about it. paid $5 and set things up.

At that point I realise that the app helps you to manually trace frames, seemed a wee bit time consuming. I though I’d leave it for another day. I did start musing on doing something similar, I was thinging FFmpeg & ImageMagick.

Google takes me to Fred’s ImageMagick Scripts: CARTOON.

so the plan is:

  1. export a movie to a series of images.
  2. cartoonise those image
  3. create a move from the cartoon images

Bounus points for getting the sound track back in.

I already have ffmpeg & imagemagick installed on my mac. These are commandline tools.

I downloaded the cartoon script

First find a video:

YouTube downloader tool – Fastesttube!

Rename the downloaded file shower.mp4

Open the terminal and cd into the folder that has the cartoon script and the movie in it.

First extract lots of still images (make a folder shower first mkdir shower):

ffmpeg -i shower.mp4 -r 6 shower/out_%04d.jpg

this takes the input file (-i) at 6 frames per second and create jpg files in the shower folder with the file name out_0001.jpg ,out_0002.jpg ect

Given the movie is 3 minutes 22 seconds long I end up with 1212 images in the folder. I delete the last few manually to give me 1023 images.

I now need to loop through all of those images and create a cartoon version.

the use of cartoon is basically:

./cartoon face.jpg temp.jpg

There are some paramaters you can use but I stuck to the default.

This would take the image on the left and create the one on the right:

face.out

So I move into the shower folder cd shower

mkdir out;for i in *.jpg; do ../cartoon $i out/$i.jpg;done

this script:
makes a nother folder called out

for i in *.jpg for each file with a .jpg extension in the current folder do does:

../cartoon $i out/$i.jpg runs the cartoon script from the folder above (../), and saves the output file in the out folder with the same name.

When I kicked this off I quickly realised it would take a while to r=un through 1212 images, so went to bed.

The next morning I have 1023 cartoons (I don’t imagine that it took all night).
out_0672

Time to stich these together, it took me a few goes to get the paramaters right. (google helps).

ffmpeg -f image2 -framerate 6 -i out_%04d.jpg -c:v libx264 -pix_fmt yuv420p out.mp4

  1. -f chooses the format image2 which makes an image sequence.
  2. -framerate 6 to get the smae length of movie as I started with
  3. -i infiles confusinging names out_0001.jpg ect the %04d means look for 4 figured numbers starting with 0001
  4. -c codec
  5. -pix_fmt is the pixel format, I dn’t know much about these but the script failed until I added it in

It took me a few shots until I got this right.

I then moved the out.mp4 video to the same folder as the original and

ffmpeg -i shower.mp4 -acodec copy -vn shower-audio.mp4

To extract the audio from the original file.

then:

ffmpeg -i out.mp4 -i shower-audio.mp4 -vcodec copy -acodec copy -shortest final.mp4

To add the audio from the original to the out video to give me a final one.

the -shortest parameter gets rid of the audio at the end to make up for the frames I removed.

Bingo:

At this point I remembered iMovie has a comic filter…

This post is mostly as an aid to my occasional dip into the world of commandline video editing. Posting helps me remember. It also plays a wee bit fast and loose with copyright.

The featured image is a gif giffed from a few of the stills.

ps this is quite a disturbing clip, I didn’t really watch it till I finished, could have picked a nice one!

Update, Ron commented:

What would it look like if you’d run the result videos through the script one more time. Keeping the same number of jpegs but making the lines more stand out and the fills less so you’d have black outlines and white?

That didn’t make much difference os I did some tests with the parameters for cartoon, and Ron went with increasing the brightness: ../cartoon -b 300

This gave me this:

13 thoughts on “This ain’t no Rotoscope”

  1. Awesome! Outstanding!

    What would it look like if you’d run the result videos through the script one more time. Keeping the same number of jpegs but making the lines more stand out and the fills less so you’d have black outlines and white. ?

    Reply

  3. I like the last result. Some parts are really powerful with her face, great acting still visible, in black outlines and lots of whites.
    The music to it is of course epic.

    Reply

      1. I think very much could be removed because the music makes it instantly recognizable.
        Thought frames would have to be repaired manually probably because the black outlines aren’t smooth running lines anymore but too much dots and stripes.
        The shower is remarkable beautiful, just so dots and stripes and with the sounds to it, it’s just okay that way.

        Long time ago they made the first Lord of the Ring animation movie by painting over frames.
        Drawing on frames has also been done in the earliest times of movie making.

        Reply

  4. I’m running into a permissions issue with the cartoon script

    $ sudo ./cartoon face.jpg temp.jpg
    sudo: ./cartoon: command not found

    Seems like I have read and write access to the file, so I am wondering if there is an issue with ImageMagick, or something else I am just missing.

    Reply

        1. I am close, another snag I hit was when I was trying to rip audio from gimmethebat.mp4 to gimmethebat-audio.mp4 I am getting this error: Output file #0 does not contain any stream

          I am using this command in the cartoon directory:
          ffmpeg -i gimmethebat.mp4 -acodec copy -vn gimmethebat-audio.mp4

          Anything glaring?

          Reply
  5. Pingback: Gimme the Bat | bavatuesdays
  6. Pingback: Danny’s Wheels | bavatuesdays
  7. Pingback: Commandline gifs: Cartoon The Nitty Gritty – 106 drop in

Leave a Reply