Mimic Lip Sync

I was fortunate enough to get my hands on Mimic 3 Pro, stand-alone lip sync software before it (apparently) went extinct. Whomever owns the rights to it now, I hope they make it available for purchase once again. Something this cool should never be lost to the annals of time!

That said, I bought Mimic Pro for Carrara in the same order when I bought Carrara Pro itself, along with aniMate 2 for Daz Studio (the paid version goes beyond the capabilities of the Free version that ships with Daz Studio).

I've never tried the Lite or the Live versions of Mimic, but both Mimic 3 Pro stand-alone and Mimic Pro for Carrara are very similar in how they work, so I'm sure that those versions would be a valuable asset to anyone seeking to lip sync their 3D figures.

The versions of Mimic that are currently available to my knowledge are:

Mimic Pro for Carrara

If you're planning on rendering in Carrara, I think that this version might just be your best bet. It really depends upon how you like to work, and how well you want to customize your lip sync to perfection.

The thing is, Mimic Pro for Carrara uses Carrara's NLA Clip technology to activate the phonemes, which are the shapes the mouth makes for any particular sound or visual word.

Mimic 3 Pro's phoneme blocks in the timeline look just like NLA clips. So it makes sense.

But Carrara's NLA Clips are much more powerful!

Mimic Pro stand-alone blocks have a single final shape in the middle, and the block allows that shape from off to on to off again. It's powerful. We can add multiple morphs to artistically shape the mouth for any sound in the viseme library.

Carrara clips can be animated. What did he just say? Yes! We can have the shapes change in time using Carrara's NLA Clips, because we have complete control over what they do to our character.

Mimic 3 Pro Stand-Alone

This software came before the Carrara plugin. It was around long before Daz 3D had anything to do with Carrara.

Mimic is an incredibly powerful, lightweight and elegant tool for creating lip sync animations for 3D figures. Figures that work in Poser, that is - because the file it exports is a sophisticated animated PZ2 (Poser software "Pose" file) of an expressive facial animation with the sound embedded into it.

Yep. Load this onto a figure in Poser or Daz Studio and the sound is attached!

Daz 3D has been creating the DMC files (more on that in a bit) for their figures since Mimic began, which was... what... the very beginning of all of this? I don't really know. When did Mimic One come out?

These DMC files are supplied by Daz 3D freely with the figure they're made for. Genesis 8 even has one, as do their dogs, cats, dragons, gorilla, frog... they help us let anyone talk! I love Daz 3D!!!

DMC files work for all versions of Mimic, so we'll get to that in its own heading next.

Mimic 3 Pro has its own elegant (if not dated) interface that can be resized and organized however you like, and it remembers how we build it - which is really refreshing for such a legacy app!

Within it we load the head of a popular figure. This figure must be a CR2 (Poser Character) file. We also load an audio file of either WAV (Win) or AIFF (MacOS) and either a text file or we can type the text into the interface. I just paste it in from my script.

If we happen to have a reference video, we can load that in as well. I've never done this, but it's clever that they thought of adding that in!

We can also ask Mimic to auto-add expressions and gestures - who could say No to that?!!!

Click Analyze and off we go into the magical interface of Mimic 3 Pro!

Within the interface, we can play back the automated calculations and see the results on the character head. But that's just the beginning - unless you like it, in which case just go File > Export PZ2, pick a folder and go!

But we can do so much more.

First of all, we can bring our CR2 in here without any audio or text or video and create our own DMC file. Just pick the phoneme (sound expression) and double-click to edit it. In this window we can open other windows for picking which morphs belong to this sound, and at which setting. This is how I designed my interface - placing my windows how I like to edit the "look" of my phonemes.

But that aside, we can do the same thing to the result of what Mimic just calculated for us. We can add morphs, switch them or even use the result of a different phoneme to replace the one that was chose by the DMC file. Let's make this a bit less confusing.

In the middle we see the head of the figure. Right-click in the view port and drag to rotate the head to how you want to see it.

Along the bottom is the timeline with a bunch of blocks for each individual waveform that the software detected, which then uses the text to help determine the final phoneme to apply. So these blocks are the automatically calculated phonemes - even one called Silence, which it places between sounds to turn them off. We can also create what "Silence" looks like.

If we select a block and right-click it, we have several options, like replace with the selected Phoneme. If we have a phoneme selected in the list on the upper left of the screen, we can replace the block selected on the timeline with that, using this command. "Edit Strength" is a popular one with me. I like to 'relax' the shapes that the mouth makes to help make the speech look more natural.

It can be quite fun to take either route: Create a few custom DMC files specific to your particular character and how you want them to emote - making a different DMC depending upon mood or other scenario, etc., or just using the base DMC file and tweaking the phonemes on a per session basis.

Once we have our figure open in Mimic, we can reload a new audio and text combination by re-entering the Session Manager: File > Session Manager. Just keep in mind that the timeline will retain the length of the previous setup unless the new one is longer. So either edit the timeline scrubber or just have a PZ2 that's longer than it needs to be. Your choice.

DMC Files

DMC files are presets of what the generic phonemes should look like according to the original artist release - Daz 3D. They're quite good, but can easily be tweaked or replaced entirely with what the user intends.

For example, the phoneme "S" might be as simple as the "S" dial from the character's viseme morph collection turned up. But for Rosie, I often replace this with a stronger version of the "IY" phoneme with a touch of lower lip drop on the right side, and maybe even a little mouth corner stretch. It really depends on what she's talking about. So in that regard, I usually prefer to simply tweak the phonemes from the default Genesis base DMC on a per session basis.

In contrast, there are also times when I know that she's going to be in a certain mood or situation, and I want her mouth shapes to reflect this, so I'll go in first and create a custom DMC and save it with a meaningful name so I can reuse it in other sessions.

DMC files are small so we can feel free to just create as many as we like. And being new to all of this, I'm having a lot of fun trying as many styles and technique that pop into my mind as I go. I often let the audio recording help direct me toward which path I want to take.

Daz Studio 32 Bit

If you don't have Mimic and can't get it but want to have automated lip sync, your free Daz Studio is also available in 32 bit form.

Due to licensing issues, Daz 3D is not (yet?) able to include the lip sync function in their 64 bit Studio, so if you haven't installed it yet, go to your Daz Install Manager (DIM) and set the download filters to include 32 bit software and download/install the 32 bit version of Daz Studio's current version.

I don't think you'll need to also install a bunch of other bells and whistles for Daz Studio, but I'll leave that to your own discretion/personal tastes/needs, etc.,

The (minor) downside for me in using this is that Octane is not available in 32 bit form, so my Octane tab disappears from my interface when I launch 32 bit Studio. I just need to open and dock it again - like I said: minor.

Likewise, 64 bit Studio doesn't have Lip Sync, so you'll have to open the Lip Sync pane every time you go back to 32 bit from 64.

All of that aside, the Lip Sync function in DS32 is adequate. The way I was imagining Mimic Lite above working... that's what Lip Sync does. You add your character, apply audio and text, analyze and get a result. No extra bells and whistles like the powerful Mimic Pro stand alone or Carrara versions, but it works. I actually got some really decent tests done for Genesis 8 using this utility - and it's absolutely Free!

So while Mimic Pro for Carrara is ultimately the Most Powerful choice for making your lip sync results look as real as your imagination can take it, working with NLA Clips does take some practice, and there's a lack of real-time play back feedback.

Mimic Pro Stand-Alone may be a one-shot for each phoneme, to be perfectly honest, that's often enough to take on any lip sync job and MSA (Mimic Stand Alone) is ultimately Easy to use while being mind-bogglingly Powerful! I said earlier that the lack of real-time feedback doesn't really bother me in Carrara - because it really doesn't. But having it in MSA really makes the experience a lot more enjoyable for me, and I also really enjoy that it's a separate, lightweight app that can run while I'm working on the scene in the main app. Just pop over to Mimic Pro to run a new animated and enhanced PZ2, refresh the library, load it, render it!

Mindsong and I are going to be delving much deeper into all of this - where we'll begin to unlock the magic of animated NLA Clips using Mimic Pro for Carrara. That's going to be a fun trip! It already has been!!!

We'll be recounting our endeavors with more pages branching from here - stay tuned!