Temporal Coherence for Facial Video Synthesis

Treating temporal coherence as a first-class property in image-to-video facial-expression synthesis.

In my doctoral research, I found that individually plausible frames could still produce videos with flicker or unnatural motion. I therefore treated temporal coherence as a separate property rather than assuming that frame-level quality would produce a coherent sequence.

  1. Source face and reference expression or video
  2. Preprocessing and facial geometry
  3. Generative model
  4. Temporal and geometric constraints
  5. Synthetic video and video-oriented evaluation

Problem

A frame-wise model can optimize each image without representing the transition between consecutive frames. Small discontinuities can then accumulate as visible flicker or unnatural facial motion.

Why it matters

Temporal behavior is part of the validity of a generated video. An evaluation based only on still frames can miss the instability that viewers perceive when the frames are shown as a sequence.

Approach

I combined previous-frame conditioning with explicit consistency objectives. Pixel-transition consistency targets abrupt appearance changes, while facial-landmark consistency provides a geometric signal tied to facial structure. The model therefore learns from relationships across frames as well as from individual images.

Evaluation and evidence

I used video-oriented evaluation and reproducible experiments to examine temporal behavior. The publication contains the reported measurements and experimental images; this summary does not add results that are not available in the repository.

Evaluation focus: plausible frame → incoherent video → temporal failure analysis → explicit temporal and geometric constraints → video-oriented evaluation.

Technical implementation

Python and TensorFlow support the generative-adversarial training pipeline, facial-landmark processing, video generation, and controlled experiments. Docker makes the research environment easier to reproduce and inspect.

What I learned

A property that exists across outputs must be represented in the method and measured at the sequence level. Frame-level results alone do not establish temporal coherence.

Code repository IEEE Access article