ImageSequenceClip repeats frames

[trygvrad]

While using ImageSequenceClip with low framerates, certain frames become repeated because of floating point arithmetic. This should have been fixed in #464 back in 2017 (for some reason I can't link to the issue properly), but I had the same problem in 2.1.2.

Expected Behavior

No frames repeat

Actual Behavior

If using a framerate of 3 fps, the 13th, 16th and 19th frame repeats are repeats of the 12th, 15th and 18th.

Steps and code to Reproduce the Problem

The following code checks if the correct frames are returned

import moviepy.video.io.ImageSequenceClip
import numpy as np

# set of frames that are all different levels of grey
frames = [np.ones((400, 400, 3))*f for f in np.linspace(0, 1, 20)]

for frames_per_second in range(1,31):
    print(f"fps: {frames_per_second}, repeated frames: ", end='')
    movie = moviepy.video.io.ImageSequenceClip.ImageSequenceClip(frames, fps=frames_per_second)
    last_frame = frames[-1]
    for i, frame in enumerate(movie.iter_frames()):
        if np.any(frame == last_frame):
            print(f" {i}", end='')
        last_frame = frame
    print('')

It outputs:

fps: 1, repeated frames: 
fps: 2, repeated frames: 
fps: 3, repeated frames:  13 16 19
fps: 4, repeated frames: 
fps: 5, repeated frames:  12 17
fps: 6, repeated frames: 
fps: 7, repeated frames: 
fps: 8, repeated frames: 
fps: 9, repeated frames:  19
fps: 10, repeated frames: 
fps: 11, repeated frames: 
fps: 12, repeated frames: 
fps: 13, repeated frames: 
fps: 14, repeated frames: 
fps: 15, repeated frames: 
fps: 16, repeated frames: 
fps: 17, repeated frames: 
fps: 18, repeated frames: 
fps: 19, repeated frames: 
fps: 20, repeated frames: 
fps: 21, repeated frames: 
fps: 22, repeated frames: 
fps: 23, repeated frames: 
fps: 24, repeated frames: 
fps: 25, repeated frames: 
fps: 26, repeated frames: 
fps: 27, repeated frames: 
fps: 28, repeated frames: 
fps: 29, repeated frames: 
fps: 30, repeated frames: 

Proposed solution.

The issue arises because find_image_index(t) (line 115, ImageSequenceClip.py) compares the given time to
self.images_starts, and the floating point arithmetic sometimes causes this to fail.

The fix in #464 ( b7de3cc ) adds a delta (np.finfo(np.float32).eps ~ 1.1920929e-07) to the calculation of the self.images_starts, which now reads:

            durations = [1.0/fps for image in sequence]
            self.images_starts = [1.0*i/fps-np.finfo(np.float32).eps for i in range(len(sequence))]

However, because of the nature of floating point arithmetic, this delta is only sufficiently large for numbers close to 1.0.
i.e.

(1.0 + np.float32(1.1920929e-07) == 1.0, # → False 
 1.5 + np.float32(1.1920929e-07) == 1.5, # → False
 2.0 + np.float32(1.1920929e-07) == 2.0, # → True
 2.5 + np.float32(1.1920929e-07) == 2.5, # → True
)

A simple solution is to increase the time delta, for example to 1e-6.

This solution would the above test:

import moviepy.video.io.ImageSequenceClip
import numpy as np

# set of frames that are all different
sequence = [np.ones((400, 400, 3))*f for f in np.linspace(0, 1, 1000)]
sequence = sequence*10
for fps in range(1,31):
    print(f"fps: {fps}, repeated frames: ", end='')
    movie = moviepy.video.io.ImageSequenceClip.ImageSequenceClip(sequence, fps=fps)
    ### changed l101+
    movie.durations = [1.0 / fps for image in sequence]
    movie.images_starts = [
        1.0 * i / fps - 1e-6 for i in range(len(sequence))
        ]
    movie.duration = sum(movie.durations)
    movie.end = movie.duration
    ### fin
    last_frame = sequence[-1]
    for i, frame in enumerate(movie.iter_frames()):
        if np.any(frame != sequence[i]):
            print(f" {i}", end='')
        last_frame = frame
    print('')

(which tests up to 1k frames)

An alternative solution is to change find_image_index():
i.e. from:

        def find_image_index(t):
            return max(
                [i for i in range(len(self.sequence)) if self.images_starts[i] <= t]
            )

to

        def find_image_index(t):
            return max(
                [i for i in range(len(self.sequence)) if self.images_starts[i] < t + 1e-6]
            )

But in this case the changes from b7de3cc should probably be reverted.

Specifications

• Python Version: 3.12
• MoviePy Version: 2.1.2
• Platform Name: Win10