As the crows fly — Video to Photo with MATLAB

Inspired by the work of Xavi Bou I try to create an image where the flight paths of birds over time are shown in a single image.

As my camera only shoots stills at 3 fps, I opted to take a short video of birds flying, as this allowed for up to 60 fps while sacrificing some quality. Some frames from the video are shown:

Due to the long lens and a lack of a tripod, there is some movement in the background as well as the movement in the birds. This is what caused me the most anguish and what I’ll try correct next time.

Now to layer the frames. The best way of doing this would be to extract every frame, layer them up in photoshop and take the differences between them to create a composite image. However, for some reason, I decided to try do this in MATLAB.

This made extracting the frames as easy as:

obj = VideoReader('Bird.MOV');
frame1 = readFrame(obj);
frame2 = readFrame(obj);
...
frameN = readFrame(obj);

My first attempt was to start with the first frame, and for every subsiquent frame add on the absolute difference between that frame and the previous frame.

obj = VideoReader('Bird.MOV');
image = readFrame(obj);
frame1 = image;
for i = 1:400
    current = readFrame(obj);
    K = imabsdiff(frame1, current);
    frame1 = current;
    image = image*1.012 - K;
end

The movement in the background caused small differences frame to frame, which were then added to the image. The 1.012 constant was needed to prevent the pixels from tending to 0, but even with this the small differences in the background compounded to create this vision of Hell:

Can you spot Lucifer?

To work around the movements in the background, a method for isolating the desired movement (the dark birds) from the undesirable moment (the lighter background) is needed. Luckily the lighting here makes using a threshold possible to do just this.

image = (imbinarize(readFrame(obj)));
%Takes the threshold of the frame and returns a logical matrix

The full code is as follows:

obj = VideoReader('Bird.MOV'); %Load video frames as before:
background = readFrame(obj); %Set a static background
for i = 1:293
    current = readFrame(obj); %Get next frame
    BW = (imbinarize(current)); %Take threshold
    BW = BW(:,:,1); %Only one layer of pixels is needed
    for i = 1:720;
        for j = 1:1280;
            if BW(i,j) == 0
                %Where the threhsold is 0 (black), set the pixel in the background to the pixel in the current frame.
                background(i,j,1) = current(i,j,1);
                background(i,j,2) = current(i,j,2);
                background(i,j,3) = current(i,j,3);
            end
        end
    end
end

While the looping is easy to follow this could also be done using matrix arithmatic:

Background = Background + not(imbinarize(current)) * current

This yields a far less demonic result:

Hope you enjoyed this short exploration in photography, please check out my profile for more side projects