Automated Deep Learning Analysis of Purple Martin Videos Depicting Incubation and Provisioning
Machine Learning/Artificial Intelligence
TimeTuesday, July 302:30pm - 3pm
DescriptionDeep learning models have been developed to analyze automatically video clips of Purple Martin nesting behavior. Two separate models have been constructed, one for incubation and one for provisioning. The incubation model is a simple two class model that analyzes the videos to determine if an adult Purple Martin is incubating the eggs/young nestlings or not. The model is a Keras/Tensor Flow convolutional neural network (CNN) trained with ~12 thousand still images and achieves a validation data set accuracy of ~99.5% on still images. A comparison of the results of the automated video analysis with sample validation videos viewed manually shows good agreement; the model approaches human accuracy. Some conclusions from the incubation analysis will be discussed. The provisioning analysis requires a much more complex 3 class model which must distinguish between zero, one parent or both parents on the nest. With training sets including ~26 thousand images the CNN model demonstrates a validation set accuracy of ~99% on the still images. However, the actual the video analysis still presents difficulties. Several different CNN models have been tried but results were similar. The best results to date on analyzing the videos for provisioning events have been ~88% accuracy with ~10% false positives. A discussion of the conclusions from the provisioning model and model analysis will be presented.