UDC 004.932.2:629.735
TECHNICAL STACK MODEL FOR ONBOARD IMAGE PROCESSING OF UAVS
R. V. Khrunichev, PhD (in technical sciences), associate professor, Department of Electronic Computers,
RSREU, Ryazan, Russia;
orcid.org/0009-0006-1442-0649, e-mail: This email address is being protected from spambots. You need JavaScript enabled to view it.
The paper considers the problem of forming the technical stack of onboard image processing for an unmanned
aerial vehicle (UAV) when implementing the systems for automatic object detection and recognition.
The relevance of the study is determined by the need to deploy computationally intensive computer vision
algorithms and neural network-based detection directly on board a UAV under strict constraints on the performance of computing modules, memory bandwidth, power consumption, and acceptable video processing
latency. The aim of the work is to develop a mathematical model of technical stack of onboard image processing that ensures the required frame processing rate and object recognition performance under limited
computational resources of onboard platform. Models of computational costs were developed for the stages
of preliminary image processing, convolutional neural network detection, and post-processing of recognition
results. A memory latency model was developed, and an optimal distribution of computations between central
processing unit, graphics processing unit, and neural processing unit was obtained. Estimates of admissible
parameters of computing architecture were received, and the distribution of computations between
graphics processing unit and neural processing unit was shown to allow achieving near real-time operation
with acceptable power consumption.
Key words: image processing, UAV, onboard computing system, convolutional neural network, technical
stack, memory bandwidth, power consumption, computational optimization