An electronic device and its operating method for controlling the performance of at least one processor when providing inference services through deep learning models.

Deep learning inference device that adjusts processor frequency based on layer milestone latency measurement

This technology is an electronic device that pre-assigns milestone identifiers to specific layers of a deep learning model and dynamically adjusts the processor frequency based on the latency value, which is calculated by comparing the time taken to reach those points during inference against a target time.

Trained deep learning models are used for inference on mobile devices such as smartphones, surveillance cameras, and autonomous vehicles. However, these devices have limited processor performance and power supply, making it difficult to balance inference latency and power efficiency.

This technology compresses models based on QoS, assigns milestones to layers operating at specific time intervals through profiling, and increases or decreases performance based on how far the latency value deviates from the threshold range. It can be applied to smartphones and edge cameras equipped with on-device AI, ensuring inference results are delivered within a predictable timeframe while reducing power consumption caused by unnecessary clock speed increases.

‍

‍

Key Features:

• A layer management unit that identifies pre-assigned milestone identifiers among multiple layers of a deep learning model during input data computation
• A performance verification unit that measures the time taken from the start of computation to the milestone identification point and calculates a latency value as a ratio against the target time
• A latency control unit and resource management unit that generate performance increase or decrease commands based on the extent to which the latency value deviates from the threshold range and changes the processor frequency
• A configuration that compresses models according to QoS data and assigns milestone identifiers based on the operating time per layer obtained through profiling

‍

This invention was developed with support from the Ministry of Education's project for developing core technologies for low-power, real-time, lightweight deep learning software frameworks for intelligent mobile/embedded systems.

‍