[SEC24] ORIC | Ruiqi Wang

Optimizing Edge Offloading Decisions for Object Detection (Qiu et al., 2024).

TL;DR:
This paper introduces ORIC, a metric for quantifying improvements in detection accuracy (mAP) when offloading images to a stronger edge detector. By factoring in the broader context that mAP relies on, ORIC improves upon existing metrics. We also present a practical method to estimate ORIC using only the local detector’s output. Tested across multiple datasets and detector configurations, ORIC achieves over 50% of the strong detector’s accuracy benefits with an offloading ratio of just 20%.

Paper published in: 2024 ACM/IEEE Symposium on Edge Computing (SEC) December 4-7, 2024 Rome, Italy

Introduction

Advancements in machine learning and embedded hardware now allow edge devices to perform real-time object detection, essential for applications like surveillance and traffic monitoring. This project tackles optimizing which images to offload from a resource-limited device to a powerful edge server to maximize detection accuracy within bandwidth constraints.

We introduce Object detection Reward per Image within Context (ORIC)—a metric predicting accuracy gains from offloading each image. ORIC adapts dynamically to offloading budgets and accounts for accuracy across various object classes, achieving substantial improvements even with limited offloading. Code and further details are available on GitHub.

Key Contributions:

ORIC Metric: Quantifies detection improvement potential, enhancing accuracy while minimizing offloaded images.
Efficient Estimation: Deploys an efficient model to predict ORIC from local detector results alone.

Method – ORIC

ORIC seeks to capture differences in mAP produced by individual images. ORIC is computed over a representative image set \(\mathcal{E}\) (the context) rather than a single image. The primary motivation is to more closely reflect the ensemble average nature of mAP.

\[mAPC_i = \text{mAP}(\{h_i, H_{\mathcal{E}}\}) = \frac{1}{E} \sum_{j=1}^E AP_{\{i\} \cup \mathcal{E}, j}\, ,\]

where \(H_{\mathcal{E}}\) denotes the consolidated detection results for the image set \(\mathcal{E}\), \(E\) is the number of classes across image \(i\) and images in \(\mathcal{E}\), and \(AP_{\{i\} \cup \mathcal{E}, j}\) refers to the AP of class \(j\) evaluated on \(\{h_i\} \cup H_{\mathcal{E}}\).

In other words, given an image \(i\), mAPC evaluates the mAP of an object detector on \(\{i\}\cup \mathcal{E}\), i.e., using not just image \(i\), but also the broader context \(\mathcal{E}\). The offloading reward metric \(ORIC_i\) for image \(i\) is then defined as:

\[\begin{aligned} ORIC_i =& (|\mathcal{E}| + 1)\cdot(mAPC_{i, s} - mAPC_{i, w})\\ =& (|\mathcal{E}| + 1)\cdot\big[\text{mAP}(\{h_{i, s}, H_{\mathcal{E}, w}\})\\ & - \text{mAP}(\{h_{i, w}, H_{\mathcal{E}, w}\})\big], \end{aligned}\]

where \(w\) and \(s\) indicate the type of detector (weak or strong). It represents the difference in mAP between offloading and not offloading image \(i\), when evaluating the mAP on \(\{i\}\cup \mathcal{E}\). The term \(\vert \mathcal{E} \vert + 1\) is a scaling factor to normalize ORIC across representative image sets of different sizes.

Estimation of ORIC

To apply ORIC in real-time, offloading decisions must rely solely on the weak detector’s output, without access to ground truth or strong detector results. Here, we introduce a practical approach using a lightweight estimator—a multi-layer perceptron (MLP) regression model—to predict ORIC values in real-time. Fine-tuned through grid search, this MLP model maps rewards to a continuous value space, allowing adaptable offloading thresholds to respond to varying network conditions. Using only the weak detector’s output, the model was trained with cross-validation on standard datasets and is optimized for efficient embedded deployment.

Input Choices for ORIC Estimation

Since ORIC estimation relies on the weak detector’s information alone, selecting optimal inputs is key. We considered several options:

The weak detector’s output, including features from the top 25 bounding box proposals, ranked by confidence.
Feature maps from specific hidden layers in the weak detector, optimized for detecting objects at various scales.

Evaluation results indicate that using feature maps provides only minor improvements over the weak detector’s output alone, especially at lower offloading ratios. Given the constraints of embedded systems, we chose to use only the weak detector’s output as input for the ORIC estimator.

Additionally, recent edge computing approaches that partition deep neural networks for local and edge deployment, allowing early prediction exits, align well with our ORIC-based estimation. This approach provides accurate offloading assessments even at early stages within the detector model, making it versatile for edge AI frameworks with embedded early exits.

MORIC: Improving ORIC’s Estimation

ORIC quantifies the reward (improvement in mAP) for offloading individual images, but any estimator has limitations. Here, we enhance our basic estimator to prioritize accuracy for high-reward images, which are most impactful for maximizing mAP. Additionally, we transform the distribution of ORIC values to better differentiate images that should be offloaded from those that should not.

As illustrated in the cumulative distribution functions (CDFs) of ORIC and ORI across the COCO and VOC datasets, many images have a reward near 0, meaning both detectors perform similarly on those images. This creates challenges for reward estimation, as an estimator could minimize Mean-Squared Error (MSE) by predicting values close to 0. Such predictions could lead to fragile offloading decisions, with small reward differences causing decision flips.

To address this, we transform ORIC values by mapping each value to its CDF position, producing modified ORIC values (MORIC) that uniformly spread in \([0,1]\):

\[MORIC_i = \mathrm{cdf}(ORIC_i).\]

This transformation normalizes offloading rewards into ranks, enhancing the estimator’s discrimination across the reward spectrum. We then train our estimator using a weighted MSE loss \(\mathcal{L}\):

\[\mathcal{L} = \sum_{i=1}^N MORIC_i \cdot \big[\mathrm{estimator}(h_{i, w}) - MORIC_i\big]^2.\]

Here, weighting by \(MORIC_i\) emphasizes accuracy on high-reward images. This approach aligns with practical edge computing requirements, where low offloading ratios (typically under 50%) necessitate focusing on images with high rewards. Furthermore, images that would yield lower rewards if offloaded have minimal impact on mAP, even if occasionally offloaded.

Results

The comparison was carried out over the validation sets from COCO (5,000 images) and VOC (4,952 images), with YOLOv5n and YOLOv5m as the object detector pair. A 5-fold cross-validation was used on both datasets to generate the mAP of each alternative. The results are in the figure, which demonstrates that our MORIC-based approach consistently achieves the highest mAP at all offloading ratios.

For more detailed results and analysis, please refer to our paper.

Collaborators

Washington University in St. Louis: Jiaming Qiu, Ruiqi Wang, Roch Guérin, Chenyang Lu
Northwestern University: Brooks Hu

Last Edited: November 1, 2024

Recent advances in machine learning and hardware have produced embedded devices capable of performing real-time object detection with commendable accuracy. We consider a scenario in which embedded devices rely on an onboard object detector, but have the option to offload detection to a more powerful edge server when local accuracy is deemed too low. Resource constraints, however, limit the number of images that can be offloaded to the edge. Our goal is to identify which images to offload to maximize overall detection accuracy under those constraints. To that end, the paper introduces a reward metric designed to quantify potential accuracy improvements from offloading individual images, and proposes an efficient approach to make offloading decisions by estimating this reward based only on local detection results. The approach is computationally frugal enough to run on embedded devices, and empirical findings indicate that it outperforms existing alternatives in improving detection accuracy even when the fraction of offloaded images is small. Code for the paper’s solution is available at https://github.com/qiujiaming315/edgeml-object-detection.