Mathematical Methods for IoT-Based Annotating Object Datasets with Bounding Boxes

Object datasets used in the construction of object detectors are typically annotated with horizontal or oriented bounding rectangles for IoT-based.e optimality of an annotation is obtained by fullling two conditions: (i) the rectangle covers the whole object and (ii) the area of the rectangle is minimal. Building a large-scale object dataset requires annotators with equal manual dexterity to carry out this tedious work. When an object is horizontal for IoT-based, it is easy for the annotator to reach the optimal bounding box within a reasonable time. However, if the object is oriented, the annotator needs additional time to decide whether the object will be annotated with a horizontal rectangle or an oriented rectangle for IoT-based. Moreover, in both cases, the nal decision is not based on any objective argument, and the annotation is generally not optimal. In this study, we propose a new method of annotation by rectangles for IoT-based, called robust semi-automatic annotation, which combines speed and robustness. Our method has two phases. e rst phase consists in inviting the annotator to click on the most relevant points located on the contour of the object. e outputs of the rst phase are used by an algorithm to determine a rectangle enclosing these points. To carry out the second phase, we develop an algorithm called RANGE-MBR, which determines, from the selected points on the contour of the object, a rectangle enclosing these points in a linear time. e rectangle returned by RANGE-MBR always satises optimality condition (i). We prove that the optimality condition (ii) is always satised for objects with isotropic shapes. For objects with anisotropic shapes, we study the optimality condition (ii) by simulations. We show that the rectangle returned by RANGE-MBR is quasi-optimal for the condition (ii) and that its performance increases with dilated objects, which is the case for most of the objects appearing on images collected by aerial photography.


Introduction
e construction of an object detector generally goes through a learning phase, followed by a testing phase, and ends with a tuning phase. Each phase requires an independent annotated object dataset. Annotating an image IoTbased signal and image processing applications consists of locating all the objects present in this image and determining their categories. e way to locate an object varies from detector to detector. For example, mask R-CNN detector uses segmentation mask to locate objects [1], CoKe detector uses key points and landmarks to locate objects [2], and poly-YOLO detector represents objects using polygons [3]. However, the rectangle is considered to be the simplest and most used polygonal shape for locating or representing an object in many computer vision applications for IoT-based [4]. Locating an object using a rectangle consists of drawing a rectangle surrounding this object. e annotation of an object dataset is performed using free or commercial software designed for a particular annotation choice.
ere are two types of annotations with rectangles depending on the orientations of the objects in the images for IoT-based. To locate objects, the rst type uses horizontal bounding rectangles (HBR), while the second type uses oriented bounding rectangles (OBR). Annotation with horizontal rectangles is suitable for natural scenes, where the photographer is usually in front of the object and adjusts their camera so that the objects appear aligned with the horizontal edges of the image. By contrast, for aerial photography, where images are captured by Earth observation satellites or other flying devices, objects often appear in the image with arbitrary orientations. e optimality of an annotation results in the satisfaction of two conditions: (i) the bounding rectangle must cover the whole of the object and (ii) the area of the bounding rectangle must be minimal. For a horizontal object, it is easy to reach the optimal HBR that satisfies conditions (i) and (ii) mentioned above. On the other hand, for an oriented object, it is difficult to reach the optimal OBR if the annotation method does not render account of optimality conditions (i) and (ii).
In the literature, we distinguish two types of object detectors. e first type includes horizontal object detectors, which detect objects using HBR. e second type includes oriented object detectors, which detect objects using OBR. Any detector of the first class is trained, tested, and tuned on horizontal object datasets (HOD). On the other hand, some oriented object detectors are trained and tuned on HOD, such as OAOD [5] and BBAVectors [6], but others are trained tuned on to oriented object datasets (OOD), such as RoI transformer [7], R-RoI [8], RRPN [9], R2CNN++ [10], DMPNet [11], FOTS [12], RPN [13], and DDR [14]. However, all of these oriented object detectors must be tested on OOD.
A large-scale HOD (OOD, respectively) is a set made up of a large number of images annotated with HBR (OBR, respectively). Each image contains objects of a wide variety of scales, orientations, and shapes. ese objects are divided into classes (or categories) that vary from one dataset to another. Table 1 (Table 2, respectively) presents the number of classes, instances, images, and year of creation of the most cited HOD (OOD, respectively) in the literature. To our knowledge, DOTA is the largest public Earth vision object detection dataset [27]. It contains objects exhibiting a wide variety of scales, orientations, and shapes. Moreover, images of DOTA are manually annotated by experts in aerial image interpretation.

State of the Art
In order to generate a dataset for object identification, we need to gather a large number of photographs, all of which must be annotated using the same technique and organized in accordance with a set of categories that have been predetermined. e dataset has to include many photographs that match to various instances of each object class. ese images must be included for each object class. e currently available annotation techniques are amenable to being split up into two primary categories. In the first class, all methods of manual annotation are grouped together, and in the second class, all techniques of semi-automatic annotation are categorized together [28].

Manual Annotation.
ere are two main methods of manually annotating objects with bounding boxes, which are used in the construction of most large-scale object datasets. e first one is called the consensus method, and the second one is called the sequential tasks method. For each instance of an object in an image, the first method asks several annotators to draw a rectangle around the object and then defines the position of the object by the rectangle elected by the majority of annotators. To annotate an object by the second method, we need at least three annotators. e first one is asked to draw a rectangle around a single instance of the object. e task of the second annotator is to validate the drawn rectangle. e third person investigates whether additional object class instances require annotation. Determining a precise bounding rectangle takes more time and resources than validating the annotation, and thus, the consensus technique is more efficient. Annotation quality affects the development of accurate object detectors. e determination of a large-scale object dataset requires expert annotators, significant working time, and remuneration in line with the desired accuracy. As examples of common use datasets, annotated with the manual methods, we cite PASCAL VOC [18], MS COCO [17], ImageNet [19], DOTA [20], etc.
Although studies on the description of OOD are abundant, very few of them explain how to draw a rectangle enclosing an oriented object. e only annotation method explicitly described in the literature appears in the article [27] by Ding et al. is method consists in drawing a HBR around the object of interest and then adjusting the angle manually by rotating the rectangle with the mouse.

Semi-Automatic Annotation.
e traditional semi-automatic approach of annotating objects with rectangles consists of four phases.
First phase: in this phase, the images of the object dataset are divided into two subsets of unequal sizes. e smallest subset is annotated with rectangles by the manual method. Second phase: the second phase consists of choosing an object detector and training this detector with the images from the dataset already annotated.  2 Mathematical Problems in Engineering ird phase: once the detector is ready, it will be used to annotate the images of the second object dataset with the prediction rectangles. Fourth phase: during this phase, the annotator validates the correctly annotated objects during the third phase and manually draws the bounding rectangles of the poorly annotated objects.
As this study is limited to locating objects by bounding boxes, we cite the Faster Bounding Box (FBB) as an example of semi-automatic annotation method [16]. is method was used to generate the Tampere University of Technology (TUT) Indoor dataset. e largest subset is annotated with prediction rectangles, generated by the Faster R-CNN object detector [29], trained on the smallest annotated subset.

Performance Measure of Detectors Using Rectangles.
e evaluation of the effectiveness of the object detector is then performed on the second subset of the annotated dataset. Since we are only dealing here with annotations using bounding boxes, the most used criterion to compare two rectangles is the Jaccard's similarity index, also known as the Intersection over Union (IoU). For each image of the testing dataset, the IoU measures the percentage of overlap between the prediction bounding rectangle A generated by the detector, and the ground truth bounding box A, as follows: We note that the IoU score lies in the interval [0, 1]. e closer this index gets to 1, the better the detection of the object. We say that an object is well detected (or true positive) if the IoU score is greater than or equal to a threshold discussed by experts. In most studies, the threshold value is set at 0.5 [30]. e IoU score causes a problem when its value is zero. is value is embarrassing since it does not explain how far is the prediction bounding rectangle from the ground truth bounding box. To work around this problem, Rezatofighi et al. [30] suggest replacing the IoU with the Generalized Intersection over Union (GIoU) index. e GIoU between two rectangles A and A is defined as follows: where C is the smallest convex set enclosing both A and A. e exact IoU computation between two HBR is simple. Much software integrates functions to calculate this index. On the other hand, the exact IoU computation between two OBR is not as simple as that between two HBR. Liu et al. (Yao et al., respectively) proposed in Ref. [31] (in [32]) heuristic to estimate the IoU between two OBR. Recently, Zaidi has shown in Ref. [28] that these heuristics give reliable results only when the centers of the rectangles are very close and the angle between the rectangles is small. Moreover, he has developed in Ref. [28] an algorithm that calculates the exact IoU value between two OBR, as well as an (ε, α) estimator of the IoU. e annotation method has a direct bearing on the performance measurement of an object detector. e OOD reserved for the test phase must be carefully annotated so that the IoU computation between the ground truth and prediction rectangles is not biased.

Motivation for the Study
e manual or semi-automatic annotation methods described in Section 2 have made it possible to build many large-scale object datasets. ese datasets have given rise to very powerful object detectors. However, we cannot ignore the following concerns: (1) According to paragraph A, accurate manual annotation is expensive, time-consuming, and requires annotation experts. (2) In many situations, the annotator can be in front of an object with an ambiguous orientation as shown in Figure 1. In this case, he will take a considerable time to decide whether the object will be annotated with a horizontal rectangle or an oriented rectangle. Moreover, in both cases, the final decision is not based on any objective argument and depends solely on the dexterity of the annotator. (3) e semi-automatic annotation defined in paragraph B does not guarantee the optimality of the annotation rectangle, in the sense that it must have a minimum area and cover the whole of the object. (4) According to paragraph C, the use of any approximation methods of IoU(A, B), whether to measure the accuracy of an oriented object detector or to compare the performance of two oriented object detectors, could lead to biased results. Indeed, the function g(x) � x/a + b − x is strictly increasing over the interval [0, a + b[, where a and b denote the areas of A and B, respectively. erefore, a large or a small ground truth bounding rectangle directly induces a bias in the computation of the IoU. e contribution of this study is the scarcity of annotation methods motivated us to develop a robust semi-automatic annotation method of OOD. Our method is semiautomatic because it consists of a manual step followed by a computer-assisted step. In addition, this method is robust because the bounding rectangle generated by our algorithm is insensitive to the dexterity of the annotator. More precisely, the steps are as follows: is study seems essential to us because it responds to a real need in the development of oriented object detectors. In addition, it allows researchers to build their own oriented object datasets in a rigorous manner. Definition 1. (relevant point of an object) Any point located on the contour of the object is said to be relevant if it is a local maximum, a local minimum, the most right point, the most left point, a cusp of the first type, a cusp of the second type, an inflection point, and so on. e remainder of this manuscript is structured as follows. Section 4 deals with the parametrization of rectangles. Section 5 is devoted to the development of the RANGE-MBR algorithm. We explain in Section 6 how to use the RANGE-MBR (or RC-MBR) algorithm to simultaneously generate horizontal and oriented datasets from a large-scale image bank. In Section 7, we perform various numerical experiments to evaluate the performance of the RANGE-MBR algorithm.

Parametrization of Bounding Rectangles
First, we give an overview of the different types of parameterization encountered in the literature. en, we justify the choice of the parameterization associated with the annotation method that we propose in this study. Our choice of parameterization was discussed in a previous study on the accurate computation of the IoU [28]. However, it seems useful to us to recall this choice to facilitate the reading of the manuscript.

Parameterization of Horizontal Rectangles.
e minimum parameterization of a horizontal rectangle Figure 2). e object dataset Ms COCO [17] annotates the rectangle A with (x 4 , y 4 , w, h), where w and h denote the lengths of the line segments , respectively, and (x 4 , y 4 ) are the coordinates of the vertex A 4 (cf. Figure 2). e object dataset ImageNet [19] annotates the rectangle A with (x 0 , y 0 , w, h), where w and h denote the lengths of the line segments [A 1 , A 2 ] and [A 1 , A 4 ], respectively, and (x 0 , y 0 ) are the coordinates of the center A 0 of A (cf. Figure 2).  Figure 2). e parameter θ ∈ [0, π/2] is used to determine the acute angle that exists between the lines Δ and (B 1 , B 2 ), where Δ denotes the horizontal line that goes through B 1 . ese two object datasets, HRSC2016 [22] and UCAS-AOD [23], are annotated with the help of this parametrization. e above θ-based parametrization is not suitable for annotating satellite image datasets.

Parameterization of Oriented
ese OOD have the particularity of containing a large number of instances per image, at different scales, and whose parts overlap [20]. A simple solution that overcomes the drawbacks of the θ-based annotation consists in defining a rectangle B � B 1 , B 2 , B 3 , B 4 } using the coordinates (x i , y i ) 1 ≤ 4 of the vertices B 1 , . . . , B 4 , respectively. To solve the indeterminacy problem linked to the permutations of the vertices, we can sort the vertices clockwise and fix the first point according to the following rules [27]: (i) For objects with a distinguished head and a tail (e.g., vehicles and helicopters), the annotator carefully selects B 1 to indicate the left corner of the instance head. (ii) For other objects (e.g., tennis courts and bridges), B 1 is the point at the top left of the instance.
is type of parametrization was used for the annotation of the DOTA object dataset [20]. 4 ] to denote an oriented rectangle whose vertices are R 1 , R 2 , R 3 , and R 4 . We assume that these vertices are sorted in counterclockwise order, such that R 1 is the left vertex with the smallest vertical coordinate.

Our Choice of Parametrization. We use the notation
To sort the vertices set B 1 , B 2 , B 3 , B 4 of the rectangle B as described above, we first consider the barycenter G of the rectangle B. en, we calculate the polar angle of each vertex relative to the barycenter G [28]. We denote by B (1) , . . . , B (4) , the sorted vertices in ascending order of their polar angle. If B (3) (2) ≤ B (4) (2), then B � [B (3) , B (4) , B (1) , B (2) ], else B � [B (4) , B (1) , B (2) , B (3) ] (cf. Figure 2). For more details on the implementation of this sorting method, we refer the reader to our article [28].
In this investigation, we describe a bounding rectangle B by using the eight-tuple ( is choice of parameterization is ideally suited to the annotation approach that we suggest since it eliminates the need for extra computations to draw rectangles or to locate the images of rectangles by applying affine transformations.
is is one of the reasons why the method is so efficient. In addition, we demonstrate in Section B how to format the outputs of the annotation technique that we propose in accordance with the annotation rule of the DOTA dataset.
is is done by referring to the annotation rule.

Determination of the Bounding Rectangle
It is natural to annotate the object with the minimum rectangle enclosing the set of relevant points. is problem is formulated as follows: . . , n}, we find a rectangle with the smallest area, enclosing M n . Such a rectangle is called a minimum bounding rectangle of M n and denoted by MBR (M n ). e solution of problem 5.1 is not unique as shown in Figure 3. So, MBR (M n ) denotes any solution of Problem 1.
Freeman and Shapira proved in 1975 that one edge of MBR (M n ) must be collinear with an edge of the convex hull (CH (M n )) of M n . ey proposed in Ref. [33] a natural algorithm to find MBR (M n ) in O(n 2 ) time, based on sweeping all minimum rectangles, enclosing M n , and having an edge collinear with an edge of CH (M n ). In 1978, Shamos proposed in Ref. [34] the famous rotating calipers algorithm, which return all pairs of antipodal vertices of a n-sided convex polygon in O(n) time. In 1983, Toussaint used the rotating calipers technique and developed the algorithm RC-MBR to find MBR (M n ) in O(n ln(n)) time [35]. In 2006, Dimitrov et al. proposed in Ref. [36,37] the algorithm PCA-MBR, to approximate MBR (M n ), in O(nln(n)) time, by the minimum bounding rectangle, which is aligned with the eigenvectors of the covariance matrix of CH (M n ). ey also proved that the relative error between PCA-MBR (M n ) and MBR (M n ) is bounded from above by 36. e main drawback of the algorithm PCA-MBR is that it admits an infinite number of solutions if the covariance matrix of CH (M n ) has a double eigenvalue. To overcome the problem of nonuniqueness inherent in algorithms RC-MBR and PCA-MBR, we propose the method RANGE-MBR to approximate MBR (M n ) in O(n) time.

e Range-MBR Algorithm.
Let B � (O, i, j) be a Cartesian frame of the two-dimensional affine plane. All angles we refer to here are measured counterclockwise from the positive x-axis. Let Δ be the line passing through O and making an angle θ with the axis (O, i). Let Δ ⊥ be perpendicular to Δ and passing through O. Let P 1 and Q 1 (P 2 and Q 2 , respectively) be the extreme points of the orthogonal projection of M n on Δ (Δ ⊥ , respectively). Let 4 ] be the minimum bounding rectangle having an edge collinear with Δ. en, A 1 , . . . , A 4 are the intersection points of the two parallel lines to Δ ⊥ and passing through P 1 and Q 1 , respectively, with the two parallel lines to Δ and passing through P 2 and Q 2 respectively. e area of A is given by the product of lengths of the line segments [P 1 , Q 1 ] and [P 2 , Q 2 ]. From now on, the MBR A enclosing M n and having an edge collinear with the direction making an angle θ from the positive x-axis will be denoted by MBR(M n , θ).
Let u � (cos(θ), sin(θ)) t , and v � (− sin(θ), cos(θ)) t ) be two unit direction vector of Δ and Δ ⊥ , respectively. en, B � (O, i, j) and B ′ � (O, u, v) are two Cartesian frames of the two-dimensional affine plane. For all i � 1, . . . , n, we denote by (x i , y i ) t ((s i , t i ) t , respectively) the coordinates of M i with respect to the frame B (B ′ , respectively). Let R θ be the rotation matrix of angle θ, and then ( where u t is the transpose of u. Let X be a random variable with mean μ and standard deviation σ. Let x 1: n � (x 1 , . . . , x n ) be n observations on the We define the range of x 1: n by Range(x 1: n ) � x (n) − x (1) . erefore, the lengths of [P 1 , Q 1 ] and [P 2 , Q 2 ] are given by: We denote by σ n any estimation of σ obtained from x 1: n . On the basis of numerous works carried out on the estimation of the standard deviation from the sample range ( [38][39][40][41][42][43]), we can assume that there is a real constant α(X, n) such that Besides, it is well known that which is an estimate of σ 2 , where x � 1/n n i�1 x i . It follows that there is a real constant α(X, n) such that: We assume that the (x, y) coordinates of the set of points M n are n observations of two random variables X and Y. Since the Area(A) � P 1 Q 1 · P 2 Q 2 , then combining equations (6) and (3) gives an approximation A 2 n (θ) of Area(A), satisfying the following relation: where (S, T) t � R t θ (X, Y) t . To alleviate notions, we will also use V x and C x,y to designate Var(x 1: n ) and Cov(x 1: n , y 1: n ).
Using (8) and some trigonometric identities, we prove that where K � V 2 Combining equations (7), (9), and (10) gives us, the area of MBR (M n , θ) is approximately equal to erefore, we propose to approximate MBR (M n ) by MBR (M n , θ * ) such that: e first derivative of f with respect to θ is given by: where e function f has a unique critical point θ * such that

Lemma 1.
e second derivative of f at the critical point θ * has the same sign as − λ. Proof.
is case is discussed below. □ 5.1.1. Illustrative Example of Case 6. It is difficult to illustrate all the scenarios, which fall into Case 6. e special case of regular polygons is the one that the annotator may encounter when annotating regular objects such as road signs, buildings, and human faces. Moreover, we have pointed out this case to warn the user to pay attention to regularly shaped objects when using our annotation method.

Proposition 1.
If the elements of M n are the vertices of a regular n-sided polygon, then V x � V y and C x,y � 0.
Proof. In the general case, the vertices of a regular polygon are uniformly distributed over a circle with radius r and center G equal to the barycenter of M n . us, without loss of generality, we can assume that r � 1, G � (0, 0) t , and the k th vertex of M n is Consider the complex sequence (z k ) 0 ≤ k ≤ n− 1 , where z � exp(2π/ni) and i is the imaginary unit. en, Since n− 1 k�0 z k � (1 − z n /1 − z) � 0, then x � y � 0, and C x,y � 1 n n k�1 x k y k , Since cos(2x) � 2 cos 2 (x) − 1 � 1 − 2 sin 2 (x), then Besides, n− 1 k�0 cos(4πk/n) is the real part of us, V x � V y � 1/2. Moreover, using 2 cos(x)sin(x) � sin(2x), we deduce that Since n− 1 k�0 sin(4πk/n) is the imaginary part of then C x,y � 0.  (i) For Case 2, the application of an affine isometry to the set of points (M n ) makes it possible to migrate to another case for which the angle θ is well determined. We then obtain the solution of the initial problem by applying the inverse isometry to the solution of the transformed problem, since isometry preserves the areas. (ii) For Case 6, the use of an extra point M n+1 belonging to the convex hull of the set M n , allows to migrate to another case for which the angle θ is well determined. Another alternative consists in asking the user to click on a point of the object's outline until we get out this state.
x,y is negative because of C 2 x,y . To get rid of C 2 x,y , we have just to follow the same reasoning on a set of points whose coordinates x 1: n ′ � (x 1 , . . . , x n ′ ) and y 1: n ′ � (y 1 , . . . , y n ′ ) are uncorrelated. Let

Mathematical Problems in Engineering
be the covariance matrix of the elements of M n , where M � (x, y) t . Let Γ � UΔU t be an eigen decomposition of Γ, where U ∈ O 2 (R) (the set of 2-orthogonal matrices), and where Proof. Since x 1: n and y 1: n are uncorrelated, then C x,y � 0, and λ � (V x − V y ) 2 and δ � 0. According to equation (16), the function f has a unique critical point θ * � 0. Using Lemma 1, f(θ) is the maximum of f(θ). Case 6. To get out of the indeterminacy of Case 6, we propose to add to the set M n an artificial point M n+1 located in the convex hull of M n so that the empirical covariance matrix of the new set M n+1 � M n ∪ M n+1 is different from a scalar matrix. We assume that (x n+1 , y n+1 ) t are the Cartesian coordinates of M n+1 with respect to B G � (G, i, j), where G � (x, y) t is the barycenter of M n . Without any loss of generality, we assume that x 1: n+1 (y 1: n+1 , respectively) is the vector of the x-coordinates (y-coordinates, respectively) of the points of M n+1 with respect to B G � (G, i, j). We denote by V x � Var(x 1: n+1 ) and V y � Var(y 1: n+1 ). Lemma 2 gives the relation between V x , V y , C x,y and V x , V y , C x,y , respectively.
C x,y � n (n + 1) 2 x n+1 y n+1 . (31) Proof. We set x � (1/n + 1) n+1 i�1 x i and x � (1/n + 1) n+1 i�1 y i . Since x � y � 0, then x � x n+1 /n + 1 and y � y n+1 /n + 1. On the one hand, since x � 0, we have On the other hand, since n+1 i�1 x i y i � 0, we have: x i y i − (n + 1)xy � n n + 1 x n+1 y n+1 . (33) Let f(θ) be the function defined from M n+1 in the same way that f(θ) was defined from M n in (10). en, where Using the identities (31), we have It follows that where D 1 , . . . , D 4 are the lines whose Cartesian equations are given by In summary, any point M n+1 � (x + x n+1 , y + y n+1 ) t defined as a convex combination of M 1 , . . . , M n , such that x n+1 y n+1 ≠ 0 overcomes the indeterminacy problem posed by Case 6. However, the resulting RANGE-MBR (M n ) depends on the values of x n+1 and y n+1 . We tested this technique on a set M n composed of vertices of a regular n-sided polygon, and we observed that the choice We allow our algorithm to reach MBR (M n ) whatever the value of n and j � 1, . . . , n. e generalization of the technique applied to the vertices of a regular polygon requires the calculation of CH (M n ), and the determination of the midpoint, M n+1 , of any two consecutive vertices of CH (M n ), such that the product of the coordinates of M n+1 with respect to B G � (G, i, j), is different from 0.
Mathematical Problems in Engineering

Annotate the DOTA Dataset Using Range-MBR.
To adapt our annotation method to the output format required by the DOTA dataset, it suffices to ask the annotator that the first click points to the object's head, and to add a binary variable T equals to 1 if we can differentiate the head of the object from its tail, and zero otherwise. When we pass the set M n to the RANGE-MBR algorithm, we save the coordinates of the first point M 1 . Next, we determine the rectangle (1) If the binary variable T equals one, then we determine the vertex R j of R that is closest to M 1 , and we permute the vertices of R clockwise so that R j occupies the first position in this permutation. (2) If the binary variable T equals zero, then we determine the top left vertex R j of R, and we permute the vertices of R clockwise so that R j occupies the first position in this permutation.

Robust Semi-Automatic Annotation
In this section, we provide a new approach for building object detection datasets based on both MBR algorithm and threshold angles. is method is semi-automatic because it consists of a manual step followed by a computer-assisted step. In addition, this method is robust because the bounding rectangle generated by our algorithm is insensitive to the dexterity of the annotator.
(i) e problem of MBR was dealt with in Section 5. If the user needs an optimal annotation in the sense given by Definition 2, then he calls the RC-MBR algorithm. Otherwise, he calls the RANGE-MBR algorithm for quasi-optimal annotation. However, optimality and complexity are inversely proportional.
(ii) By default, the threshold angle is equal to zero. It can also be adjusted by experts in object detection, or it can be defined experimentally as the largest angle between the ground truth bounding box and the positive x-axis that gives no significant difference between the performance of horizontal and oriented detectors when tested on oriented objects. e experimental determination of the threshold angle requires horizontal and oriented object detectors, as well as the already existing oriented object datasets.

Properties of the Robust Semi-Automatic Annotation
Method. By construction, the robust semi-automatic annotation method ensures the following properties: (1) e bounding rectangle generated by our approach is quasi-optimal in the sense that it covers the whole object, and its area is close to the area of the MBR enclosing the object. (2) e angle of the rectangle is determined by an algorithm based on some relevant points collected on the contour of the object.
(3) e bounding rectangle is insensitive to annotators provided all relevant points on the object have been selected. (4) e determination of the bounding rectangle requires O(n) elementary operations, where n is the number of relevant points. (5) It allows the user to build simultaneously, from an image bank, two databases: one for horizontal objects and another for oriented objects.
In summary, the robust semi-automatic annotation provides a simple solution to all the drawbacks mentioned in paragraph III, which are inherent in the old annotation methods. Let:

Experimental Study
is experimental study was carried out exclusively with the Matlab R2007b software. We implemented the RC-MBR, PCA-MBR, and RANGE-MBR algorithms in MATLAB language, and we wrote a script (see the Appendix), which (1) reads the image then displays it (2) ask the annotator to click on the most relevant points of the object (M n is a 2 × n matrix) is experiment consists of studying the optimality condition (ii) of the RANGE-MBR algorithm. Equation (28) describes how to generate the coordinates of the vertices M n � (x k , y k ); k � 1, . . . , n of a random n-sided polygon where each rand is call of a generator of uniform random numbers on the interval [0, 1]. e parameter v y , called the factor of dilation, controls the aspect ratio of the polygon as shown in Figure 6. e more v y is greater than 1, the more the polygon is dilated in the direction of the y-axis. Note that such a polygon is not necessarily convex as shown in Figure 7.
is is also the case for any polygon whose vertices are defined by the relevant points of an object. Since (1) the optimal annotation criterion that we have chosen results in a rectangle which has a minimum area and which covers the maximum of visible parts of the object, (2) RC-MBR is the fastest algorithm that determines the smallest rectangle enclosing a set of points, it seems natural to consider this algorithm as a reference in the comparative study that we carried out. Mathematical Problems in Engineering For each value of the pair (n, v y ), we generate r � 10000 random n-sided polygons. For each polygon M (i) n , i � 1, . . . , r, we determine the relative error e i , between the areas of RANGE-MBR (M (i) n ) and RC-MBR (M (i) n ), as well as the CPU times t i and t i , used by the algorithms RANGE-MBR and RC-MBR to compute RANGE-MBR (M (i) n ) and RC-MBR (M (i) n ), respectively. Finally, we denote by e, t, t, and Std(e), the means and the standard deviation of the sequences (e i ) 1 ≤ i ≤ r , (t i ) 1 ≤ i ≤ r , (t i ) 1 ≤ i ≤ r , and (e i ) 1 ≤ i ≤ r respectively. We have retained the ranges of values [4,20] for n and [1,4] for v y , because in our estimation, they are those which correspond most to reality.
In practice, rare are objects that have more than 20 relevant points or less than 4 relevant points.
We deduce from Figures 8 and 9 that: (1) e more the polygon is dilated in one direction, the more the algorithm RANGE-MBR is accurate and precise (2) e more vertices the polygon has, the more the algorithm RANGE-MBR is accurate and precise Δ Δ y Figure 5: When θ is fixed, the minimum bounding rectangle enclosing the blue polygon (object) is the rectangle A whose vertices A 1 , . . . , A 4 are the intersection points of the two parallel lines to Δ ⊥ and passing through P 1 and Q 1 , respectively, with the two parallel lines to Δ and passing through P 2 and Q 2 , respectively. A is the rectangle whose edges pass through the red dotted segments.   Figure 7: P 1 and P 2 are two random heptagons, generated according to Equation (28) with v y � 4. P 1 is concave, while P 2 is convex. We ran other simulations with larger values of n and v y , and we got the same previous conclusion. e parameter v y controls the dilation of the polygon in the vertical direction. In fact, we could choose any other direction, since the RANGE-MBR algorithm is not sensitive to the direction of expansion. On the other hand, it is sensitive to the number of vertices of the polygon and to its dilation.
Out of the 680000 generated polygons, we do not encounter any case for which V x � V y and C x,y � 0. In addition, the algorithm RANGE-MBR is 9 times faster than algorithm RC-MBR. e mean CPU times of RANGE-MBR and CR-MBR are both independent of n and v y . It is about 1.38 × 10 − 4 seconds for the algorithm RANGE-MBR, and 1.20 × 10 − 3 seconds for the algorithm RC-MBR. Since the complexity of the algorithm RANGE-MBR is O(n), and that of the algorithm RC-MBR is O(n ln(n)), then the difference in CPU time, for each algorithm, is only observed if n is large enough. Based on the response time per click given in [44] and considering the mean CPU time of the RANGE-MBR algorithm, we can state that determining a bounding rectangle requires 2.5 + 1.5(n − 1) seconds, where n is the number of relevant points collected on the contour of the image.

Experiment 2.
We made a comparison between RANGE-MBR and PCA-MBR based on sets of vertices of regular n-sided polygons, with n � 4, . . . , 20. We observe that the relative error of the algorithm RANGE-MBR is always equal to 0, while that of the algorithm PCA-MBR is different from 0 for the values of n reported in Table 3. Although Case 6 does not contain only regular polygons, this experiment shows that the RANGE-MBR method achieves optimality on regular polygons, but for the other shapes (which go in Case 6), it offers a better solution than that obtained by PCA-MBR.

Experiment 3.
is experiment consists in studying the effect of the dilation factor on the performance of the RANGE-MBR algorithm. Figure 10 represents a basic experiment. e relevant points M n are colored in yellow. We observe that RANG-MBR (M n ) is close to RC-MBR (M n ). Although the bird on the left is smaller than the bird on the right, the relative error e equals 0.65% for the bird on the right, and 3.12% for the bird on the left. ese scores are expected, since the bird on the right has a more elongated shape than the bird on the left, and RANGE-MBR is more effective on dilated objects.

Experiment 4.
is experiment consists in confirming the conclusion obtained in the experiment D. Figure 11 contains two mangoes. e one on the left is almost circular, while the one on the right is clearly elongated. e relevant points M n are colored in yellow. e red rectangle corresponds to RANG-MBR (M n ), while the green rectangle corresponds to RC-MBR (M n ). e relative error e equals 6.93% for the left mango and 1.86% for the right mango. is real example agrees with the simulation results shown in Figure 8. e mango on the right is more dilated than the one on the left that is why the relative error for the circular mango is greater than relative error for the oval mango.

Experiment 5.
In order to verify the robustness of our annotation method with respect to the annotator, we asked a colleague to click on the relevant points of the two mangoes. Figure 12 illustrates the result of this experiment. e relative error e equals 8.92% for the left mango, and 2.32% for the right mango. e relative errors reported in Figure 12 are larger than those reported in Figure 11. We explain this difference by the number of points used in each experiment: the more points we use, the more the RANGE-MBR algorithm reduces the relative error. is real example agrees with the simulation results shown in Figure 8.

Experiment 6.
is experiment consists in comparing our annotation method to the FBB method introduced in Section B. For this, we have chosen a random image from the   TUT indoor dataset, and we have annotated the objects it contains with FBB, RC-MBR, and RANGE-MBR. Note that the FBB annotation uses blue rectangles, RC-MBR annotation uses green rectangles, RANGE-MBR annotation uses red rectangles, and the relevant points are marked in yellow.
Based on Figure 13 and Table 4, it can be seen that the annotation by the FBB method is not optimal. Indeed, (i) For the upper extinguisher and the exit sign, condition (i) is violated (ii) For the lower extinguisher, condition (ii) is violated In addition, the annotation by the RANGE-MBR method satisfies condition (i) and the relative error between the area of RANGE-MBR and RC-MBR is 12% for the exit sign, 3% for the upper extinguisher, and 0.3% for the lower extinguisher. We can conclude that condition (ii) is almost satisfied by the RANGE-MBR method.
(i) e values of the relative error are in agreement with the simulation results presented in Figure 8. Indeed, in terms of dilation, the lower extinguisher is the most dilated, followed by the upper extinguisher, then the exit sign. (ii) e exit sign and the upper extinguisher have regular shapes, while the lower extinguisher has an irregular shape. We have already underlined in Section 1 and that the RANGE-MBR method is sensitive to regular forms. is example then illustrates the situation of Case 6.
For all these reasons, the annotation of the lower extinguisher by the RANGE-MBR method is the best.

Experiment 7.
is experiment highlights the importance of the annotation method on the calculation of the IoU. Figure 14 corresponds to image P 1128 from the DOTA dataset. e objects of interest in these images are the airplanes. We use the BBAVectors detector [6] to generate the red prediction bounding boxes. Ground truth bounding boxes used in [6] are colored green. e blue rectangles correspond to the annotations of the airplanes by the RANGE-MBR method. For all i � 1, 2, 3, we denote by IoU g i (IoU b i , respectively) the Intersection over Union between the green (blue, respectively) rectangle enclosing the airplane A i and the corresponding red rectangle. e results of this experiment are reported in Table 5. e use of minimum ground truth rectangles best reflects the performance of an object detector. In view of the results of Table V, it is reasonable to rely on the results of the second row rather than those of the first row.

Conclusion
In this piece, we provide a novel approach that is both resilient and semi-automatic in nature for the annotation of objects. According to the results of the experimental research, we are able to assert that (1) Robust semi-automatic annotation is quasi-optimal in the sense that Definition 2 describes, and that its optimality increases with dilated objects, which is the case for the majority of the objects that appear on the images that were collected by the use of aerial photography.
(2) Robust semi-automatic annotation is quick and reliable in the sense that the bounding rectangle does not have any gaps, and is oblivious to the skill of the person doing the annotation.
(3) Robust semi-automatic annotation is simple to put into action and might be easily incorporated into platforms for annotating text.
(4) Robust semi-automatic annotation is sensitive to the annotation of items displaying symmetry, with respect to one or more directions. Given this scenario, it is not appropriate for the relevant points to follow the same symmetry in order for the created rectangle to be somewhat close to the ideal rectangle.
qe critical angle, as described in Section 6, would look somewhat like this: the emphasis on experimental research using large-scale collected data on horizontal and oriented object datasets, as well as detectors of stateof-the-art oriented objects. Once a good estimate of the threshold angle has been made, the robust semi-automatic annotation technique is the one that allows the simultaneous construction, from the same image library, of two different data sets: one for the elements horizontal and another for oriented objects.