1 
Industry-oriented Performance Measures for 
Design of Robot Calibration Experiment 
Yier Wu1,2,  Alexandr Klimchik1,2,  Anatol Pashkevich1,2,  Stéphane Caro2,  
Benoît Furet2 
1Ecole des Mines de Nantes, France, e-mail: yier.wu@mines-nantes.fr 
2 Institut de Recherche en Communications et Cybernétique de Nantes, France 
Abstract. The paper focuses on the accuracy improvement of geometric and elasto-static calibration of 
industrial robots. It proposes industry-oriented performance measures for the calibration experiment 
design. They are based on the concept of manipulator test-pose and referred to the end-effector location 
accuracy after application of the error compensation algorithm, which implements the identified para-
meters. This approach allows the users to define optimal measurement configurations for robot calibra-
tion for given work piece location and machining forces/torques. These performance measures are suit-
able for comparing the calibration plans for both simple and complex trajectories to be performed. The 
advantages of the developed techniques are illustrated by an example that deals with machining using 
robotic manipulator.  
Key words: robot calibration, experiment design, performance measures, geometric calibration, elasto-
static calibration 
1 Introduction 
Since most of the existing industrial robots employ the close-loop control on 
the level of the joint coordinates only, the robot accuracy highly depends on the 
validity of the mathematical expression used for computation of the end-effector 
position and orientation. These expressions include a number of parameters (geo-
metric and elasto-static) that should be precisely identified using the data obtained 
from calibration experiments. However, existing methods of the optimal pose se-
lection for calibration experiments do not take into account the particularities of 
many industrial applications, where the robot accuracy becomes a key factor. 
In robotics, most of the previous studies concentrate on geometric calibration, 
which consider the effects of joint offsets and link dimensional errors [1]. Only 
very limited number of works address elasto-static calibration [2]. For both cases, 
one of the important issues is the selection of optimal measurement poses, which 
allow to minimize the measurement noise impact on identification accuracy. To 
select the optimal measurement poses, most of the authors used performance 
measures directly related to the covariance matrix, which should be obviously as 
small as possible. In particular, Khalil et al. used the condition number of the ki-
nematic Jacobian [3]. Daney et al. applied the observability indices that are based 
2 
Y. Wu, A. Klimchik, A. Pashkevich, S. Caro and B. Furet 
on singular values of the covariance matrix [4, 5]. Nevertheless, these approaches, 
which are based on rather abstract performance measures, do not ensure the best 
accuracy that can be achieved after applying relevant compensation algorithm in 
the open-loop control of the end-effector location. 
Related problem has been also studied in classical experiment design theory, 
which operates mainly with the linear regression models. Here, for optimal expe-
riment design, different norms of the covariance matrix are usually formalized in 
T-, A-, D-optimality principles [6]. However, they are obviously not related to the 
primary objective of the robot calibration, its accuracy.  
2 Problem Statement  
The calibration procedure may be treated as the best fitting of the experimental da-
ta (given input variables and measured output variables) by using the geometric 
and/or elasto-static models. It is usually solved using the standard least-square 
technique, assuming that the calibration includes measurement of Cartesian coor-
dinates that accommodate errors . These errors are assumed to be i.i.d. (inde-
pendent identically distributed) random values with zero expectation and standard 
deviation . Consequently, the estimates of the identified parameters differ from 
their actual values. So the problem of interest is to find the measurement poses of 
the robotic manipulator, which ensure the lowest impact of the measurement er-
rors (assuming that the number of poses is limited). The set of these measurement 
poses is defined by the corresponding joint coordinates 
i
q , let us denote them as 
 


1,...,
m

Q
q
q
 
(1) 
where m  is the number of experiments. 
For the geometric calibration, each experiment produces two vectors 

,
i
i
p
q
, 
which define the end-effector displacements and the corresponding joint angles. 
The linear relation between them can be written as 
 
 = (
)
i
i


p
J q
Π  
(2) 
where 
(
)
i
J q
 is the Jacobian matrix that depends on manipulator configuration 
i
q  
and vector Π  collects the unknown parameters to be identified. 
For the elasto-static calibration, each experiment produces three vectors 


,
,
i
i
i
p
q
F
, where 
iF  defines the applied force and torque. In accordance 
with [7], the corresponding mapping can be expressed as 
 
 = (
)
(
)
T
i
i
i
i

p
J q
k J
q
F  
(3) 
where 

k
is a matrix that aggregates the unknown compliance parameters 


1,...,
n
k
k
to be identified. 
In the case when both geometric and elasto-static parameters should be cali-
brated simultaneously, the mapping between input and output data can be ex-
pressed as 
Industry-oriented Performance Measures for Design of Robot Calibration Experiment 
3 
 
(
)
(
)
(
)
T
i
i
i
i
i





p
J q
Π
J q
k J
q
F  
(4) 
From the identification theory, the unknown parameters 


,




X
Π k
 can be 
obtained using the least square method, which minimizes the residuals for all ex-
perimental data. Corresponding optimization problem is formulated as 
 


2
,
1
(
,
)
min
m
i
i
i
f








Π k
p
q
X
 
(5) 
where 
(
,
)
i
f

q
X
is defined by the right hand side of the expressions (2), (3) 
or (4). 
In most of the related works, the calibration accuracy is evaluated with respect 
to statistical properties of the unknown parameter estimates obtained from Eq.(5). 
It can be easily proved that, under the above assumptions, the expectation of X  
is equal to the true values (i.e. the estimates are non-biased) and corresponding 
covariance matrix can be expressed as 
 
1
2
1
cov(
)
(
)
(
)
m
T
i
i
i












X
B
q
B q
 
(6) 
where 
(
)
i
B q
is the transformation matrix, which can be derived from expressions 
(2)-(4) (required details are presented in the next section). Using this matrix, a 
number of approaches for design of calibration experiment were proposed (A-, T-, 
D-optimality). All of them are looking at minimization of the covariance matrix 
norm, which is equivalent to achieve the highest accuracy of the parameters. 
However, from practical point of view, it is necessary to achieve the best accuracy 
in the robot end-effector location, assuming the control is based on the obtained 
model. It can be proved that these two objectives are not equivalent. So, some re-
visions of the existing approaches in this area are required. 
The goal of this work is to develop a new performance measure that ensures the 
highest accuracy of the robot end-effector location after error compensation. It is 
based on the concept of manipulator test pose (desired machining configuration), 
and allows essentially improving the robot accuracy at the given configuration via 
proper selection of measurement poses in the calibration experiments. Taking into 
account different application areas of the robot-based machining, the problem of 
defining the performance measures can be treated in accordance with two cases: 
(i) the robot end-effector operates without significant changes in the manipulator 
configuration; (ii) the robot performs machining of long and/or complex trajecto-
ries. These two cases will be address sequentially in the following sections.  
3 Performance Measure Based on a Single Test Pose 
The concept of the manipulator test pose [8] is introduced in this section for the 
case when the machining configuration does not have significant changes (in pick 
and place application and machining of small trajectories, for instance). In this 
4 
Y. Wu, A. Klimchik, A. Pashkevich, S. Caro and B. Furet 
particular case, it is assumed that the quality of the calibration is evaluated for the 
so-called test configuration. This configuration is defined by either 

0
q
or 


0
0
,
q
F
for geometric and elasto-static calibration respectively, which are given 
by the user. For this configuration, the influence of measurement errors in the ca-
libration experiments should be minimized in order to achieve the best end-
effector location accuracy. It is defined by the mean square error of the end-
effector displacements E(
)
T


p
p , which evaluates the accuracy of errors com-
pensation for the given test pose.  
For computational convenience, the models of linear mapping that are pre-
sented in the previous section can be expressed in the following general form  
 





p
B
X
 
(7)   
where B  is the transformation matrix and X  collects the parameters to be iden-
tified. They vary with different cases, which are listed in Table 1. 
Table 1.  Transformation matrix and desired parameters for different cases of calibration 
Parameters to be identified 
Transformation matrix B 
Desired parameters X  
(i) Geometric 
J  
Π  
(ii) Elasto-static 
A  
k  
(iii) Geometric and elasto-static 


J
A  


T
Π
k
 
 
For instance, for the case of elasto-static calibration, expression (3) is rewritten 
using the general form (7) as  
 



p
A k  
 (8) 
where the 
1
n   vector k collects the desired parameters 

1,...,
n
k
k
, which are ar-
ranged in the n
n

compliance matrix 

k  in Eq.(3). Here, matrix A  is defined by 
the columns of Jacobian and the external force and can be expressed as 
 
1
1
,...,
    
T
n
nT




A
J J
F
J J
F
 
(9) 
where 
n
J  is the 
th
n
 column vector of the Jacobian matrix. Similarly, expres-
sion (4) can be rewritten as  
 





p
J
Π
A k .  
(10) 
It can be proved that after application of the relevant compensation algorithm, 
the influence of the errors in the identified parameters X  and the error in the 
end-effector location p  can be evaluated using expression 



p
B
X . Besides, 
corresponding value of the mean square error of the robot end-effector location 
E(
)
T


p
p  can be computed as trace(
)
T


p
p
, which leads to 
 
0
0
E(
)
trace(E(
))
T
T
T





p
p
B
X
X B
 
(11) 
Industry-oriented Performance Measures for Design of Robot Calibration Experiment 
5 
where the subscript ''0'' denotes the test configuration. Since E(
)
T



X
X
is the 
covariance matrix (6) of the identified parameters, the generalized performance 
measure can be expressed as  
 
2
0
1
0
1
trace(
(
)
)
m
p
T
T
i
i
i







B
B B
B
  
(12) 
where 
p

 is the expected mean square errors in the end-effector location, sub-
script  '' i '' indicates the experiment number and m  is the number of experiments. 
In more details, for different case studies described above, these performance 
measures are presented in Table 2. 
 Table 2.  Performance measures for different parameters to be identified (single test pose) 
Parameters to be 
identified 
Performance measures 
(i) Geometric  
2
0
1
0
1
trace(
(
)
)
m
p
T
T
G
i
i
i







J
J J
J
 
(ii) Elasto-static  
2
0
1
0
1
trace(
(
)
)
m
p
T
T
E
i
i
i







A
A A
A
 
(iii) Geometric and 
elasto-static 



2
0
0
1
0
0
1
trace(
,  
(
,  
,  
)
,  
)
m
T
T
p
B
i
i
i
i
i















J
A
J
A
J
A
J
A
 
 
However, it should be stressed that the proposed test-pose based approach is 
limited for the accuracy estimation in a single configuration. The next section 
deals with a more general case where changes in the manipulator configurations 
are significant during machining. 
4 Performance Measure Based on a Set of Test Poses  
In practice, it is often necessary to perform machining along rather long and 
complex trajectories (It is common for aerospace and ship building industries). 
This type of motions may require significant changes in the manipulator configu-
ration and a good accuracy should be achieved for the whole machining trajectory. 
For this reason, it is proposed to extend the application area of the performance 
measure proposed in Section 3 and to minimize the maximum end-effector loca-
tion errors after compensation along the trajectory. In this case, it is reasonable to 
apply the trajectory segmentation technique and adopt the proposed test-pose 
based approach for each node point. In the frame of this paper, we are not consi-
dering the problem of specifying the node points. Here, it is assumed that these 
points are already defined and the corresponding manipulator configurations as 
well as the applied forces are given. These data allow us to define a set of test pos-
6 
Y. Wu, A. Klimchik, A. Pashkevich, S. Caro and B. Furet 
es 
0
0
{
,
1, }
j
j
j
s

q
F
 (for geometric calibration 
0
j 
F
0 ), where s  is the number of 
node points.  
For each test pose that corresponds to the node point of the trajectory, the end-
effector displacement error can be computed as  
 
0
j
j



p
B
X  
(14) 
where 
0
j
B  is the transformation matrix for the configuration that is associated with 
the 
th
j  node-point. Using the technique developed in Section 3, the generalized 
performance measure that is based on the maximum end-effector displacement er-
rors after compensation for all node points can be expressed as  
 
2
0
1
0
1
max {
trace(
(
)
)}
m
t
T
T
j
i
i
j
j
i







B
B B
B
 
(15) 
where 
i
B  defines the transformation matrix for the 
th
i
 calibration experiment. In 
order to increase the identification accuracy, the performance measure 
t
 should 
be minimized. Hence, it is required to solve the min-max problem. In more details, 
for different case studies described above, these performance measures are pre-
sented in Table 3. 
Table 3. Performance measures for different parameters to be identified (multiple test poses) 
Parameters to be 
identified 
Performance measures 
(i) Geometric 
2
0
1
0
1
max {
trace(
(
)
)}
m
t
T
T
G
j
i
i
j
j
i







J
J J
J
 
(ii) Elasto-static 
2
0
1
0
1
max {
trace(
(
)
)}
m
t
T
T
E
j
i
i
j
j
i







A
A A
A
 
(iii) Geometric 
and elasto-static 



2
0
0
1
0
0
1
max{
trace(
,
(
,
,
)
,
)}
m
T
T
t
B
j
j
i
i
i
i
j
j
j
i














J
A
J
A
J
A
J
A
 
 
Hence, The proposed performance measures can be used both for comparing 
different calibration plans and as optimization functions for the experiment design. 
In contrast to the existing ones, the developed performance measures are directly 
related to the end-effector location accuracy after error compensation, which is the 
primary factor for industry. 
5. Illustrative Example 
Let us highlight the advantages of the proposed performance measure by an ex-
ample of a 2-link planar manipulator with rigid links (
1
600mm,
l 
2
 
400mm
l

) 
and compliant joints. It is assumed that the geometric parameters are well cali-
brated, while the elasto-static parameters should be identified. The robot should 
Industry-oriented Performance Measures for Design of Robot Calibration Experiment 
7 
realize a machining operation along a straight line trajectory from the point A(-
600, 400) to the point B(600, 400) with a constant cutting force 


0
0
100N

F
. 
This trajectory includes a set of node points that define the test poses. It is also as-
sumed that the main source of inaccuracy is due to the measurement system used 
in the calibration experiments and the errors are i.i.d. (independent identically dis-
tributed) with the zero expectation and the standard deviation 
0.1mm

. For 
comparison purposes, two cases are considered: one and five calibration experi-
ments (it should be mentioned that usually the elasto-static parameters cannot be 
identified by one experiment, but for the considered manipulator, it gives enough 
data). The optimal configurations have been found both for proposed and existing 
performance measures (D-optimality [6] and SVD based approach (minimum sin-
gular value) [4]). For the considered cases, the deviations of the mean square er-
rors from the target trajectory have been computed using (12) for all node points. 
 Relevant results are summarized in Fig. 1 and Table 4. As it shown, for the 
case of one experiment, the test-pose based approach ensures the accuracy of 
about 43% better than using SVD based approach and about 20% better compar-
ing to the results obtained using the D-optimality. The maximum error along the 
whole trajectory does not overcome 0.14mm. In the case of five experiments, the 
test-pose based approach achieves the best accuracy of the error compensation that 
is 55% and 20% better comparing to the results obtained using SVD based ap-
proach and D-optimality respectively. It should be mentioned that increasing the 
number of experiments from one to five allowed us to improve the accuracy by a 
factor of 2.5 (at the same time, repeating experiments in one configuration im-
proves the accuracy by the factor of 
5
2.2

only). As a result, the maximum er-
ror has been reduced to 0.055mm. Hence, the proposed performance measure that 
is referred to the manipulator end-effector position accuracy provides essential ad-
vantages. So, it is reasonable to use it for calibration of robot used in industrial 
applications.   
-600
-400
-200
0
200
400
600
-0.3
-0.2
-0.1
0
0.1
0.2
0.3
-600
-400
-200
0
200
400
600
-0.3
-0.2
-0.1
0
0.1
0.2
0.3
(mm)
p
E

Test-pose approach
D-optimality
SVD approach
(mm)
x
(mm)
p
E

(mm)
x
Test-pose approach
SVD approach
D-optimality
Target trajectory
Target trajectory
0.055
0.121
0.069
0.137
0.170
0.240
(a) case of one calibration experiment
(b) case of five calibration experiments
 
Figure 1 Accuracy of the compliance errors compensation using different calibration plans 
8 
Y. Wu, A. Klimchik, A. Pashkevich, S. Caro and B. Furet 
Table 4.  Maximum deviation of square root position errors 
p
E

 (mm)   
Performance measure 
Single experiment 
5 experiments 
SVD based approach 
0.240 
0.121 
D-optimality 
0.170 
0.069 
Test-pose based approach 
0.137 
0.055 
 
5 Conclusions  
This paper deals with the evaluation of existing approaches in the area of the 
calibration experiment design for robotic manipulators, taking into account parti-
cularities of some industrial applications. Particular attention is given to the identi-
fication of geometric and elasto-static parameters, whose accuracy directly influ-
ences on the quality of the robot-based machining. For this type of applications, it 
is proposed a new performance measure (directly related to the robot accuracy af-
ter error compensation based on the obtained parameters), which should be used 
for optimal design of the calibration experiments. It is shown that, for the consi-
dered case study, the proposed approach allows to improve the robot accuracy by 
about 20%.  
Acknowledgments   The authors would like to acknowledge the financial support 
of the Project ANR-2010-SEGI-003-02-COROUSSO. 
References 
1. Hollerbach, J. M., Khalil, W., Gautier, M.: Model Identification, In: Springer handbook of ro-
botics, Part B, pp. 321-344 (2008). 
2. Meggiolaro, M. A., Dubowsky, S., Mavroidis,C.: Geometric and Elastic Error Calibration of a 
High Accuracy Patient Positioning System. In: Mechanism and Machine Theory, 40, pp. 415-
427 (2005). 
3. Khalil, W. and Besnard, S.: Geometric Calibration of Robots with Flexible Joints and Links. 
In: Journal of Intelligent and Robotic Systems, 34, pp. 357-379 (2002). 
4. Daney, D.: Optimal Measurement Configurations for Gough Platform Calibration. In: Robot-
ics and Automation, ICRA/ IEEE International Conference, 1, pp. 147-152 (2002). 
5. Daney, D., Papegay, Y., Madeline, B.: Choosing Measurement Poses for Robot Calibration 
with the Local Convergence Method and Tabu Search. In: The International Journal of Ro-
botics Research, 24(6), pp. 501-518 (2005). 
6. Atkinson, A.C. and Donev, A.N.: Optimal Experiment Designs, Oxford University Press, Ox-
ford, (1992). 
7. Pashkevich, A., Klimchik, A., Chablat, D.: Enhanced Stiffness Modeling of Manipulators with 
Passive Joints. In: Mechanism and Machine Theory, 46, pp. 662-679 (2011). 
8. Imoto, J., Takeda, Y., Saito, H., Ichiryu, K.: Optimal Kinematic Calibration of Robots Based 
on Maximum Positioning-Error Estimation (Theory and Application to a Parallel-Mechanism 
Pipe Bender). In: Computational Kinematics, Springer, pp.133-140 (2009).