Evaluation Metrics

Object person tracking evaluation
This evaluation is performed using ViPER-PE. Parameter file #BEGIN_OBJECT_EVALUATION OBJECT Person [- -] Position_BBOX : [dice -] #END_OBJECT_EVALUATION #BEGIN_FRAMEWISE_EVALUATION OBJECT Person Position_BBOX : dice overlap matchedpixels missedpixels \ falsepixels [arearecall 0.6] [areaprecision 0.7] #END_FRAMEWISE_EVALUATION Properties file # # Level of analysis # 1=detection (vedi capitolo: 5.1.1) # 2=localization (vedi capitolo: 5.1.2) # 3=statistical comparison (vedi capitolo: 5.1.3) # level = 3 # # How to crop the set of possibles at the end. #SINGLE-BEST, SINGLE-GREEDY, MULTIPLE, NONE (vedi capitolo: 5.1.4) # target_match = MULTIPLE # # Range Distance Metric (vedi figura: 29) # dice = Dice coefficient # overlap # extent # E = equality # range_metric = dice # # String Distance Metric # L = Levenshtein (Edit distance) # H = Hamming # E = equality # string_metric = L # # Level Specific Metrics (vedi capitolo: 5.1.3) # mean, median, minimum, maximum # level3_metric = mean # ############################################################### # Default Tolerance Configuration [0 = exact match] ############################################################### # # Temporal Range (vedi capitolo: 5.1.1) # range_tol = 0.2 # # Attributes, <attribute type>_tol (vedi capitolo: 5.1.2) # bvalue_tol = 1.0 dvalue_tol = 1.0 fvalue_tol = 0.9 svalue_tol = 1.0 point_tol = 1.0 circle_tol = 1.0 bbox_tol = 0.25 obox_tol = 0.99 lvalue_tol = 1.0 ellipse_tol = 0.99 # # Level Specific (vedi capitolo: 5.1.3) # level3_tol = 0.3 # ############################################################### # # Presentation Parameters # # verbose: stampa i coefficienti delle distanze # attrib_width: numero massimo di caratteri stampato per gli # attributi dinamici # verbose = true attrib_width = 50 # See the ViPER-PE web site for the documentation about the file format of the performance evaluation parameters and results. (http://viper-toolkit.sourceforge.net/ )
Concept detection (Precision and Recall)
The two annotations are compared without keeping temporal information of the concepts. Only the detection inside a video is important. No matches among instances of the same concept are performed. In other words, for each concept the total number of instances in each annotation is used. where: C_i: i-th concept AA_instances_i: number of instances of the i-th concept in the automatic annotation GT_instances_i: number of instances of the i-th concept in the ground-truth reference annotation

Object person tracking evaluation

This evaluation is performed using ViPER-PE.

Parameter file

Properties file

#
# Level of analysis
# 1=detection (vedi capitolo: 5.1.1)
# 2=localization (vedi capitolo: 5.1.2)
# 3=statistical comparison (vedi capitolo: 5.1.3)
#
level = 3
#
# How to crop the set of possibles at the end.
#SINGLE-BEST, SINGLE-GREEDY, MULTIPLE, NONE (vedi capitolo: 5.1.4)
#
target_match = MULTIPLE
#
# Range Distance Metric (vedi figura: 29)
# dice = Dice coefficient
# overlap
# extent
# E = equality
#
range_metric = dice
#
# String Distance Metric
# L = Levenshtein (Edit distance)
# H = Hamming
# E = equality
#
string_metric = L
#
# Level Specific Metrics (vedi capitolo: 5.1.3)
# mean, median, minimum, maximum
#
level3_metric = mean
#
###############################################################
# Default Tolerance Configuration [0 = exact match]
###############################################################
#
# Temporal Range (vedi capitolo: 5.1.1)
#
range_tol = 0.2
#
# Attributes, <attribute type>_tol (vedi capitolo: 5.1.2)
#
bvalue_tol = 1.0
dvalue_tol = 1.0
fvalue_tol = 0.9
svalue_tol = 1.0
point_tol = 1.0
circle_tol = 1.0
bbox_tol = 0.25
obox_tol = 0.99
lvalue_tol = 1.0
ellipse_tol = 0.99
#
# Level Specific (vedi capitolo: 5.1.3)
#
level3_tol = 0.3
#
###############################################################
#
# Presentation Parameters
#
# verbose: stampa i coefficienti delle distanze
# attrib_width: numero massimo di caratteri stampato per gli
# attributi dinamici
#
verbose = true
attrib_width = 50
#

See the ViPER-PE web site for the documentation about the file format of the performance evaluation parameters and results. (http://viper-toolkit.sourceforge.net/ )

Concept detection (Precision and Recall)

The two annotations are compared without keeping temporal information of the concepts. Only the detection inside a video is important. No matches among instances of the same concept are performed.
In other words, for each concept the total number of instances in each annotation is used.

where:

C_i: i-th concept
AA_instances_i: number of instances of the i-th concept in the automatic annotation
GT_instances_i: number of instances of the i-th concept in the ground-truth reference annotation