THIS - Transport Hub Intelligent Video System
Automatic behavior analysis through video processing for transportation hubs, focused on crowded scenarios and abnormal event detection (2009 -2011)
With the support of the Prevention, Preparedness and Consequence Management of Terrorism
and other Security-related Risks Programme European Commission
Directorate-General Justice, Freedom and Security
Reference code: JLS/2009/CIPS/AG/C1-028


The following tables contain the ontology defined for the project. The ontology is a subset of the VISOR surveillance ontology .

THIS - Ontology Export: ViPER format - Mpeg-7 - OWL .



Descriptor List

Descriptor NameDescriptionCategoryViper Type
1. PersonKind of person, appearance, age, sex PhysicalObject OBJECT
2. Bodypartlegs, arms, and so on PhysicalObject OBJECT
3. Group_of_peoplemore than one person PhysicalObject OBJECT
4. Fixed_objectbuildings, furniture, trees, and so on PhysicalObject OBJECT
5. Mobile_objectmoving or mobile object, like chair, pack, luggage PhysicalObject OBJECT
6. Action_by_a_personAction by a single person Action/Event OBJECT
7. Action_by_group_of_peopleaction by a group of people, like meeting Action/Event OBJECT
8. Object_eventevents related to objects, like AbandonedObject Action/Event OBJECT
9. Generic_eventgeneric events, like fire Action/Event OBJECT
10. Locationeverithing describing the video location Context CONTENT
11. Animalscats, dogs, and so on PhysicalObject OBJECT
12. Weathersun, cloud, rain… Context CONTENT
13. VideoInfoglobal video information Context CONTENT


Concepts and Attributes List

1. Person - Kind of person, appearance, age, sex

Static concepts (Is-a)

Concept Definition ID ViSOR ID LSCOMM ID MediaMill ID UniFi Note Dyn
1.Adult Shots showing a person over the age of 18. 182 181 False
2.Armed_person Any person carrying a weapon. 117 116 False
3.Baby images of babies (children that are tooyoung to walk) 248 247 False
4.Boy One or more male children. 184 183 False
5.Civilian_person One or more persons not in the armed services or police force. 106 105 False
6.Dangerous Dangerous Object 1013 THIS False
7.Female_person One of more female persons. 104 103 False
8.Firefighter A person whose job it is to extinguish fires. 118 117 False
9.FrequentVisitor Person visiting the same place very frequently 1012 THIS False
10.Girl One or more female children. 185 184 False
11.Individual Shots showing only one person. 181 180 False
12.Intruder Person entering a zone without permission 1015 THIS False
13.Male_person One or more male persons. 105 104 False
14.Old_people Seniors or elderlypeople 360 359 False
15.Single_person one person only isvisible 411 410 False
16.Terrorist 831 830 False
17.Vandal 846 845 False

Dynamic concepts (Has-a)

Concept Description DataType ViperDataType Extended Dyn
18.Position_BBOX bbox bbox True True
19.PositionBar 2D Position of the gravity center point point True True
20.Contour Contour of the object polygon polygon True True
21.IDPerson Application defined ID reference dvalue False False
22.RealHeight Real height of the person fvalue fvalue False False
23.PersonName* Name of the person svalue svalue False False
24.FeetPosition x,y coordinates of the feet point point True True
25.HeadPosition x,y coordinates of the head point point True True
26.ObjectMaskFileName_CVC Name of the file containing the mask in cvc format svalue svalue False True
27.Position_Ellipse ellipse obox False True
28.Position_wrtFrame Position of the target with respect to the current frame
Values: Inside Touching_Borders Touching_Borders_Minor Touching_Major 		lvalue lvalue False True
29.VisibilityDegree Occlusion rate
Values: 100_NotOccluded 66_SlightlyOccluded 33_HeavilyOccluded 		lvalue lvalue False True
2. Bodypart - legs, arms, and so on

Static concepts (Is-a)

Concept Definition ID ViSOR ID LSCOMM ID MediaMill ID UniFi Note Dyn
30.Body_parts parts of a human body 340 339 False
31.Face shots depicting a face 215 214 2 False
32.Head_and_shoulder A view of a person showingonly the torso and head. Both the torso and head must bevisible. 151 150 False
33.Leg 981 UoM False
34.Trunk 980 UoM False

Dynamic concepts (Has-a)

Concept Description DataType ViperDataType Extended Dyn
35.Position_BBOX bbox bbox True True
36.PositionBar 2D Position of the gravity center point point True True
37.Contour Contour of the object polygon polygon True True
38.IDPerson Application defined ID reference dvalue False False
39.Owner_ID reference dvalue False False
40.Position_Ellipse ellipse obox False True
41.FaceTilt Tilt Angle of the face (0:frontal - 90: lateral -180:rear)
Values: 0_front 30 60 90_side 180_rear 		lvalue lvalue False True
42.Side Side (0: Left, 1: Right) dvalue dvalue False False
3. Group_of_people - more than one person

Static concepts (Is-a)

Concept Definition ID ViSOR ID LSCOMM ID MediaMill ID UniFi Note Dyn
43.Criminals 551 550 False
44.Crowd Shots depicting a crowd 203 202 10 False
45.Demonstrators 554 553 False
46.Group We defined a group as 3-10 people. It onlyincluded shots of 3-10 people, not animals, such as pets, nor animationedpeople, such as in previews of "The Incredibles." 317 316 False
47.Intruder Person entering a zone without permission 1015 THIS False
48.Protesters People engaged in some form of protest 383 382 False

Dynamic concepts (Has-a)

Concept Description DataType ViperDataType Extended Dyn
49.Position_BBOX bbox bbox True True
50.PositionBar 2D Position of the gravity center point point True True
51.Contour Contour of the object polygon polygon True True
52.IDGroup reference dvalue False False
53.NumberOfPeople dvalue dvalue False True
4. Fixed_object - buildings, furniture, trees, and so on

Static concepts (Is-a)

Concept Definition ID ViSOR ID LSCOMM ID MediaMill ID UniFi Note Dyn
54.Abandoned Abandoned object 1016 THIS False
55.bench 990 UoM False
56.Building Shots of an exterior of a building 227 226 16 False
57.Dangerous Dangerous Object 1013 THIS False
58.Door Door / entrance of a building or a room 1003 UoM False
59.Furniture TV is not furniture. Tabletop isfurniture (if it is a main part of the scene), Patio umbrella is furniture,Voting booth tables are furniture, Frames where people were sitting on a sofawere counted as containing furniture as long as the sofa was easily vis 313 312 False
60.Office Office environment with desks, chairsand/or white-collar workers. 86 85 33 False
61.Waste_bin 991 UoM False
62.Windows An opening in the wall or roof of a building or vehicle fitted with glass or other transparent material. 110 109 False

Dynamic concepts (Has-a)

Concept Description DataType ViperDataType Extended Dyn
63.Position_BBOX bbox bbox True True
64.PositionBar 2D Position of the gravity center point point True True
65.Contour Contour of the object polygon polygon True True
66.IDObject ID of who/what undergo the action/event reference dvalue False False
5. Mobile_object - moving or mobile object, like chair, pack, luggage

Static concepts (Is-a)

Concept Definition ID ViSOR ID LSCOMM ID MediaMill ID UniFi Note Dyn
67.Abandoned Abandoned object 1016 THIS False
68.Backpack Backpacks must visually apparent i.e.in the foreground. At least the strap must be visible. we did not include otherpacks, like handbags, fanny packs, purses etc. Only backpacks that are beingworn were counted. We included utilities backpack like leaf blo 250 249 False
69.Bicycle A person riding a bicycle. 198 197 74 False
70.Boat 503 502 49 False
71.Bomb 506 505 False
72.Bus Shots of a bus 228 227 59 False
73.Car Shots of a car 222 221 19 False
74.ChangeSpeed_down Moving Person/Object decreasing speed 1020 THIS False
75.ChangeSpeed_up Moving Person/Object increasing speed 1019 THIS False
76.Dangerous Dangerous Object 1013 THIS False
77.Door Door / entrance of a building or a room 1003 UoM False
78.Knife 646 645 False
79.Shotgun 783 782 False
80.Smoke Shots with smoke present. 162 161 38 False
81.Vehicle Any thing used for transporting people or goods, such as a car, bus, truck, cart, plane,etc. 109 108 15 False
82.Waste_bin 991 UoM False

Dynamic concepts (Has-a)

Concept Description DataType ViperDataType Extended Dyn
83.Position_BBOX bbox bbox True True
84.PositionBar 2D Position of the gravity center point point True True
85.Contour Contour of the object polygon polygon True True
86.ObjectMaskFileName_CVC Name of the file containing the mask in cvc format svalue svalue False True
87.IDObject ID of who/what undergo the action/event reference dvalue False False
6. Action_by_a_person - Action by a single person

Static concepts (Is-a)

Concept Definition ID ViSOR ID LSCOMM ID MediaMill ID UniFi Note Dyn
88.ChangeSpeed_down Moving Person/Object decreasing speed 1020 THIS False
89.ChangeSpeed_up Moving Person/Object increasing speed 1019 THIS False
90.Intrusion Action of a person entering a zone without permission 1014 THIS False
91.Jumping A person jumps 997 UoM False
92.Person_enters_a_bilding Person enters a buiding through a door or a gate 1004 UoM False
93.Person_enters_area 967 UniFi A.1 False
94.Person_enters_object 966 UniFi A False
95.Person_exits_a_building Person leaving a buiding through a door or a gate 1005 UoM False
96.Person_exits_area 969 UniFi A.3 False
97.Person_exits_object 968 UniFi A.2 False
98.Person_falls_down 983 UoM False
99.Person_firing_weapon 979 UniFi E False
100.Person_interacts_object 970 UniFi B False
101.Person_leaves_object 972 UniFi B.2 False
102.Person_takes_object 971 UniFi B.1 False
103.Running One or more people running. 4 3 False
104.Sitting One or more people sitting down. 107 106 False
105.Standing One or more people standing up. 108 107 False
106.Start_To_Run Person starting to run 1017 THIS False
107.Stop_To_Run Person stopping to run 1018 THIS False
108.Walking One or more people walking. 13 12 False

Dynamic concepts (Has-a)

Concept Description DataType ViperDataType Extended Dyn
109.IDPerson Application defined ID reference dvalue False False
110.Description svalue svalue False False
111.IDObject ID of who/what undergo the action/event reference dvalue False False
112.ActionDescription svalue svalue False False
7. Action_by_group_of_people - action by a group of people, like meeting

Static concepts (Is-a)

Concept Definition ID ViSOR ID LSCOMM ID MediaMill ID UniFi Note Dyn
113.ChangeSpeed_down Moving Person/Object decreasing speed 1020 THIS False
114.ChangeSpeed_up Moving Person/Object increasing speed 1019 THIS False
115.Crowd_dispersal 974 UniFi C.2 False
116.Crowd_formation 973 UniFi C.1 False
117.Fight People fighting 1009 THIS False
118.Handshaking Two people shaking hands. Does not include hugging or holding hands. 3 2 False
119.Hug Hugs between 2 or more people 1010 THIS False
120.Intrusion Action of a person entering a zone without permission 1014 THIS False
121.Meeting Scenes of meeetings. We did not considerspeeches/talks to be a meeting. We did not consider anchor meeting with expertsto be a meeting. 221 220 20 False
122.People_aggregation 984 UoM False
123.Quarrel 1011 THIS False
124.Start_To_Run Person starting to run 1017 THIS False
125.Stop_To_Run Person stopping to run 1018 THIS False

Dynamic concepts (Has-a)

Concept Description DataType ViperDataType Extended Dyn
126.Description svalue svalue False False
127.ActionDescription svalue svalue False False
128.IDGroup reference dvalue False False
8. Object_event - events related to objects, like AbandonedObject

Static concepts (Is-a)

Concept Definition ID ViSOR ID LSCOMM ID MediaMill ID UniFi Note Dyn
129.Car_Accident 978 UniFi D.4 False
130.Car_runs_red_light 977 UniFi D.3 False
131.Car_starts 976 UniFi D.2 False
132.Car_stops 975 UniFi D.1 False
133.Exiting_car A car exiting from somewhere, such as a highway, building, or parking lot. 2 1 False
134.Smoke Shots with smoke present. 162 161 38 False

Dynamic concepts (Has-a)

Concept Description DataType ViperDataType Extended Dyn
135.IDObject ID of who/what undergo the action/event reference dvalue False False
9. Generic_event - generic events, like fire

Static concepts (Is-a)

Concept Definition ID ViSOR ID LSCOMM ID MediaMill ID UniFi Note Dyn
136.Accident 456 455 False
137.Bomber_bombing An airborne bomber dropping bombson some target 39 38 False
138.Explosion_fire Shots of an explosion or a fire 204 203 False
139.Violence 954 13 False

Dynamic concepts (Has-a)

Concept Description DataType ViperDataType Extended Dyn
140.Description svalue svalue False False
141.EventDescription svalue svalue False False
10. Location - everithing describing the video location

Static concepts (Is-a)

Concept Definition ID ViSOR ID LSCOMM ID MediaMill ID UniFi Note Dyn
142.Airport Exterior shots of an airport, showingone or more buildings (such as the air traffic control tower or the terminals). 41 40 False
143.Airport_terminal Interior shots of airport terminals, including ticket counters, waiting areas, and security checkpoints. 95 94 False
144.Alley Small, narrow passage way between two buildings. 155 154 False
145.Building Shots of an exterior of a building 227 226 16 False
146.Bus_terminal 513 512 False
147.Hall Hall of a building 1002 UoM False
148.High_security_facility Any facility that is highly guarded, requiring security checkpoints or clearance forentry. 123 122 False
149.Indoor 911 6 False
150.Office Office environment with desks, chairsand/or white-collar workers. 86 85 33 False
151.Outdoor Shots of Outdoor locations 225 224 4 False
152.Parking_Lot Outdoor area for parkingcars. 90 89 False
153.Pedestrian_zone shots of pedestrian zones andwalkways for people only 309 308 False
154.Riot Many people engaging in violence or mayhem in city streets 21 20 False
155.Road Shots depicting a road 207 206 14 False
156.Security_checkpoint Any security checkpoint.Includes military checkpoints and airport security. 148 147 False
157.Sky Shots depicting sky 208 207 11 False
158.Streets regular paved streets (not highways,dirt roads, or special types of road) 419 418 False
159.Subway_station Interior views of a subway station. 103 102 False
160.Train_station 841 840 False
161.Tunnel Views of the inside of a tunnel. May be a tunnel for cars, trains, sewage, or anything else. 82 81 False
162.University University campus 1001 UoM False
163.Urban_park A public park in a city, such as Central Park. 102 101 False

Dynamic concepts (Has-a)

Concept Description DataType ViperDataType Extended Dyn
11. Animals - cats, dogs, and so on

Static concepts (Is-a)

Concept Definition ID ViSOR ID LSCOMM ID MediaMill ID UniFi Note Dyn
164.Dogs shots of dogs, does not include wolves or foxes 293 292 82 False

Dynamic concepts (Has-a)

Concept Description DataType ViperDataType Extended Dyn
165.Position_BBOX bbox bbox True True
166.PositionBar 2D Position of the gravity center point point True True
167.Contour Contour of the object polygon polygon True True
168.Description svalue svalue False False
169.Position_Ellipse ellipse obox False True
12. Weather - sun, cloud, rain…

Static concepts (Is-a)

Concept Definition ID ViSOR ID LSCOMM ID MediaMill ID UniFi Note Dyn
170.Clouds Scenes where clouds are visible 277 276 45 False
171.Rainy Rainy Scenes includes slick roads, andrain on windows 388 387 False
172.Snow Snow falling or already accumulated on the ground 25 24 False
173.Sunny Sunny Scenes may also be inside. In general, if it would be uncomfortable to look towards the sun, it is a sunnyscene. 424 423 False
174.Windy Scenes showing windyweather 448 447 False

Dynamic concepts (Has-a)

Concept Description DataType ViperDataType Extended Dyn
13. VideoInfo - global video information

Static concepts (Is-a)

Concept Definition ID ViSOR ID LSCOMM ID MediaMill ID UniFi Note Dyn

Dynamic concepts (Has-a)

Concept Description DataType ViperDataType Extended Dyn
175.Description svalue svalue False False
176.CalibrationData svalue svalue False False

* - These concepts can be seen by logged in users only.

To add, modify or delete a concept contact Roberto Vezzani

.
For problems or suggestions: contact Roberto Vezzani - Webmaster: Roberto Vezzani - ©2007