FREE ELECTRONIC LIBRARY - Dissertations, online materials

Pages:   || 2 |

«Semantic Units Based Event Detection in Soccer Videos1) TONG Xiao-Feng LIU Qing-Shan LU Han-Qing JIN Hong-Liang (National Laboratory of Pattern ...»

-- [ Page 1 ] --

Vol. 31, No. 4 ACTA AUTOMATICA SINICA July, 2005

Semantic Units Based Event Detection in Soccer Videos1)

TONG Xiao-Feng LIU Qing-Shan LU Han-Qing JIN Hong-Liang

(National Laboratory of Pattern Recognition, Institute of Automation,

Chinese Academy of Sciences, Beijing 100080)

(E-mail: {xftong, qsliu, luhq, hljin}@nlpr.ia.ac.cn)


A semantic units based event detection scheme in soccer videos is proposed in this paper.

The scheme can be characterized as a three-layer framework. At the lowest layer, low-level features including color, texture, edge, shape, and motion are extracted. High-level semantic events are defined at the highest layer. In order to connect low-level features and high-level semantics, we design and define some semantic units at the intermediate layer. A semantic unit is composed of a sequence of consecutives frames with the same cue that is deduced from low-level features. Based on semantic units, a Bayesian network is used to reason the probabilities of events. The experiments for shoot and card event detection in soccer videos show that the proposed method has an encouraging performance.

Event detection, semantic unit, video semantic analysis, Bayesian network Key words 1 Introduction With the increasing of multimedia data, it is crucial to find an efficient way to manage the media data, including browse, filtering and retrieval[1∼3]. Low-level features are too oversimple to use for semantic requirement. Recently, event based multimedia indexing and retrieval is widely concerned[4∼6], and it is much more significant and valuable than shot based video analysis. Generally speaking, an event can be regarded as an interesting activity in a video segment, and it should have the three basic characteristics: 1) domain-dependent; 2) spatial-temporal context related; 3) difficult to be simply characterized and identified by low-level features. This paper focuses on semantic event detection and analysis in soccer programs. For a lengthy soccer game, highlights often take up a small part, so it is very significant to detect and analysis events in soccer video.

At present, some studies have been done on the event detection and analysis in sports video.

Naphade et al.[7] presented concepts of multi-objects and multi-nets, and set up a multi-nets framework based on graph probabilistic reasoning for semantic video indexing. A general “event + non-event” framework for indexing and summarizing sports broadcast programs was presented in [8]. Vasconcelous et al.[9] put forward a Bayesian framework to extract video semantic features to depict content of movies, but their method did not consider temporal context. In [10], a scene detection and structure analysis method for sports video was developed, which combined domain-specific knowledge, supervised machine learning and hierarchical features analysis technology. P. Xu et al.[11] developed a method that divided a sports video into play and break segments. Based on this work, L. Xie et al.[12] employed HMM and dynamic programming to enhance the performance of segment detection and classification with taking field-ratio and motion activity as observations. [13] analyzed video editing ways and object based features. They proposed an automatic soccer program analysis and summarization method. In their experiments, they detected slow-motion replay, close-up, break, and utilized a heuristic rule to identify highlights. X. Sun et al.[14] used Bayesian network to detect score events based on goalnet, audience, scoreboard and face cues.

In this paper, we propose a semantic unit based event detection scheme according to the characteristics of events in sports video. The scheme can be characterized as a three-layer framework shown in Fig. 1. At the lowest layer, low-level features, such as color, texture, edge, shape and motion, are extracted from visual frames. Events describing interesting activities in video segments are defined at the highest layer. In order to bridge low-level features and high-level semantic events, we define semantic units at the intermediate layer. A semantic unit is composed of continuous frames that contain the same cue. Semantic units are derived from low-level features and taken as observation of event inference. Generally, an event consists of several semantic units. Presence of some specific semantic units

1) Supported by National Natural Science Foundation of P. R. China (60475010, 60121302) Received January 14, 2004; in revised form September 17, 2004 524 ACTA AUTOMATICA SINICA Vol. 31 indicates a specific event. In our experiments, we employ this scheme to detect shoot and card events in soccer videos. Considering the domain knowledge, we define six types of semantic units: replay, goalmouth, caption, close-up, audience, and close+caption unit. Taking these units as observations, a Bayesian network is to reason the probabilities of defined events.

–  –  –

The rest of the paper is organized as below. Section 2 introduces low-level features. Section 3 discusses detection of semantic units. Section 4 describes event inference. Experiments are given in Section 5. Conclusions are drawn in Section 6.

2 Low-level features Low-level features include field dominant color, skin color, frame-to-frame difference, edge, texture, shape of region, and scale of objects in the field.

1) Field dominant color: Game field extraction is an important procedure in event detection. To reduce the effect of illumination, we select HSV color space, and only use hue and saturation components.

Assuming Hmean and Smean the values of hue and saturation components of filed dominant color, they can be obtained through statistics at the start period of the game[13]. The distance from a pixel f (i, j) to the dominant color values is defined as below.

S 2 (i, j) + Smean − 2S(i, j)Smean cos(θ) 2 dhsv = where θ = |H(i, j) − Hmean |, H(i, j) and S(i, j) are hue and saturation components of the pixel f (i, j).

If the distance is smaller than a threshold, this pixel belongs to the field.

2) Skin detection: An effective unimodal Gaussian model with multi-variable is utilized to detect skin region[15]. Then, morphological filtering is applied to remove small and crash areas. The shape and scale of the skin area are used to identify close-up views.

3) Frame-to-frame difference: The mean square difference (MSD) of intensity is used to measure the difference of adjacent frames. MSD is utilized to detect logo-transitions in replay segments.

4) Edge: Edge is also a very useful feature. We apply a Sobel operator with the size of 3 × 3 to extract edges of an image. Edge information is used to discriminate goalmouth and caption area.

5) Texture: Texture describes the repeated mode of local changes of image intensity, and it often takes the gray spatial distribution of neighbors of pixels as features. It is utilized to distinguish audience from out-field close-up views.

6) Shape: Shape is used for verifying head area after shin detection. Shape feature includes: 1) scale, i.e., the height of region; 2) compactness, i.e., ratio of actual area to the area of the min-boundingbox; 3) elongation, i.e., ratio of height to width of the min-bounding-box.

7) Scale of objects in field: It is defined as the ratio of average height of objects to that of game field in the frame. It directly reflects the distance from camera to the captured objects. Before scale estimation, object in field segmentation is necessary. For detailed algorithm, please refer to our previous work[15].

–  –  –

of semantic units usually needs to consider domain-dependant knowledge and video editing rules. We concern shoot and card events in soccer videos in this paper. Correspondingly, we define six types of units: replay, goalmouth, caption, close-up, audience, and close+caption units. An interesting shoot event usually contains replay, goalmouth and player close-up units. Furthermore, a scene of excited audience and scoreboard will appear if score. A serial of typical views in a score event are shown in Fig. 2. In a server foul event, such as red/yellow card event, a red/yellow card recorder will be superimposed onto the player close-up views in addition to replay segment.

Fig. 2 Typical views of a goal event (a) Goalmouth, (b) Close-up, (c) Replay, (d) Close-up, (e) Audience, (f) Scoreboard The operation of semantic units is carried out on frames. If the counter of continuous frames containing the same cue exceeds a threshold, a semantic unit is declared. Semantic unit detection is

kept in the following order:

Step 1. Replay segments detection.

Further processing in the rest segments apart from replays in the following order:

Step 2. Caption detection.

Step 3. Views classification, obtain close-up and audience view.

Step 4. Based on step 2 and 3, identify close+caption views.

Step 5. Detect goalmouth in long views.

1) Replay. Replay is a video editing way, and it is often used to emphasize an important segment with a slow-motion pattern for once and several times. At present, there are two methods for replay detection, i.e., adjacent frame difference based method[16] and compressed prediction vectors based method[17]. They are valid for some replay segments generated by special means. In this paper, we apply a simple and effective detection method based on replay-logo.

In sports video, there is often a highlighted logo that wipes at the start and end of a replay segment and the logo is invariant in the whole video. Therefore, we can firstly obtain the logo from these wipe transitions and then employ it to detect replay segments. The replay detection algorithm[15,18] consists of the following steps: 1) Detect no less than n logo-transitions and extract an optimal candidate of logo in each of them. 2) Take these candidates as a cluster and get its center. Compute the mean image of those candidates near to the center to eliminate the effect of background. The mean image is then regarded as the logo template. 3) Extract other logos through the logo template matching in the video. A pair of logos determines a replay segment. A logo-transition and extracted logo are displayed in Fig. 3.

Fig. 3 Five images in a transition (a∼e) and a logo-template image (f)

2) Caption. In soccer videos, caption is appeared at these cases: recorder score, red/yellow card, player substitution and technical statistics. It is difficult to recognize the text in a caption, such as player names, score, but the appearance of caption usually indicates an occurrence of special event.

The caption region can be treated as a special texture area aligned by vertical strokes, in which the gradients of local neighbors are greater and more uniform than those of other regions. The procedure of caption area detection[19] consists of gradient computation, run-length smoothing, morphological open 526 ACTA AUTOMATICA SINICA Vol. 31 operation, region segmentation and verification. Because captions are often appeared at the bottom of an image, we just need to do such detection at the bottom of frames.

3) Close-up and audience. The focal players are attracted with close-up view in a highlight segment, such as shoot and card events. In red/yellow card events, close-up views usually are superimposed upon the caption of card recorder. In shoot events, views of excited audience will also be shown. We utilize a decision tree to classify views into long, medium, close-up or audience type based on field-ratio, texture, qualified head area and object scale in game field[15].

4) Close+Caption. When caption appears in a close-up frame, we treat it as a close+caption view independently. Close+caption views usually appear in the case of server foul or players substitution.

5) Goalmouth. Goalmouth is also a valid cue for highlights. Fig. 4 (a) shows a long side view in a shoot segment. A goalmouth is composed of a goal line, goal posts and a crossbar. We restrict the region of edge detection to reduce noise. The detection procedure includes: 1) Compute the coarse spatial representation CSR (i, j) of the original image, shown in Fig. 4 (b)[20]. 2) Extract edges in the region between field and non-field in CSR, shown in Fig. 4(c). 3) Search the longest line in the edge image, L(ρ, θ). In common, the slant angle of the goal line in the image captured by the main camera (placed at near the middle of game field) is relatively fixed. So, we can define an interval to restrict the angle of the goal line and filter false alarms. 4) Goal posts and crossbar detection based on gray growing after the goal line extraction. The final result is shown in Fig. 4 (d) Fig. 4 Goalmouth detection. (a) Original image, (b) CSR, (c) Edge in CSR, (d) Result (overlay with red line)

6) Video decomposition based on semantic units. According the above definition and discussion, we can partition a video into a sequence of semantic units. Combination of special semantic units indicates the presence of a special event. Fig. 5 gives the comparison of semantic units and shots based video decomposition in a video. The upper seven rows are timelines of semantic units, and every horizontal red line segment denotes a semantic unit. The bottom rows describe the video decomposition based on shots.

Fig. 5 Semantic units representation of a video clip. L – long view unit, M – medium view unit, U – close-up view unit, S – SMR unit, G – goalmouth unit, C – caption unit, A – audience unit; St - shot

–  –  –

using prior probabilities in conditional probabilities dataset and known nodes. The correlation between observations and conclusions can be measured by mutual information.

In this paper, we construct a Bayesian network shown in Fig. 6 to detect shoot and card events in soccer videos. For shoot event, replay, audience, goalmouth, caption and close-up units are taken as observations. For red/yellow card event, close+caption unit replaces caption and close-up unit.

Fig. 6 Structure of the Bayesian network

5 Experiments We apply the proposed scheme to detect shoot and red/yellow card events in real soccer videos.

Pages:   || 2 |

Similar works:

«NAVAL POSTGRADUATE SCHOOL MONTEREY, CALIFORNIA THESIS PREDICTING HAIL SIZE USING MODEL VERTICAL VELOCITIES by Gregory J. Barnhart March 2008 Thesis Advisor: Wendell Nuss Second Reader: Patrick Harr Approved for public release; distribution is unlimited THIS PAGE INTENTIONALLY LEFT BLANK REPORT DOCUMENTATION PAGE Form Approved OMB No. 0704-0188 Public reporting burden for this collection of information is estimated to average 1 hour per response, including the time for reviewing instruction,...»

«Insects 2015, 6, 325-332; doi:10.3390/insects6020325 OPEN ACCESS insects ISSN 2075-4450 www.mdpi.com/journal/insects/ Review Ecology of Fungus Gnats (Bradysia spp.) in Greenhouse Production Systems Associated with Disease-Interactions and Alternative Management Strategies Raymond A. Cloyd Department of Entomology, Kansas State University, Manhattan, KS 66506, USA; E-Mail: rcloyd@ksu.edu; Tel.: +1-785-532-4750; Fax: +1-785-532-6232 Academic Editor: Andrew G. S. Cuthbertson Received: 12 February...»

«Annual Report of the Town of Brookfield VERMONT January 1, 2014 to December 31, 2014 EMERGENCY TELEPHONE NUMBERS Fire...911 White River Valley Ambulance..911 (802-234-6800) Williamstown Rescue Unit..911 (476-4111) Sheriff-Orange County..685-4875 State Police..911 (802-234-9933) Fire Warden..728-5739 Town Garage...276-3090 Town Clerk..276-3352 Sheriff – Orange County..685-4875 Town Clerk’s Office Hours: Tuesday, Wednesday and Thursday: 8:30 A.M. – 12:00 P.M., 1:00 P.M. – 4:30 P.M....»

«Thalidomide Pharmion™ Pregnancy Prevention Programme Information for Patients Taking Thalidomide Pharmion ™ This booklet contains information about: • Preventing harm to unborn babies: If Thalidomide Pharmion™ is taken during pregnancy it can cause severe birth defects or death to an unborn baby • Other side effects of Thalidomide Pharmion™: These include nerve damage, blood clots, severe skin problems, dizziness, drowsiness, constipation and low white blood cell counts •...»

«Education and Care Services National Regulations 2011 EXPLANATORY STATEMENT Background National Quality Framework 1. The Council of Australian Governments (COAG) released a consultation Regulation Impact Statement in July 2009.2. In December 2009, COAG agreed to a National Partnership Agreement on the National Quality Agenda for Early Childhood Education and Care (National Partnership Agreement). The National Partnership Agreement sets out a new, jointly governed National Quality Framework...»

«1 A Sordid God: Melville, Dante, and the Voyage to Hell Honors Research Thesis Presented in partial fulfillment of the requirements for graduation “with honors research distinction in English Literature” in the undergraduate colleges of The Ohio State University by Christine Maria Kenngott The Ohio State University May 2014 Project Advisor: Professor Elizabeth Renker, Department of English 2 ABSTRACT In my thesis I will examine the relationship between Herman Melville’s Moby-Dick and...»

«SPC/Fisheries 23/Information Paper. 9 1 August 1991 ORIGINAL ENGLISH (Noumea, New Caledonia, 5-9 August 1991) LIBRARY Secretariat of the Pacific Communiiy PRELIMINARY BIBLIOGRAPHY OF PACIFIC ISLAND TRADITIONAL FISHERY *KAl_ 11CLS C ° with contributions ? P H SPC, 1USP,eICLARM, FFA and ° K S ^ K ? U N from f e F i s h i y Su PP° rt ^gramme Johannes/Ruddle/Hviding August 1991 1 Preliminary Bibliography of Pacific Island Traditional Fishery Practices Compiled by FAO/UNDP Regional Fishery...»

«Capital Markets Practice September 2, 2014 | Number 1736 Five Eye-Opening Facts About the Philippine US$ Bond Market The Philippine bond market differs significantly from other markets, but is expected to change as growth continues. Philippine companies 1 have become avid borrowers of funds provided by the international high yield bond market, characterized in Asia by bonds denominated in US dollars without an investment-grade rating. In fact, Philippine companies issued US$11.4 billion of such...»

«VYSOKÉ UČENÍ TECHNICKÉ V BRNĚ FAKULTA ELEKTROTECHNIKY A KOMUNIKAČNÍCH TECHNOLOGIÍ ÚSTAV RADIOELEKTRONIKY Ing. Jan Puskely RECONSTRUCTION OF THE ANTENNA NEAR-FIELD REKONSTRUKCE BLÍZKÉHO POLE ANTÉN DOCTORAL THESIS SHORT VERSION Studijní obor: Elektronika a sdělovací technika Školitel: doc. Ing. Zdeněk Nováček, CSc. KLÍČOVÁ SLOVA Blízké pole antén, rekonstrukce fáze, rovinné a válcové snímání blízkého pole, bezfázová měření, globální optimalizace, obrazové...»

«ARCASIA (Updated as at 2013) ARCASIA: INTRODUCTION ARCASIA, the Architects Regional Council Asia, is a council consisting of the Presidents of National Institutes of architects in the Asian region that arc members of the organization. The organization serves as an extension for each Member Institute's regional programme and relations. This council meets annually to deliberate and to give collective direction and representation to matters that affect the architectural profession in the region....»

«INVOLUNTARY LONG HOURS IN MINING David Peetz & Georgina Murray Griffith University Long hours worked in the mining industry might reflect employee preferences. We analyse quantitative and qualitative data from the mining industry, and relevant literature, and find that employee preferences are for substantially shorter hours than are actually worked. This links to „interference‟ of work in life, including through lost family time, fatigue, interference with community and sporting activities...»

«Explaining stable partnerships among FTMs and MTFs: a significant difference? Frank Lewins School of Social Sciences, Australian National University Abstract Research on male to female (MTFs) and female to male (FTMs) transsexuals has pointed to a number of important differences between these categories, namely their different propensity towards cross dressing and relative levels of mental stability. Recent research demonstrates that these assumed differences are not supported by evidence. One...»

<<  HOME   |    CONTACTS
2016 www.dissertation.xlibx.info - Dissertations, online materials

Materials of this site are available for review, all rights belong to their respective owners.
If you do not agree with the fact that your material is placed on this site, please, email us, we will within 1-2 business days delete him.