Proc. of Int. Conf. on Recent Trends in Information, Telecommunication and Computing, ITC
Motion Trajectory of Players using Optical Flow Ashwini D.Narhare1, G.V.Molke2 and Dr. Rajendra Kanphade3 1
Department of E&TC,PG Student,Padmashree Dr.D.Y.Patil institute of engineering and technology,pune,India Email: narhare.ashwini@rediffmail.com 2 Department of E&TC, Asso. Professor,Padmashree Dr.D.Y.Patil institute of engineering and technology,pune,India Email: molkegv72@gmail.com 3 Principal,Nutan Maharastra institute of engineering and technology, Talegaon Dabhade, pune,India Email:rdkanphade@gmail.com
Abstract— This paper gives a brief idea of the moving objects tracking and its application. In sport it is challenging to track and detect motion of players in video frames. Task represents optical flow analysis to do motion detection and particle filter to track players and taking consideration of regions with movement of players in sports video. Optical flow vector calculation gives motion of players in video frame. This paper presents improved Luacs Kanade algorithm explained for optical flow computation for large displacement and more accuracy in motion estimation. Index Terms— optical flow analysis, improved Lucas Kanade algorithm, segmentation, particle filter tracking
I. INTRODUCTION Motion detection and tracking of players estimates motion of players. Tracking of players from video is automatic method and it initializes player’s position and corrects mistakes by observing position during tracking. Motion tracking of players is useful for broadcasters and sportsmen’s at individual level of player so can improve quality of game. Coach can get information about team quality .Coach can give best development in game. Also coach can get hide movements of players due to the eyes through the camera. In broadcasting player’s motion information can enhance using graphical analysis and movement can display on the screen to see for the viewers. Observation of movements of players is important to improve skill of player and team energy. Also it is useful to understanding the game. Tracking motions of players provide picturing of path in the game. Because of this the research is concentrating now a days to do analysis automatically or semi automatically using different data source like statistical data of game or manually labeling events of game .This task needs information of players movement . Motion trajectory data gives movement of players which is important in motion analysis. For many years analysis of motion of players have done by observation sheets. In 1980’ video recording is developed for motion understanding. Motion analysis of players were done manually .It was time consuming and difficult task. Previous computer vision technology was slow due to computational abilities. But recently tracking process is growing to understand motion of players to do motion analysis of players. It is fast, time saving and easy technique to improve game for players. Moving object detection in computer vision involves identification of the presence of an object in consecutive frames where as it is used to monitor the movements with respect to the region of interest .Many technology and methods developed for player’s motion trajectory. Ref. [1], [2],[3],[5],[6], provided methods for tracking motion of players for motion analysis of games. Each DOI: 02.ITC.2014.5.82 © Association of Computer Electronics and Electrical Engineers, 2014
author worked on the players movement information by using motion calculation algorithms .Players tracking is done by using particle filters . Ref. [1], used motion estimation algorithm to know the motion of players. Support vector machine recognized action of players. Tactics analysis of tennis is based on recognized result and giving stroke is scored or not Ref. [2]represented action recognition on template based methodology in basketball game did analysis of game in the form of small scale and large scale model .Ref.[3], homography is computed by merging RANSAC with KLT and DLT (direct linear transform) algorithm and tracked object with static background by using particle filter. Ref. [5] represented multiple football players tracking using multiple particle filters. Segmented players region to know movement of players. Multiple states generated with filter follow the track of players. For tracking did players inter camera and inner camera operations. Ref. [7], motion of players is estimated by using optical flow which is gradient based assuming brightness is constant in neighboring pixel. Partial derivative of brightness is found and applied edge detector. A motion characteristic within tracking area is done by statistical tool .Ref. [8], image intensity function derivative solved and motion related statics generated.Ref. [9], algorithm is introduced convergent optical flow computation and combined with Lucas Kanade pyramid different to LKP method Implemented algorithm for tracking a single player in a sport.Ref.[10], presented flexible and expandable approach for detection, tracking and recognition of players. For tracking of players particle filter is used. Ref. [13], used particle filter for tracking and presented how particle filter has speed in calculation and accuracy. This paper has six sections. Introduction part is over in section 1.Section 2 is about system overview and problem definition. Section 3 covered input details. Sec. 4 explaining motion detection of players. Sec 5.providing Particle filter tracking of players. Sec. 6 is showing result. II. PROPOSED SYSTEM Now a day’s huge amount of multimedia information needs to develop motion analysis. These papers explain motion detection and tracking method in the video sequence by using optical flow. The key to calculate motion in video frame is optical flow. Optical flow is calculated by using improved lucas kanade algorithm. Ref. [1],used horn-shunuk motion estimation algorithm. Ref.[2] did segmentation to get movement of players Ref.[3], combination of KLT and DLT and RSNSC algorithm estimated motion of hockey players Previous methods of motion estimation has limitation to calculate optical flow for big displacement of players and in calculation accuracy. Present system gives more correctness in estimation and calculates large displacements of pixel in image sequences.Fig.1, shows System for motion analysis of sports players. Proposed sytem consists three parts.Video clip ,Detection of playres which consist of optical flow analysis,multiscale lucas kanade pyramidal implememtation and segmentation,tracking of players using particle filter.In the system it mainly has three parts one is video clipt,players motion detection and playres tracking .video clip has number of frames.Frame extraction is done to get important and complete imformation for motion detection and tracking of playres.Optical flow algorithm of Lucas Kanade applied further for detection of players motion.Segmetation is done for thresholding which separates images to easily read.Then tracking players is done by particle filter it updates the current state of players .It first finds current state of player by using prediction equation then upade the state of players. III. VIDEO CLIP In the current information epoch, maximum data are representing and processing in multimedia form. In this data video consists frames which contains similar information are usually processed. This cause to unwanted time wastage, slow speed and results complexity. Video data has number of frames in sequence with different frame rates. Frame rate also known as frame frequency or frames per second .Ref.[4], at this frequency (rate) an imaging device produces unique consecutive images called frames.Frame are extracted from given input video . It is essential to do analysis of video and select frames. In the huge video data sometimes invalid data is transfer in the process. Frame extraction manages and passes valid video data in the system. For given video this frames are use motion features. For given video this frames are use motion features. Video data manages and processes due to extraction. For one second video, at least thirty frames are needed .Ref. [14, 4], All frames are not required to display important information as frame is similar some of them don’t need. Frame which need and has important information is key frame. When complete information of a video want to display fast then key framed are needed instead of all frames. Key frames are then packed into file called shot. Each key frame consists all necessary information in a video shot of the video. This is video 349
summarization. It is also a compact illustration of a video sequence. Frame extraction is main feature of a video summarization.
Figure 1.Motion estimation and tracking system using optical flow for sports players
IV. PLAYERS MOTION DETECTION A. Optical Flow Analysis Optical flow is a two-frame differential method for motion estimation. We use the Lucas窶適anade method [Lucas and Kanade, 1981], which is one of the most popular & simple implementation. Only movement between frames defines motion. Motion vector plays one important feature in moving object. However, the motion vector is required to characterize the real motion displacement. Optical flow is gradient-based approach, is estimated based on the rapid change in intensity of pixels. Optical motion is pattern of apparent motion of objects and viewers in sequential frames. It calculates translation of each pixel in a region. It is a field displacement vector or motion vector. Motion is calculated by motion vectors using optical flow. It calculates distance of moving object from frame to frame using feature extraction. Optical flow computation is easy and accurate methodology to identify moving object. It works on assumption that brightness of neighbouring pixel is constant. Optical flow computes flows of motion from adjacent frame. Fig. 2 is representation of motion of players in frame to frame. However, due to the optical flow constraint, the obtained optical flow does not represent the true motion, but only the motion estimate on the direction of image gradient. Takes two images at time t and t+dt. Assuming image intensity at t and t+dt is constant. Then we can get .
(1)
Also assume small motion and after applying Taylor series getting +
+ H.O.T.
.
(2) (3)
Rewriting equation .
(4)
350
(4), equation which represent optical flow lying on x axis and optical flow component lying on y axis. are the first derivative of image at (x, y, t).
Figure 2.Motion of players in sequence of frames
Simplifying and writing this equation and Put dx/dt=u and dy/dt=v by rewriting equation in terms of u and v get, . (5) (u.v) is optical flow vector which is also motion vector. Key assumptions of Lucas-Kanade Tracker 1) pixel’s neighbors have same (u, v).2) Brightness constancy: projection of the same point looks the same in every frame. (x, y) are coordinates of frame t and (u, v) are coordinates at t+1.u and v are image velocity .Taking small widow size w*w with w>1. (6) Creates one equation and two unknowns. By applying equation to more pixels in the window which is centered at(x, y) giving matrix form (6) Therefore ATA
=-ATB where
T
A
Now get =- (ATA)-1ATB (7) (5),is basic optical flow equation.In these system optical flow computation approach is improved LucasKanade algorithm for better accuracy called Multiscale Lucas kanade Pyramid. Lucas Kanade method is less sensitive to noise developed by Bruce D. Lucas and Takeo Kanade. It is depend on assumption that optical flow is constant in neighbouring pixels. It solves basic equations by the least square criteria to determine optical flow. MSLKP assume input sequence with high resolution .At each pair of image in input sequence, pyramid transform is applied to derive low resolution images. The result of one level on pyramid is correct solution. B. Improved Lucas Kanade Algorithm Until now, were dealing with small motions. So it fails when there is large motion. So again we go for pyramids. pyramid, removed small motions and large motions becomes small motions. So applying LucasKanade there, get optical flow along with the scale. At each level displacement of pixels calculated and improved accuracy because of image sampling. Fig. 3, is a multi-resolution representation of an image formed by few images, each one a subsampled version of the original one at increasing standard aberration of frame at t and frame at t+1 .Lucas Kanade assume that motion vector in any given region is constant it merely shifts form one position to another. Multiscale Lucas Kanade Pyramid methodology is based on resolution of images. At each level of pyramid resolution of image is changed. Image represented in Multiscales .Smoothing and subsampling is done of images .Firstly it reads 351
image. And level L to L-1 variation in resolution of image is done. First level has original image. Ref. [7][8], Taking Gaussian pyramid of images at time t and t+1 in the frame from high resolution to low resolution and applying and running Lucas-kanade to each level from high level to lower level wrapping and up sampling is done. Steps in the algorithm: • Optical flow is calculated in the deepest pyramid level using Lucas-Kanade flow algorithm. • Guess displacement of pixels initially and result of lower level is applied to upper level • At this level optical flow computation is done. • The process repeated until highest level of pyramid. Thus u and v is calculated. The result of optical flow computation is shown in Fig. 2, of players. By applying Lucas Kanade on each level by up sampling and wrapping got a good result for large displacement of players.For motion estimation optical flow gives motion vector μ and ν. Optical flow is motion vector where it is corresponds to the movement of perceived feature.
Figure 3.Multiscale Lucas Kanade pyramid
C. Segmentation Thresholding is approach for segmentation the images in video frames. Segmentation separates background and foreground. To segment the movable object the motion vector determines the pixels in the frame is moving or not. Thresholding is applied on optical flow to separate moving group of pixels. Thresholding value is variable and its value varies from frame to the next frame. Factors like environmental conditions, illumination and camera setting parameters changes value of thresholding. Handling small optical flow vector is main challenge in segmentation .This difficulties for small optical flow vectors are due to background noise. The segmentation result is shown in fig.3
Where absolute value for optical flow and threshold value is mean absolute of the segmented object is input to tracking .It is described using following condition, Else f(x, y) =255 V. PLAYERS TRACKING Tracking is done after segmentation. Ref. [1], [3], [5], [4], for Tracking of players particle filter is used Particle filter track interested region and particles. Particle filters is also SMC sequential Monte Carlo method. This algorithm estimate density by implementing Bayesian recursion equations. SMC method use set of particles to estimate density. This methodology generates samples through state distribution. Distribution samples are nothing but set of particles. Ref. [11], each particle consists of weight to represent probability of that particle sampled from the probability distribution function It is robust over noise. In other words, each particle is a guess representing one possible location of the object being which is to be tracked. The set of particles contains more weight at locations where the object being tracked is more likely to be. 352
Ref. [12],[13],this weighted distribution is propagated through time using Bayesian filtering equations, and can calculate the trajectory of the tracked object by taking the particle with the highest weight. Particle filterbased object trackers have proven to be very effective. Conceptually, a particle filter-based tracker maintains a probability distribution over the state (location, scale, etc.) of the object being tracked. Let rt-1 represent the state of tracked object at time t-1And ot-1 is observation at t-1.o1: t-1represent a set of all observations up to t-1.All information about the target’s state rt-1 is exemplified by its posterior P (rt−1|o1:t−1).The aim of tracking is then to evaluate these subsequent new observations ot arrive. This process is divided by two steps: 1. prediction 2. Update. Prediction shown by using following equation P (rt|o1: t-1) = ∫p (rt|rt-1) p (rt−1|o1:t−1)drt-1
(8)
Update step is shown by following equation P (rt|o1: t)α p (ot|rt)
p(rt|o1:t-1)
(9)
Monte Carlo simulation is done for these steps. In application, we use condensation algorithm, means simple particle filter, where the posterior at time-step t-1 is represented by a finite set of weighted particles. Dynamical model describing the state progress P (rt |rt-1) and a model that estimates the likelihood of any state given the observation P (ot|rt).Players’ tracking is shown in fig. 7.In which prediction of state of players is estimated and then next state is updated. Observation is depending only on current state. Particle filter tracking flow consist of following steps. • Start with initial state and distribution. • Draw samples to find current state means predict step. • Using prediction equation find next state means update step. • Defined the weight of particles. • Resample the particles using resampling technique. • Go to predict step till all observations are exhausted. VI. RESULT Covers here result and motion estimation and trajectory of player’s algorithm discussion. Algorithm tested with real and various data under static and dynamic environment .Implementation and motion estimation analysis is done and tested under the mat lab with its functionality. Simulation is done in Matlab to give simulation of system. Fig. 4 simulation model in Matlab is shown for this system. Motion estimation is done by calculating displacement of pixels from neighbouring frames. In the motion estimation system optical flow algorithm is applied. Multiscale Lucas Kanade Pyramidal implementation is improved Lucas-Kanade method. it gained better robustness and accuracy. Detection of player’s motion used corner feature extraction.
Figure 4.Motion estimation of player’s system simulation model under the Matlab using optical flow
Fig. 5,The optical flow vector result of players using Gaussian smoothing based Lucas Kanade with pyramidal implementation under the Matlab.Pyramidal implementation of Lucas Kanade gives smoother motion vector result with less density, high accuracy, and better speed in calculation of motion vector. 353
Fig.6,Thresholding done segmentation of images it creates binary image. This gives simple representation of images. It separate background and foreground images. It extracts foreground image. Some different
Figure 5.Motion vector of player’s using Multiscale Lucas Kanade Pyramid implementation in Matlab
problems need to be solving while tracking players 1.occulsion between players 2.size of players in pixels3.players shape and colors. Particle filter is efficient for this solution in visual tracking. It handles uncertainty and provides better result in complex background. It increases robustness and accuracy.
Figure 6. Result obtained by selecting threshold over motion vector result
Figure 7. Tracking of players using particle filter
VII. CONCLUSIONS A system of trajectory of motion of players in any game is presented. Motion analysis of sports players is now a demanding concept for study and understands improvement in game in which movement of players estimated by using multi scale Lucas Kanade pyramidal implementation .MSLKP method is giving accurate result for large displacement. Then players tracked on a pitch using a particle filter. This is robust and effective tracking method for players.
354
ACKNOWLEDGMENT We place on record and warmly acknowledge the continuous encouragement invaluable support and timely help offered by Dr. R. K. Jain, Principal, Pd.Dr.D.Y. Patil Institute of Engineering and Technology, Pimpri in bringing this report to successful completion we express our sincere thanks to our teachers who have rendered the basic knowledge and sowed the seeds of interest and inquisitiveness in each one of us. We express our special thanks to all our friends who have patiently extended all sort of help directly or indirectly. REFERENCES [1] Guangyu Zhu , Changsheng Xu 2, Qingming Huang , Wen Gao , and Liyuan Xing .Player Action Recognition in Broadcast Tennis Video with Applications to Semantic Analysis of Sports Game .Beijing Natural Science Foundation: 4063041 MM'06, October 23-27, 2006, Santa Barbara, California, USA. [2] Graz, Austria ,A Template-Based Multi-Player Action Recognition of the Basketball Game. CVBASE '06 Proceedings of ECCV Workshop on Computer Vision, Based Analysis in Sport Environments, , May 12th, 2006, pp. 71-82 [3] Kenji Okuma James J. Little David Lowe .Automatic Acquisition of Motion Trajectories: Tracking Hockey Players. Internet Imaging V. Edited by Santini, Simone; Schettini, Raimondo. Proceedings of the SPIE, Volume 5304, pp. 202-213 (2003). [4] Alper Yilmaz, Omar Javed, Mubarak Shah. Object Tracking: A Survey . ACM Computing Surveys, Vol. 38, No. 4, Article 13, Publication date: December 2006. [5] Anthony Dearden', Yiannis Demirisa and Oliver Grau b. Tracking Football Player Movement From a Single Moving Camera Using Particle Filters. Published in: Visual 2006 [6] Matej Kristan, Janez Perˇs, Matej Perˇse, and Stanislav Kovaˇc.Faculty of Electrical Engineering.Towards fast and efficient methods for tracking, players in sports. IEEE Trans. Aerospace and Electronic Systems, 2006 [7] Wolfgang Schulz.Optical Flow as a property of moving objects Used for their registration. Computer Vision Course Project. www.anigators.com/cvision/optflow.pdf. [8] R. Fablet, P. Bouthemy.Statistical motion-based object indexing using optic flowfield. 2000. Proceedings. 15th International Conference on, Volume: 4 [9] Naoya Ohnishi, Atsushi Imiya, Leo Dorst, and Reinhard Klette.Zooming Optical Flow Computation.2007. [10] Vladimir Pleština, Hrvoje Dujmić, Vladan Papić, Jean-Marie Zogg.A modular system for tracking players in sports, Games. GPS basics, Volume 1, AIAA-Inc. Page 89 INTERNATIONAL JOURNAL OF EDUCATION AND INFORMATION TECHNOLOGIES Issue 4, Volume 3, 2009. [11] Catarina B. Santiago, Lobinho Gomes, Armando Sousa.Tracking Players in Indoor Sports Using a Vision System Inspired in Fuzzy and parallel processing .Cutting Edge Research in New Technologies, April 5, 2012. [12] Rob Hess” Particle Filter Object Tracking”, IEEE journal 2011 [13] Kristan, JanezPerˇs, Matej Perˇse, andStanislav Kovaˇc. Towards fast and efficient methods for tracking players insports .IEEE Trans. Aerospace and Electronic Systems, 2006. [14] Khin Thandar Tint, Dr. Kyi Soe I. Key Frame Extraction for Video Summarization Using DWT Wavelet Statistics. International Journal of Advanced Research in Computer Engineering & Technology (IJARCET) Volume 2, No 5, May 2013.
355