2010-05-26 147 views
4

我在做一個項目稱爲用戶發起的實時目標跟蹤系統。在這裏,是我想要什麼項目發生實時目標跟蹤(一些Java API)或C#(emgucv,dshownet,Aforge.NET)

1)取來自Web攝像頭連續流。

2)使用鼠標,用戶可以在感興趣的對象周圍繪製正方形。

3)然後從那裏開始,廣場隨感興趣的對象一起移動。因此,跟蹤物體移動的每個地方,因此術語物體跟蹤。


目前進展


我已經使用dshownet(對於DirectShow的.NET包裝)取輸入從網絡攝像機。而且我正在將視頻分解爲幀。我有4個方面考慮做項目:


技術1


  • 有一個保存的視頻
  • 我加載它。
  • 當視頻運行時,我暫停(使用暫停按鈕)它,在特定的場景,並在對象上畫一個正方形。
  • 當我按下播放按鈕時,正方形會隨着沒有/ 5秒處理時間的對象一起移動[OR]我會給應用程序一些處理時間(例如3分鐘),然後它將從該點開始播放跟蹤發生。

技術2


  • 有一個保存的視頻
  • 我加載它。
  • 當視頻運行時,我不暫停它,但很快在對象上畫一個正方形(當對象仍然在某個點上時)。
  • 然後在沒有處理時間的情況下跟蹤對象。 [OR]一些處理時間(延遲10秒)使文件播放時間更長一些。

技術3


  • 我所需的時間從網絡攝像頭的輸入1分鐘。
  • 保存該視頻文件
  • 並執行方法1或方法2

技術4 - (顯然,這似乎是很多更難


  • 不斷從網絡攝像頭輸入
  • 當對象沒有任何移動時(例如,對象沒有任何暫停),在對象周圍繪製正方形。當一個人坐在椅子上時)
  • 然後通過在沒有處理時間的情況下隨着對象移動平方來顯示跟蹤[OR]輕微的處理時間爲2秒,使得延遲不明顯。

對象跟蹤: -


基本上我可以跟蹤任何東西,因爲我用鼠標繪製

  • 我計劃使用全身(但如果這是麻煩的..下一個選項)
  • 我會盡力跟蹤fac個體的E(顯然通過用鼠標繪製的區域。)

時間操作的代碼: 1和1/2個月


進展:仍然得到錯誤與分裂。 (有人建議開始第一分割保存的視頻,我在想,現在的過程)


我的問題


1)技術(出四個)我可能在1個半月的時間框架內實施?

2)向代碼,是的java +一些Java框架好這個或C#。淨emgucv/AForge.net/Dshownet [順便說一下我的知識的javaC#.NET和不那麼好]?


預先感謝

回答

2

技術1,2,3,你可以使用Java媒體框架和ImageJ的庫在Java中實現。對於技術4,在給定時間限制的情況下,最好使用C++或其他非解釋語言來實現。

+0

THKS的答覆阿德里安 – Haxed 2010-05-26 09:53:46

+0

您的歡迎,這是一個有點模糊。非常有趣的項目,祝你好運。 – 2010-05-26 09:57:13

0

This article fully explains算法非常類似於你想要的和the accompanying source code is here。你可以see it in action in this video。你需要添加的部分是,當用戶繪製一個盒子時,確定哪些對象(已經被算法找到)是盒子的周圍,然後在整個幀中簡單地跟隨帶有該int ID的對象(該算法相關對象逐幀知道它是整個視頻中的相同對象)。

聲明:我是作者,但我確實認爲這是非常有用的,並且已經成功地使用了我自己的算法。)

當談到商業計算機視覺應用時,OpenCVPoint Cloud Library aka PCL是你最好的朋友。像這樣的文章解釋瞭如何使用OpenCV等工具來完成整個堆棧運動跟蹤。 (純Java實現顯示了它如何工作到單個像素。)

0

This example基本上實現了您提到的技術4。用戶在要跟蹤的模式或對象周圍繪製矩形。在這種情況下,被跟蹤的元素用於控制乒乓遊戲中的槳。因此用戶可以使用物體在相機前面玩遊戲。

我認爲它解決了大部分問題。

截圖: enter image description here

的源代碼:

package video.trackingPong; 

import java.awt.BorderLayout; 
import java.awt.Color; 
import java.awt.Container; 
import java.awt.event.MouseEvent; 
import java.awt.event.MouseListener; 

import javax.swing.JFrame; 
import javax.swing.JLabel; 
import javax.swing.JPanel; 
import javax.swing.JSlider; 
import javax.swing.event.ChangeEvent; 
import javax.swing.event.ChangeListener; 

import marvin.gui.MarvinImagePanel; 
import marvin.image.MarvinImage; 
import marvin.image.MarvinImageMask; 
import marvin.io.MarvinImageIO; 
import marvin.plugin.MarvinImagePlugin; 
import marvin.util.MarvinAttributes; 
import marvin.util.MarvinPluginLoader; 
import marvin.video.MarvinJavaCVAdapter; 
import marvin.video.MarvinVideoInterface; 
import marvin.video.MarvinVideoInterfaceException; 

public class TrackingPong extends JFrame implements Runnable{ 

    private final static int   BALL_INITIAL_PX=100; 
    private final static int   BALL_INITIAL_PY=100; 
    private final static int   BALL_INITIAL_SPEED=3; 

    private MarvinVideoInterface videoInterface; 
    private MarvinImagePanel   videoPanel; 

    private Thread      thread; 

    private MarvinImage    imageIn, 
            imageOut; 

    private JPanel     panelSlider; 

    private JSlider     sliderSensibility; 

    private JLabel     labelSlider; 

    private int      regionPx, 
            regionPy, 
            regionWidth, 
            regionHeight; 

    private boolean     regionSelected=false; 
    private int[]     arrInitialRegion; 

    private int      sensibility=30; 



    // Pong Game Attributes 
    private double     ballPx=BALL_INITIAL_PX, 
            ballPy=BALL_INITIAL_PY; 

    private int      ballSide=15; 


    double       ballIncX=5; 
    private double     ballIncY=5;  

    private int      imageWidth, 
            imageHeight; 

    private Paddle     paddlePlayer, 
            paddleComputer; 

    private int      playerPoints=0, 
            computerPoints=0; 

    private MarvinImagePlugin   findColorPattern, 
            flip, 
            text; 

    private MarvinImage    imageBall, 
            imagePaddlePlayer, 
            imagePaddleComputer; 

    private MarvinAttributes  attributesOut; 

    public TrackingPong(){ 
     videoPanel = new MarvinImagePanel(); 

     try{ 
      // 1. Connect to the camera device. 
      videoInterface = new MarvinJavaCVAdapter(); 
      videoInterface.connect(0); 

      imageWidth = videoInterface.getImageWidth(); 
      imageHeight = videoInterface.getImageHeight(); 

      imageOut = new MarvinImage(imageWidth, imageHeight); 

      // 2. Load Graphical Interface. 
      loadGUI(); 

      // 3. Load and set up Marvin plug-ins. 
      findColorPattern  = MarvinPluginLoader.loadImagePlugin("org.marvinproject.image.pattern.findColorPattern"); 
      flip    = MarvinPluginLoader.loadImagePlugin("org.marvinproject.image.transform.flip"); 
      text    = MarvinPluginLoader.loadImagePlugin("org.marvinproject.image.render.text"); 
      text.setAttribute("fontFile", MarvinImageIO.loadImage("./res/font.png")); 
      text.setAttribute("color", 0xFFFFFFFF); 

      // 3. Load game images 
      imageBall = MarvinImageIO.loadImage("./res/ball.png"); 
      imagePaddlePlayer = MarvinImageIO.loadImage("./res/paddleA.png"); 
      imagePaddleComputer = MarvinImageIO.loadImage("./res/paddleB.png"); 

      attributesOut = new MarvinAttributes(null); 

      // Set up plater and computer paddle properties. 
      paddlePlayer = new Paddle(); 
      paddlePlayer.px=100; 
      paddlePlayer.py=420; 
      paddlePlayer.width=100; 
      paddlePlayer.height=30; 

      paddleComputer = new Paddle(); 
      paddleComputer.px=100; 
      paddleComputer.py=30; 
      paddleComputer.width=100; 
      paddleComputer.height=30; 

      thread = new Thread(this); 
      thread.start(); 
     } 
     catch(MarvinVideoInterfaceException e){ 
      e.printStackTrace(); 
     } 
    } 

    private void loadGUI(){  
     setTitle("Video Sample - Tracking Pong"); 

     videoPanel.addMouseListener(new MouseHandler()); 

     sliderSensibility = new JSlider(JSlider.HORIZONTAL, 0, 60, 30); 
     sliderSensibility.setMinorTickSpacing(2); 
     sliderSensibility.setPaintTicks(true); 
     sliderSensibility.addChangeListener(new SliderHandler()); 

     labelSlider = new JLabel("Sensibility"); 

     panelSlider = new JPanel(); 
     panelSlider.add(labelSlider); 
     panelSlider.add(sliderSensibility); 

     Container container = getContentPane(); 
     container.setLayout(new BorderLayout()); 
     container.add(videoPanel, BorderLayout.NORTH); 
     container.add(panelSlider, BorderLayout.SOUTH); 

     setSize(videoInterface.getImageWidth()+20,videoInterface.getImageHeight()+100); 
     setVisible(true); 
    } 

    public void run(){ 
     long time = System.currentTimeMillis(); 
     int ticks=0; 

     // The game loop. 
     try{ 
      while(true){ 

       ticks++; 
       if(System.currentTimeMillis() - time > 1000){ 
        System.out.println("FPS: "+ticks+"  "); 
        ticks=0; 
        time = System.currentTimeMillis();      
       } 

       // 1. Get the current video frame. 
       imageIn = videoInterface.getFrame(); 
       MarvinImage.copyColorArray(imageIn, imageOut); 

       // 2. Flip the frame horizontally so the player will see him on the screen like looking at the mirror. 
       flip.process(imageOut, imageOut); 

       if(regionSelected){ 

        // 3. Find the player paddle position. 
        findColorPattern.setAttribute("differenceColorRange", sensibility); 
        findColorPattern.process(imageOut, imageOut, attributesOut, MarvinImageMask.NULL_MASK, false); 
        regionPx   = (Integer)attributesOut.get("regionPx"); 
        regionPy   = (Integer)attributesOut.get("regionPy"); 
        regionWidth  = (Integer)attributesOut.get("regionWidth"); 
        regionHeight = (Integer)attributesOut.get("regionHeight"); 

        // 4. Invoke the game logic 
        pongGame(); 

        // 5. Draw the detected region 
        imageOut.drawRect(regionPx, regionPy, regionWidth, regionHeight, Color.red); 

        // 6. Draw the player and computer points. 
        text.setAttribute("x", 105); 
        text.setAttribute("y", 3); 
        text.setAttribute("text", "PLAYER:"+playerPoints); 
        text.process(imageOut, imageOut); 

        text.setAttribute("x", 105); 
        text.setAttribute("y", 460); 
        text.setAttribute("text", "COMPUTER:"+computerPoints); 
        text.process(imageOut, imageOut); 
       } 


       videoPanel.setImage(imageOut); 
      } 
     } 
     catch(MarvinVideoInterfaceException e){ 
      e.printStackTrace(); 
     } 
    } 

    private void pongGame(){ 
     // 1. Move the ball 
     ballIncX*=1.001; 
     ballIncY*=1.001; 
     ballPx+=ballIncX; 
     ballPy+=ballIncY; 

     // 2. Set the player paddle position to the the coordinates of the detected region. 
     paddlePlayer.px = regionPx+((regionWidth-paddlePlayer.width)/2); 

     // 3. Invoke simple computer AI 
     computerAI(); 

     // 4. Check object positions and collisions. 
     checkPaddlePosition(paddlePlayer); 
     checkPaddlePosition(paddleComputer); 
     collisionScreen(); 
     collisionTap(); 

     // 5. Draw the game elements. 
     imageOut.fillRect(horizontalMargin, 0, 5, imageHeight, Color.black); 
     imageOut.fillRect(imageWidth-horizontalMargin, 0, 5, imageHeight, Color.black); 

     combineImage(imagePaddlePlayer, paddlePlayer.px, paddlePlayer.py); 
     combineImage(imagePaddleComputer, paddleComputer.px, paddleComputer.py); 
     combineImage(imageBall,(int)ballPx, (int)ballPy); 
    } 

    private void checkPaddlePosition(Paddle a_paddle){ 
     if(a_paddle.px < horizontalMargin){ 
      a_paddle.px = horizontalMargin; 
     } 
     if(a_paddle.px+a_paddle.width > imageWidth-horizontalMargin){ 
      a_paddle.px = imageWidth-horizontalMargin-a_paddle.width; 
     }   
    } 

    private void computerAI(){ 
     if(ballPx < paddleComputer.px+(paddleComputer.width/2)-10){ 
      paddleComputer.px-=4; 
     } 
     if(ballPx > paddleComputer.px+(paddleComputer.width/2)+10){ 
      paddleComputer.px+=4; 
     } 
    } 

    private int horizontalMargin = 100; 
    private void collisionScreen(){ 

     if(ballPx < horizontalMargin){ 
      ballPx = horizontalMargin; 
      ballIncX*=-1; 
     } 
     if(ballPx+ballSide >= imageWidth-horizontalMargin){ 
      ballPx=(imageWidth-horizontalMargin)-ballSide; 
      ballIncX*=-1; 
     } 
     if(ballPy < 0){ 
      playerPoints++; 
      ballPx = BALL_INITIAL_PX; 
      ballPy = BALL_INITIAL_PY; 
      ballIncY=BALL_INITIAL_SPEED; 
      ballIncX=BALL_INITIAL_SPEED; 
     } else if(ballPy+ballSide >= imageHeight){ 
      computerPoints++; 
      ballPx = BALL_INITIAL_PX; 
      ballPy = BALL_INITIAL_PY; 
      ballIncY=BALL_INITIAL_SPEED; 
      ballIncX=BALL_INITIAL_SPEED; 
     } 
    } 

    private void collisionTap(){ 
     if(ballCollisionTap(paddlePlayer)){ 
      ballIncY*=-1; 
      ballPy = paddlePlayer.py-ballSide; 
     } 
     if(ballCollisionTap(paddleComputer)){ 
      ballIncY*=-1; 
      ballPy = paddleComputer.py+paddleComputer.height; 
     } 
    } 

    private boolean ballCollisionTap(Paddle a_tap){ 
     if 
     ( 
      ( 
       ballPx >= a_tap.px && ballPx <= a_tap.px+a_tap.width || 
       ballPx <= a_tap.px && ballPx+ballSide >= a_tap.px 
      ) 
      && 
      ( 
       ballPy >= a_tap.py && ballPy <= a_tap.py+a_tap.height || 
       ballPy <= a_tap.py && ballPy+ballSide >= a_tap.py 
      ) 
     ) 
     { 
      return true; 
     } 
     return false; 
    } 

    private void combineImage(MarvinImage img, int x, int y){ 
     int rgb; 
     int width = img.getWidth(); 
     int height = img.getHeight(); 

     for(int iy=0; iy<height; iy++){ 
      for(int ix=0; ix<width; ix++){ 
       if 
       ( 
        ix+x > 0 && ix+x < imageWidth && 
        iy+y > 0 && iy+y < imageHeight 
       ) 
       { 
        rgb=img.getIntColor(ix, iy);     
        if(rgb != 0xFFFFFFFF){ 
         imageOut.setIntColor(ix+x, iy+y, rgb); 
        } 
       } 
      } 
     }   
    } 

    public static void main(String args[]){ 
     TrackingPong trackingPong = new TrackingPong(); 
     trackingPong.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE); 
    } 

    private class SliderHandler implements ChangeListener{ 
     public void stateChanged(ChangeEvent a_event){ 
      sensibility = (60-sliderSensibility.getValue()); 
     } 
    } 

    private class MouseHandler implements MouseListener{ 
     public void mouseEntered(MouseEvent a_event){} 
     public void mouseExited(MouseEvent a_event){} 
     public void mousePressed(MouseEvent a_event){} 
     public void mouseClicked(MouseEvent a_event){} 

     public void mouseReleased(MouseEvent event){ 
      if(!regionSelected){ 
       if(arrInitialRegion == null){ 
        arrInitialRegion = new int[]{event.getX(), event.getY(),0,0}; 
       } 
       else{ 
        arrInitialRegion[2] = event.getX()-arrInitialRegion[0]; 
        arrInitialRegion[3] = event.getY()-arrInitialRegion[1]; 

        findColorPattern.setAttribute("regionPx", arrInitialRegion[0]); 
        findColorPattern.setAttribute("regionPy", arrInitialRegion[1]); 
        findColorPattern.setAttribute("regionWidth", arrInitialRegion[2]); 
        findColorPattern.setAttribute("regionHeight", arrInitialRegion[3]); 

        regionSelected = true; 
       }  
      } 
     }   
    } 

    private class Paddle{ 
     public int px,py,width,height; 
    } 
}