CGImageRef訪問像素數據的更快方式？

Question

我目前的方法是：

CGDataProviderRef provider = CGImageGetDataProvider(imageRef); 
imageData.rawData = CGDataProviderCopyData(provider); 
imageData.imageData = (UInt8 *) CFDataGetBytePtr(imageData.rawData); 

我只得到約每秒30幀。我知道性能命中的一部分是複製數據，如果我可以訪問字節流並且不會自動爲我創建副本，那將會很好。

我試圖讓它儘快處理CGImageRefs，有沒有更快的方法？

這是我的工作方案片段：

- (void)applicationDidFinishLaunching:(NSNotification *)aNotification { 
    // Insert code here to initialize your application 
    //timer = [NSTimer scheduledTimerWithTimeInterval:1.0/60.0 //2000.0 
    //           target:self 
    //          selector:@selector(timerLogic) 
    //          userInfo:nil 
    //          repeats:YES]; 
    leagueGameState = [LeagueGameState new]; 

    [self updateWindowList]; 
    lastTime = CACurrentMediaTime(); 






    // Create a capture session 
    mSession = [[AVCaptureSession alloc] init]; 

    // Set the session preset as you wish 
    mSession.sessionPreset = AVCaptureSessionPresetMedium; 

    // If you're on a multi-display system and you want to capture a secondary display, 
    // you can call CGGetActiveDisplayList() to get the list of all active displays. 
    // For this example, we just specify the main display. 
    // To capture both a main and secondary display at the same time, use two active 
    // capture sessions, one for each display. On Mac OS X, AVCaptureMovieFileOutput 
    // only supports writing to a single video track. 
    CGDirectDisplayID displayId = kCGDirectMainDisplay; 

    // Create a ScreenInput with the display and add it to the session 
    AVCaptureScreenInput *input = [[AVCaptureScreenInput alloc] initWithDisplayID:displayId]; 
    input.minFrameDuration = CMTimeMake(1, 60); 

    //if (!input) { 
    // [mSession release]; 
    // mSession = nil; 
    // return; 
    //} 
    if ([mSession canAddInput:input]) { 
     NSLog(@"Added screen capture input"); 
     [mSession addInput:input]; 
    } else { 
     NSLog(@"Couldn't add screen capture input"); 
    } 

    //**********************Add output here 
    //dispatch_queue_t _videoDataOutputQueue; 
    //_videoDataOutputQueue = dispatch_queue_create("com.apple.sample.capturepipeline.video", DISPATCH_QUEUE_SERIAL); 
    //dispatch_set_target_queue(_videoDataOutputQueue, dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_HIGH, 0)); 

    AVCaptureVideoDataOutput *videoOut = [[AVCaptureVideoDataOutput alloc] init]; 
    videoOut.videoSettings = @{ (id)kCVPixelBufferPixelFormatTypeKey : @(kCVPixelFormatType_32BGRA) }; 
    [videoOut setSampleBufferDelegate:self queue:dispatch_get_main_queue()]; 

    // RosyWriter records videos and we prefer not to have any dropped frames in the video recording. 
    // By setting alwaysDiscardsLateVideoFrames to NO we ensure that minor fluctuations in system load or in our processing time for a given frame won't cause framedrops. 
    // We do however need to ensure that on average we can process frames in realtime. 
    // If we were doing preview only we would probably want to set alwaysDiscardsLateVideoFrames to YES. 
    videoOut.alwaysDiscardsLateVideoFrames = YES; 

    if ([mSession canAddOutput:videoOut]) { 
     NSLog(@"Added output video"); 
     [mSession addOutput:videoOut]; 
    } else {NSLog(@"Couldn't add output video");} 


    // Start running the session 
    [mSession startRunning]; 

    NSLog(@"Set up session"); 
} 




- (void)captureOutput:(AVCaptureOutput *)captureOutput didOutputSampleBuffer:(CMSampleBufferRef)sampleBuffer fromConnection:(AVCaptureConnection *)connection 
{ 
    //NSLog(@"Captures output from sample buffer"); 
    //CMFormatDescriptionRef formatDescription = CMSampleBufferGetFormatDescription(sampleBuffer); 
/* 
     if (self.outputVideoFormatDescription == nil) { 
      // Don't render the first sample buffer. 
      // This gives us one frame interval (33ms at 30fps) for setupVideoPipelineWithInputFormatDescription: to complete. 
      // Ideally this would be done asynchronously to ensure frames don't back up on slower devices. 
      [self setupVideoPipelineWithInputFormatDescription:formatDescription]; 
     } 
     else {*/ 
      [self renderVideoSampleBuffer:sampleBuffer]; 
     //} 
} 

- (void)renderVideoSampleBuffer:(CMSampleBufferRef)sampleBuffer 
{ 
    //CVPixelBufferRef renderedPixelBuffer = NULL; 
    //CMTime timestamp = CMSampleBufferGetPresentationTimeStamp(sampleBuffer); 

    //[self calculateFramerateAtTimestamp:timestamp]; 

    // We must not use the GPU while running in the background. 
    // setRenderingEnabled: takes the same lock so the caller can guarantee no GPU usage once the setter returns. 
    //@synchronized(_renderer) 
    //{ 
    // if (_renderingEnabled) { 
    CVPixelBufferRef sourcePixelBuffer = CMSampleBufferGetImageBuffer(sampleBuffer); 

    const int kBytesPerPixel = 4; 

    CVPixelBufferLockBaseAddress(sourcePixelBuffer, 0); 

    int bufferWidth = (int)CVPixelBufferGetWidth(sourcePixelBuffer); 
    int bufferHeight = (int)CVPixelBufferGetHeight(sourcePixelBuffer); 
    size_t bytesPerRow = CVPixelBufferGetBytesPerRow(sourcePixelBuffer); 
    uint8_t *baseAddress = CVPixelBufferGetBaseAddress(sourcePixelBuffer); 

    int count = 0; 
    for (int row = 0; row < bufferHeight; row++) 
    { 
     uint8_t *pixel = baseAddress + row * bytesPerRow; 
     for (int column = 0; column < bufferWidth; column++) 
     { 
      count ++; 
      pixel[1] = 0; // De-green (second pixel in BGRA is green) 
      pixel += kBytesPerPixel; 
     } 
    } 

    CVPixelBufferUnlockBaseAddress(sourcePixelBuffer, 0); 


    //NSLog(@"Test Looped %d times", count); 

    CIImage *ciImage = [CIImage imageWithCVImageBuffer:sourcePixelBuffer]; 


    /* 
    CIContext *temporaryContext = [CIContext contextWithCGContext: 
              [[NSGraphicsContext currentContext] graphicsPort] 
                  options: nil]; 

    CGImageRef videoImage = [temporaryContext 
          createCGImage:ciImage 
          fromRect:CGRectMake(0, 0, 
               CVPixelBufferGetWidth(sourcePixelBuffer), 
               CVPixelBufferGetHeight(sourcePixelBuffer))]; 

    */ 

    //UIImage *uiImage = [UIImage imageWithCGImage:videoImage]; 

    // Create a bitmap rep from the image... 
    NSBitmapImageRep *bitmapRep = [[NSBitmapImageRep alloc] initWithCIImage:ciImage]; 
    // Create an NSImage and add the bitmap rep to it... 
    NSImage *image = [[NSImage alloc] init]; 
    [image addRepresentation:bitmapRep]; 
    // Set the output view to the new NSImage. 
    [imageView setImage:image]; 

    //CGImageRelease(videoImage); 



    //renderedPixelBuffer = [_renderer copyRenderedPixelBuffer:sourcePixelBuffer]; 
    // } 
    // else { 
    //  return; 
    // } 
    //} 

    //Profile code? See how fast it's running? 
    if (CACurrentMediaTime() - lastTime > 3) //10 seconds 
    { 
     float time = CACurrentMediaTime() - lastTime; 
     [fpsText setStringValue:[NSString stringWithFormat:@"Elapsed Time: %f ms, %f fps", time * 1000/loopsTaken, (1000.0)/(time * 1000.0/loopsTaken)]]; 
     lastTime = CACurrentMediaTime(); 
     loopsTaken = 0; 
     [self updateWindowList]; 
     if (leagueGameState.leaguePID == -1) { 
      [statusText setStringValue:@"No League Instance Found"]; 
     } 
    } 
    else 
    { 
     loopsTaken++; 
    } 

} 

我甚至通過數據循環後得到每秒的一個非常漂亮的60幀。

它捕獲屏幕，我得到的數據，我修改數據，我重新顯示數據。

Answer 1

你的意思是哪個「字節流」？ CGImage代表最終的位圖數據，但仍然可能被壓縮。位圖可能當前存儲在GPU上，因此獲取它可能需要GPU-> CPU提取（這是昂貴的，並且當你不需要時應該避免）。

如果你想比30fps的更大做到這一點，你可能要重新考慮你如何攻擊問題，並使用專爲像核心圖片，核心顯卡，或金屬工具。核心圖形針對顯示進行了優化，而不是處理（並且絕對不是實時處理）。 Core Image等工具的一個主要區別是，您可以在GPU上執行更多工作，而無需將數據重新混合回CPU。這對於維護快速管道來說是絕對關鍵的。只要有可能，你想避免得到實際的字節。

如果您已經有CGImage，可以使用imageWithCGImage:將其轉換爲CIImage，然後使用CIImage進一步處理它。如果您確實需要訪問字節，您的選項就是您正在使用的選項，或者使用CGContextDrawImage將其呈現爲位圖上下文（這也需要複製）。沒有任何承諾CGImage在任何給定的時間都有一堆位圖字節可供您查看，並且它不提供「鎖定緩衝區」方法，就像在Core Video等實時框架中發現的那樣。

一些非常好的介紹給高速圖像處理的WWDC視頻：

• WWDC 2013屆509核心圖像效果和技術
• WWDC 2014屆在覈心的圖像514個進展
• WWDC 2014會話603-605使用金屬