基于虹软SDK，适配Camera1、Camera2、CameraX，实现人脸识别（Android）
作者：互联网
这篇文章主要介绍分别采用Camera1、Camera2、CameraX API接口获取Camera数据流，并集成ArcSoft人脸识别算法。
ArcSoft官方的demo是采用的Camera1接口，我前面也写过一篇单独Camera2 接口集成Arcsoft接口的文章（全网首发：Android Camera2 集成人脸识别算法）
01应用设计流程图
如下图所示，应用流程比较简单，分别从不同的API接口获取到Camera数据流数据，然后送到ArcSoft人脸识别算法库中进行识别，最终将识别结果绘制到界面上。
02应用界面
CameraX需要和界面生命周期进行绑定，所以主界面设计成了2个Button入口，一个入口是Camera1和Camera2共用，一个是CameraX独立的入口。
如下图所示：Camera1和Camera2之间可以互相切换。
CameraX是单独的界面。
03代码实现
1） Camera1 API的使用：
private void startCameraByApi1() {
DisplayMetrics metrics = new DisplayMetrics();
getWindowManager().getDefaultDisplay().getMetrics(metrics);
CameraListener cameraListener = new CameraListener() {
@Override
public void onCameraOpened(Camera camera, int cameraId, int displayOrientation, boolean isMirror) {
Camera.Size previewSize = camera.getParameters().getPreviewSize();
mPreviewSize = new Size(previewSize.width,previewSize.height);
drawHelper = new DrawHelper(previewSize.width, previewSize.height, previewView.getWidth(), previewView.getHeight(), displayOrientation
, cameraId, isMirror, false, false);
}
@Override
public void onPreview(byte[] nv21, Camera camera) {
drawFaceInfo(nv21);
}
@Override
public void onCameraClosed() {
}
@Override
public void onCameraError(Exception e) {
}
@Override
public void onCameraConfigurationChanged(int cameraID, int displayOrientation) {
if (drawHelper != null) {
drawHelper.setCameraDisplayOrientation(displayOrientation);
}
}
};
cameraAPI1Helper = new Camera1ApiHelper.Builder()
.previewViewSize(new Point(previewView.getMeasuredWidth(), previewView.getMeasuredHeight()))
.rotation(getWindowManager().getDefaultDisplay().getRotation())
.specificCameraId(Camera.CameraInfo.CAMERA_FACING_BACK)
.isMirror(false)
.previewOn(previewView)
.cameraListener(cameraListener)
.build();
cameraAPI1Helper.init();
cameraAPI1Helper.start();
}
2） Camera2 API的使用：
private void openCameraApi2(int width, int height) {
Log.v(TAG, "---- openCameraAPi2();width: " + width + ";height: " + height);
setUpCameraOutputs(width, height);
CameraManager manager = (CameraManager) getSystemService(Context.CAMERA_SERVICE);
try {
if (!mCameraOpenCloseLock.tryAcquire(2500, TimeUnit.MILLISECONDS)) {
throw new RuntimeException("Time out waiting to lock back camera opening.");
}
manager.openCamera(mCameraId, mStateCallback, mBackgroundHandler);
} catch (CameraAccessException e) {
e.printStackTrace();
} catch (InterruptedException e) {
throw new RuntimeException("Interrupted while trying to lock camera opening.", e);
}
}
private final CameraDevice.StateCallback mStateCallback = new CameraDevice.StateCallback() {
@Override
public void onOpened(@NonNull CameraDevice cameraDevice) {
initArcsoftDrawHelper();
mCameraOpenCloseLock.release();
mCameraDevice = cameraDevice;
try {
Thread.sleep(500);
} catch (InterruptedException e) {
e.printStackTrace();
}
createCameraPreviewSession();
}
@Override
public void onDisconnected(@NonNull CameraDevice cameraDevice) {
mCameraOpenCloseLock.release();
cameraDevice.close();
mCameraDevice = null;
}
@Override
public void one rror(@NonNull CameraDevice cameraDevice, int error) {
mCameraOpenCloseLock.release();
cameraDevice.close();
mCameraDevice = null;
finish();
}
};
private void createCameraPreviewSession() {
try {
SurfaceTexture texture = previewView.getSurfaceTexture();
assert texture != null;
// We configure the size of default buffer to be the size of camera preview we want.
texture.setDefaultBufferSize(mPreviewSize.getWidth(), mPreviewSize.getHeight());
// This is the output Surface we need to start preview.
Surface surface = new Surface(texture);
mPreviewRequestBuilder
= mCameraDevice.createCaptureRequest(CameraDevice.TEMPLATE_PREVIEW);
mPreviewRequestBuilder.addTarget(surface);
mPreviewRequestBuilder.addTarget(mImageReader.getSurface());
mCameraDevice.createCaptureSession(Arrays.asList(surface, mImageReader.getSurface()),
new CameraCaptureSession.StateCallback() {
@Override
public void onConfigured(@NonNull CameraCaptureSession cameraCaptureSession) {
Log.v(TAG, "--- Camera2API:onConfigured();");
if (null == mCameraDevice) {
return;
}
mPreviewCaptureSession = cameraCaptureSession;
try {
mPreviewRequestBuilder.set(CaptureRequest.CONTROL_AF_MODE,
CaptureRequest.CONTROL_AF_MODE_CONTINUOUS_PICTURE);
mPreviewRequest = mPreviewRequestBuilder.build();
mPreviewCaptureSession.setRepeatingRequest(mPreviewRequest,
null, mBackgroundHandler);
} catch (CameraAccessException e) {
e.printStackTrace();
}
}
@Override
public void onConfigureFailed(
@NonNull CameraCaptureSession cameraCaptureSession) {
showToast("Failed");
}
}, null
);
} catch (CameraAccessException e) {
e.printStackTrace();
}
}
private final ImageReader.OnImageAvailableListener mOnImageAvailableListener
= new ImageReader.OnImageAvailableListener() {
@Override
public void onImageAvailable(ImageReader reader) {
Image image = reader.acquireLatestImage();
if(image == null){
return;
}
synchronized (mImageReaderLock) {
if(!mImageReaderLock.equals(1)){
Log.v(TAG, "--- image not available,just return!!!");
image.close();
return;
}
if (ImageFormat.YUV_420_888 == image.getFormat()) {
Image.Plane[] planes = image.getPlanes();
lock.lock();
if (y == null) {
y = new byte[planes[0].getBuffer().limit() - planes[0].getBuffer().position()];
u = new byte[planes[1].getBuffer().limit() - planes[1].getBuffer().position()];
v = new byte[planes[2].getBuffer().limit() - planes[2].getBuffer().position()];
}
if (image.getPlanes()[0].getBuffer().remaining() == y.length) {
planes[0].getBuffer().get(y);
planes[1].getBuffer().get(u);
planes[2].getBuffer().get(v);
if (nv21 == null) {
nv21 = new byte[planes[0].getRowStride() * mPreviewSize.getHeight() * 3 / 2];
}
if(nv21 != null && (nv21.length != planes[0].getRowStride() * mPreviewSize.getHeight() *3/2)){
return;
}
// 回传数据是YUV422
if (y.length / u.length == 2) {
ImageUtil.yuv422ToYuv420sp(y, u, v, nv21, planes[0].getRowStride(), mPreviewSize.getHeight());
}
// 回传数据是YUV420
else if (y.length / u.length == 4) {
ImageUtil.yuv420ToYuv420sp(y, u, v, nv21, planes[0].getRowStride(), mPreviewSize.getHeight());
}
//调用Arcsoft算法,绘制人脸信息
drawFaceInfo(nv21);
}
lock.unlock();
}
}
image.close();
}
};
3） CameraX API的使用：
private void startCameraX() {
Log.v(TAG,"--- startCameraX();");
mPreviewSize = new Size(640,480);
setPreviewViewAspectRatio();
initArcsoftDrawHelper();
Rational rational = new Rational(mPreviewSize.getHeight(), mPreviewSize.getWidth());
// 1. preview
PreviewConfig previewConfig = new PreviewConfig.Builder()
.setTargetAspectRatio(rational)
.setTargetResolution(mPreviewSize)
.build();
Preview preview = new Preview(previewConfig);
preview.setOnPreviewOutputUpdateListener(new Preview.OnPreviewOutputUpdateListener() {
@Override
public void onUpdated(Preview.PreviewOutput output) {
previewView.setSurfaceTexture(output.getSurfaceTexture());
configureTransform(previewView.getWidth(),previewView.getHeight());
}
});
// 2. capture
ImageCaptureConfig imageCaptureConfig = new ImageCaptureConfig.Builder()
.setTargetAspectRatio(rational)
.setCaptureMode(ImageCapture.CaptureMode.MIN_LATENCY)
.build();
final ImageCapture imageCapture = new ImageCapture(imageCaptureConfig);
// 3. analyze
HandlerThread handlerThread = new HandlerThread("Analyze-thread");
handlerThread.start();
ImageAnalysisConfig imageAnalysisConfig = new ImageAnalysisConfig.Builder()
.setCallbackHandler(new Handler(handlerThread.getLooper()))
.setImageReaderMode(ImageAnalysis.ImageReaderMode.ACQUIRE_LATEST_IMAGE)
.setTargetAspectRatio(rational)
.setTargetResolution(mPreviewSize)
.build();
ImageAnalysis imageAnalysis = new ImageAnalysis(imageAnalysisConfig);
imageAnalysis.setAnalyzer(new MyAnalyzer());
CameraX.bindToLifecycle(this, preview, imageCapture, imageAnalysis);
}
private class MyAnalyzer implements ImageAnalysis.Analyzer {
private byte[] y;
private byte[] u;
private byte[] v;
private byte[] nv21;
private ReentrantLock lock = new ReentrantLock();
private Object mImageReaderLock = 1;//1 available,0 unAvailable
@Override
public void analyze(ImageProxy imageProxy, int rotationDegrees) {
Image image = imageProxy.getImage();
if(image == null){
return;
}
synchronized (mImageReaderLock) {
if(!mImageReaderLock.equals(1)){
image.close();
return;
}
if (ImageFormat.YUV_420_888 == image.getFormat()) {
Image.Plane[] planes = image.getPlanes();
if(mImageReaderSize == null){
mImageReaderSize = new Size(planes[0].getRowStride(),image.getHeight());
}
lock.lock();
if (y == null) {
y = new byte[planes[0].getBuffer().limit() - planes[0].getBuffer().position()];
u = new byte[planes[1].getBuffer().limit() - planes[1].getBuffer().position()];
v = new byte[planes[2].getBuffer().limit() - planes[2].getBuffer().position()];
}
if (image.getPlanes()[0].getBuffer().remaining() == y.length) {
planes[0].getBuffer().get(y);
planes[1].getBuffer().get(u);
planes[2].getBuffer().get(v);
if (nv21 == null) {
nv21 = new byte[planes[0].getRowStride() * image.getHeight() * 3 / 2];
}
if(nv21 != null && (nv21.length != planes[0].getRowStride() * image.getHeight() *3/2)){
return;
}
// 回传数据是YUV422
if (y.length / u.length == 2) {
ImageUtil.yuv422ToYuv420sp(y, u, v, nv21, planes[0].getRowStride(), image.getHeight());
}
// 回传数据是YUV420
else if (y.length / u.length == 4) {
nv21 = ImageUtil.yuv420ToNv21(image);
}
//调用Arcsoft算法,绘制人脸信息
drawFaceInfo(nv21,mPreviewSize);
}
lock.unlock();
}
}
}
}
04遇到的问题
1）预览变形
这个是由于设置Camera预览的size和TextureView的size比例不一致导致。
我们一般会根据当前设备屏幕的size,遍历camera支持的preview size，找到适合当前设备的预览size，再根据当前预览size，动态调整textureView的显示。
2）Arcsoft Sdk Error 异常
中间遇到的关于Arcsoft sdk error异常的，可以在Arcsoft开发中心，帮助界面，输入对应的error code，根据提示信息，可以帮助我们快速定位排查问题。
05附录
1）Demo地址：
公众号:小驰笔记，回复：人脸识别，即可获取代码下载地址。
2） Arcsoft官网sdk下载地址：
https://ai.arcsoft.com.cn/
了解更多人脸识别产品相关内容请到虹软视觉开放平台哦
标签：CameraX,人脸识别,nv21,适配,image,planes,new,getBuffer,void 来源： https://blog.csdn.net/qq_40110356/article/details/120547077