13. Android MultiMedia框架完全解析 - MediaCodec解析
作者：互联网
前言
MeidaCodec API: https://developer.android.com/reference/android/media/MediaCodec
MediaCodec是一个Codec，通过硬件加速解码和编码。它为芯片厂商和应用开发者搭建了一个统一接口。MediaCodec几乎是所有安卓播放器硬解的标配，要深入分析一个播放器的源码，如NuPlayer, ijkplayer，有必要了解其基础的使用方法。
先来看看MediaCodec在NuPlayer中的位置：
同样，我们想要深入了解MediaCodec，首先需要知道它在Android App中是如何使用的：
1. MediaCodec使用
1.1 MediaCodecList：
MediaCodec是一个统一API，支持不同编码格式，在创建MediaCodec的时候需要根据视频编码选择合适的解码器。这是通过 MediaCodec.createByCodecName 完成的。
然而不同厂商提供的解码器名称有所不同，编写通用播放器的时候，无法预见。所以Android API中提供了一个MediaCodecList用于枚举设备支持的编解码器的名字、能力，以查找合适的编解码器。
我们先枚举下设备支持的解码器：
private void displayDecoders() {
MediaCodecList list = new MediaCodecList(MediaCodecList.REGULAR_CODECS);
MediaCodecInfo[] codecs = list.getCodecInfos();
for (MediaCodecInfo codec : codecs) {
if (codec.isEncoder())
continue;
Log.i(TAG, "displayDecoders: "+codec.getName());
}
}
1.2 创建MediaCodec
所以，一种比较合理创建MediaCodec的方法是：
MediaFormat selTrackFmt = chooseVideoTrack(extractor);
codec = createCodec(selTrackFmt, surface);
private MediaFormat chooseVideoTrack(MediaExtractor extractor) {
int count = extractor.getTrackCount();
for (int i = 0; i < count; i++) {
MediaFormat format = extractor.getTrackFormat(i);
if (format.getString(MediaFormat.KEY_MIME).startsWith("video/")){
extractor.selectTrack(i);//选择轨道
return format;
}
}
return null;
}
private MediaCodec createCodec(MediaFormat format, Surface surface) throws IOException{
MediaCodecList codecList = new MediaCodecList(MediaCodecList.REGULAR_CODECS);
MediaCodec codec = MediaCodec.createByCodecName(codecList.findDecoderForFormat(format));
codec.configure(format, surface, null, 0);
return codec;
}
先使用上一篇文章中提到的MediaExtractor来选择需要的轨道，这里我们简单选择第一条视频轨道。轨道选择后，从MediaExtractor那里取到了目标轨道的MediaFormat，然后就可以通过codecList.findDecoderForFormat(format)获取到最合适的解码器了。
1.3 MediaCodec的使用方式
MediaCodec有两种使用方式 —— 同步和异步
1.3.1 同步方法
同步的主流程是在一个循环内不断调用dequeueInputBuffer -> queueInputBuffer填充数据 -> dequeueOutputBuffer -> releaseOutputBuffer显示画面
if (inIndex >= 0) {
ByteBuffer buffer = codec.getInputBuffer(inIndex);
int sampleSize = extractor.readSampleData(buffer, 0);
if (sampleSize < 0) {
codec.queueInputBuffer(inIndex, 0, 0, 0, MediaCodec.BUFFER_FLAG_END_OF_STREAM);
isEOS = true;
} else {
long sampleTime = extractor.getSampleTime();
codec.queueInputBuffer(inIndex, 0, sampleSize, sampleTime, 0);
extractor.advance();
}
}
通过MediaExtractor的readSampleData和advance我们可以从视频文件中不断取到所选轨道的未解码数据。
当sampleSize大于等于0，说明取到了数据，通过queueInputBuffer通知MediaCodec inIndex的input buffer已经准备好了。如果返回值小于0，说明已经到的文件末尾，此时可以通过BUFFER_FLAG_END_OF_STREAM标记通知MediaCodec已经达到文件末尾。
从MediaCodec取解码后的数据（实际拿到的是一个Index），一般流程是这样的：
int outIndex = codec.dequeueOutputBuffer(info, 10000);
Log.i(TAG, "run: outIndex="+outIndex);
switch (outIndex) {
case MediaCodec.INFO_OUTPUT_FORMAT_CHANGED:
Log.i(TAG, "run: new format: "+codec.getOutputFormat());
break;
case MediaCodec.INFO_TRY_AGAIN_LATER:
Log.i(TAG, "run: try later");
break;
case MediaCodec.INFO_OUTPUT_BUFFERS_CHANGED:
Log.i(TAG, "run: output buffer changed");
break;
default:
codec.releaseOutputBuffer(outIndex, true);
break;
}
if ((info.flags & MediaCodec.BUFFER_FLAG_END_OF_STREAM) != 0) {
Log.i(TAG, "OutputBuffer BUFFER_FLAG_END_OF_STREAM");
break;
}
dequeueOutputBuffer从MediaCodec中获取已经解码好的一帧的索引，再通过releaseOutputBuffer(outIndex, true)就可以让MediaCodec将这一帧输出到Surface上。(如果想控制播放速率，或丢帧，可以在这里控制)
dequeueOutputBuffer还会在特殊情况返回一些辅助值，如视频格式变化、或解码数据未准备好等。
1.3.2 异步方法
codec.setCallback(new MediaCodec.Callback() {
@Override
public void onInputBufferAvailable(MediaCodec codec, int index) {
ByteBuffer buffer = codec.getInputBuffer(index);
int sampleSize = extractor.readSampleData(buffer, 0);
if (sampleSize < 0) {
codec.queueInputBuffer(index, 0, 0, 0, MediaCodec.BUFFER_FLAG_END_OF_STREAM);
} else {
long sampleTime = extractor.getSampleTime();
codec.queueInputBuffer(index, 0, sampleSize, sampleTime, 0);
extractor.advance();
}
}
@Override
public void onOutputBufferAvailable(MediaCodec codec,
int index, MediaCodec.BufferInfo info) {
codec.releaseOutputBuffer(index, true);
}
@Override
public void one rror(MediaCodec codec, MediaCodec.CodecException e) {
Log.e(TAG, "onError: "+e.getMessage());
}
@Override
public void onOutputFormatChanged(MediaCodec codec, MediaFormat format) {
Log.i(TAG, "onOutputFormatChanged: "+format);
}
});
codec.start();
异步播放的方式是给MediaCodec注册了一个回调，由MediaCodec通知何时input buffer可用，何时output buffer可用，何时format changed等。
我们需要做的就是，当inuput buffer可用时，就把MediaExtractor提取的数据填充给指定buffer；当output buffer可用时，决定是否显示该帧（实际应用中，不会立即显示它，而是要根据fps控制显示速度）。
2. MediaCodec源码跟踪
下面是提炼出来的几行重要代码：
MediaCodec codec = MediaCodec.createByCodecName("video/avc");
MediaFormat format = MediaFormat.createVideoFormat("video/avc", 320, 480);
// format配置的代码...
codec.configure(format, surface, null, 0);
codec.start()
那我们先来看下MediaCodec.createByCodecName()方法，这个方法事实上包装一些参数后来到
private MediaCodec(@NonNull String name, boolean nameIsType, boolean encoder) {
Looper looper;
if ((looper = Looper.myLooper()) != null) {
mEventHandler = new EventHandler(this, looper);
} else if ((looper = Looper.getMainLooper()) != null) {
mEventHandler = new EventHandler(this, looper);
} else {
mEventHandler = null;
}
mCallbackHandler = mEventHandler;
mOnFrameRenderedHandler = mEventHandler;
mBufferLock = new Object();
native_setup(name, nameIsType, encoder);
}
这里我们看到我们初始化了一个Handler和一个缓冲对象的对象锁。我们接着来看native_setup()方法
static void android_media_MediaCodec_native_setup(
JNIEnv *env, jobject thiz,
jstring name, jboolean nameIsType, jboolean encoder) {
// ... 省略部分代码
const char *tmp = env->GetStringUTFChars(name, NULL);
if (tmp == NULL) {
return;
}
sp codec = new JMediaCodec(env, thiz, tmp, nameIsType, encoder);
const status_t err = codec->initCheck();
// ... 省略部分代码
codec->registerSelf();
setMediaCodec(env,thiz, codec);
}
看到JMediaCodec的构造方法
JMediaCodec::JMediaCodec(
JNIEnv *env, jobject thiz,
const char *name, bool nameIsType, bool encoder)
: mClass(NULL),
mObject(NULL) {
jclass clazz = env->GetObjectClass(thiz);
CHECK(clazz != NULL);
mClass = (jclass)env->NewGlobalRef(clazz);
mObject = env->NewWeakGlobalRef(thiz);
cacheJavaObjects(env);
mLooper = new ALooper;
mLooper->setName("MediaCodec_looper");
mLooper->start(
false, // runOnCallingThread
true, // canCallJava
PRIORITY_FOREGROUND);
if (nameIsType) {
mCodec = MediaCodec::CreateByType(mLooper, name, encoder, &mInitStatus);
} else {
mCodec = MediaCodec::CreateByComponentName(mLooper, name, &mInitStatus);
}
CHECK((mCodec != NULL) != (mInitStatus != OK));
}
这里我们看到JMediaCodec同样有一个mLooper成员变量，这个变量是一个ALooper对象，初始化mLooper后开始start()，start()方法内部会开启一个线程并run。nameIsType此时为true，来到MediaCodec::CreateByType()方法
sp MediaCodec::CreateByType(
const sp &looper, const AString &mime, bool encoder, status_t *err, pid_t pid) {
sp codec = new MediaCodec(looper, pid);
const status_t ret = codec->init(mime, true /* nameIsType */, encoder);
if (err != NULL) {
*err = ret;
}
return ret == OK ? codec : NULL; // NULL deallocates codec.
}
请记住此时传入的Looper对象是android_media_MediaCodec中JMediaCodec对象的mLooper成员变量，我们知道在Android消息队列机制中一个线程只能和一个Looper实例对应。好，我们接着看MediaCodec的构造函数：
MediaCodec::MediaCodec(const sp &looper, pid_t pid)
: mState(UNINITIALIZED),
mReleasedByResourceManager(false),
mLooper(looper),
mCodec(NULL),
mReplyID(0),
mFlags(0),
mStickyError(OK),
mSoftRenderer(NULL),
mResourceManagerClient(new ResourceManagerClient(this)),
mResourceManagerService(new ResourceManagerServiceProxy(pid)),
mBatteryStatNotified(false),
mIsVideo(false),
mVideoWidth(0),
mVideoHeight(0),
mRotationDegrees(0),
mDequeueInputTimeoutGeneration(0),
mDequeueInputReplyID(0),
mDequeueOutputTimeoutGeneration(0),
mDequeueOutputReplyID(0),
mHaveInputSurface(false),
mHavePendingInputBuffers(false) {
}
就是参数列表，先不急着一个个看，来到代码逻辑后面看MediaCodec:init()方法
status_t MediaCodec::init(const AString &name, bool nameIsType, bool encoder) {
mResourceManagerService->init();
// save init parameters for reset
mInitName = name;
mInitNameIsType = nameIsType;
mInitIsEncoder = encoder;
// Current video decoders do not return from OMX_FillThisBuffer
// quickly, violating the OpenMAX specs, until that is remedied
// we need to invest in an extra looper to free the main event
// queue.
mCodec = GetCodecBase(name, nameIsType);
if (mCodec == NULL) {
return NAME_NOT_FOUND;
}
bool secureCodec = false;
if (nameIsType && !strncasecmp(name.c_str(), "video/", 6)) {
mIsVideo = true;
} else {
AString tmp = name;
if (tmp.endsWith(".secure")) {
secureCodec = true;
tmp.erase(tmp.size() - 7, 7);
}
const sp mcl = MediaCodecList::getInstance();
if (mcl == NULL) {
mCodec = NULL; // remove the codec.
return NO_INIT; // if called from Java should raise IOException
}
ssize_t codecIdx = mcl->findCodecByName(tmp.c_str());
if (codecIdx >= 0) {
const sp info = mcl->getCodecInfo(codecIdx);
Vector mimes;
info->getSupportedMimes(&mimes);
for (size_t i = 0; i < mimes.size(); i++) {
if (mimes[i].startsWith("video/")) {
mIsVideo = true;
break;
}
}
}
}
if (mIsVideo) {
// video codec needs dedicated looper
if (mCodecLooper == NULL) {
mCodecLooper = new ALooper;
mCodecLooper->setName("CodecLooper");
mCodecLooper->start(false, false, ANDROID_PRIORITY_AUDIO);
}
mCodecLooper->registerHandler(mCodec);
} else {
mLooper->registerHandler(mCodec);
}
mLooper->registerHandler(this);
mCodec->setNotificationMessage(new AMessage(kWhatCodecNotify, this));
sp msg = new AMessage(kWhatInit, this);
msg->setString("name", name);
msg->setInt32("nameIsType", nameIsType);
if (nameIsType) {
msg->setInt32("encoder", encoder);
}
status_t err;
Vector resources;
MediaResource::Type type =
secureCodec ? MediaResource::kSecureCodec : MediaResource::kNonSecureCodec;
MediaResource::SubType subtype =
mIsVideo ? MediaResource::kVideoCodec : MediaResource::kAudioCodec;
resources.push_back(MediaResource(type, subtype, 1));
for (int i = 0; i <= kMaxRetry; ++i) {
if (i > 0) {
// Don't try to reclaim resource for the first time.
if (!mResourceManagerService->reclaimResource(resources)) {
break;
}
}
sp response;
err = PostAndAwaitResponse(msg, &response);
if (!isResourceError(err)) {
break;
}
}
return err;
}
我们看到这里用getCodecBase()方法初始化了mCodec成员变量
sp<CodecBase> MediaCodec::GetCodecBase(const AString &name, bool nameIsType) {
// at this time only ACodec specifies a mime type.
if (nameIsType || name.startsWithIgnoreCase("omx.")) {
return new ACodec;
} else if (name.startsWithIgnoreCase("android.filter.")) {
return new MediaFilter;
} else {
return NULL;
}
}
这里nameIsType为true，于是我们就会构建一个ACodec实例
ACodec::ACodec()
: mQuirks(0),
mNode(0),
mUsingNativeWindow(false),
mNativeWindowUsageBits(0),
mLastNativeWindowDataSpace(HAL_DATASPACE_UNKNOWN),
mIsVideo(false),
mIsEncoder(false),
mFatalError(false),
mShutdownInProgress(false),
mExplicitShutdown(false),
mIsLegacyVP9Decoder(false),
mEncoderDelay(0),
mEncoderPadding(0),
mRotationDegrees(0),
mChannelMaskPresent(false),
mChannelMask(0),
mDequeueCounter(0),
mInputMetadataType(kMetadataBufferTypeInvalid),
mOutputMetadataType(kMetadataBufferTypeInvalid),
mLegacyAdaptiveExperiment(false),
mMetadataBuffersToSubmit(0),
mNumUndequeuedBuffers(0),
mRepeatFrameDelayUs(-1ll),
mMaxPtsGapUs(-1ll),
mMaxFps(-1),
mTimePerFrameUs(-1ll),
mTimePerCaptureUs(-1ll),
mCreateInputBuffersSuspended(false),
mTunneled(false),
mDescribeColorAspectsIndex((OMX_INDEXTYPE)0),
mDescribeHDRStaticInfoIndex((OMX_INDEXTYPE)0) {
mUninitializedState = new UninitializedState(this);
mLoadedState = new LoadedState(this);
mLoadedToIdleState = new LoadedToIdleState(this);
mIdleToExecutingState = new IdleToExecutingState(this);
mExecutingState = new ExecutingState(this);
mOutputPortSettingsChangedState =
new OutputPortSettingsChangedState(this);
mExecutingToIdleState = new ExecutingToIdleState(this);
mIdleToLoadedState = new IdleToLoadedState(this);
mFlushingState = new FlushingState(this);
mPortEOS[kPortIndexInput] = mPortEOS[kPortIndexOutput] = false;
mInputEOSResult = OK;
memset(&mLastNativeWindowCrop, 0, sizeof(mLastNativeWindowCrop));
changeState(mUninitializedState);
}
我们先回到刚才的MediaCodec::init()方法来，看到后面的一段代码
if (mIsVideo) {
// video codec needs dedicated looper
if (mCodecLooper == NULL) {
mCodecLooper = new ALooper;
mCodecLooper->setName("CodecLooper");
mCodecLooper->start(false, false, ANDROID_PRIORITY_AUDIO);
}
mCodecLooper->registerHandler(mCodec);
} else {
mLooper->registerHandler(mCodec);
}
mLooper->registerHandler(this);
mCodec->setNotificationMessage(new AMessage(kWhatCodecNotify, this));
sp msg = new AMessage(kWhatInit, this);
msg->setString("name", name);
msg->setInt32("nameIsType", nameIsType);
if (nameIsType) {
msg->setInt32("encoder", encoder);
}
status_t err;
Vector resources;
MediaResource::Type type =
secureCodec ? MediaResource::kSecureCodec : MediaResource::kNonSecureCodec;
MediaResource::SubType subtype =
mIsVideo ? MediaResource::kVideoCodec : MediaResource::kAudioCodec;
resources.push_back(MediaResource(type, subtype, 1));
for (int i = 0; i <= kMaxRetry; ++i) {
if (i > 0) {
// Don't try to reclaim resource for the first time.
if (!mResourceManagerService->reclaimResource(resources)) {
break;
}
}
sp response;
err = PostAndAwaitResponse(msg, &response);
if (!isResourceError(err)) {
break;
}
}
在这里我们看到init()方法发送了一条kWhatInit的Msg，我们直接来看对应的回调MediaCodec::onMessageReceived()方法：
case kWhatInit:
{
sp replyID;
CHECK(msg->senderAwaitsResponse(&replyID));
if (mState != UNINITIALIZED) {
PostReplyWithError(replyID, INVALID_OPERATION);
break;
}
mReplyID = replyID;
setState(INITIALIZING);
AString name;
CHECK(msg->findString("name", &name));
int32_t nameIsType;
int32_t encoder = false;
CHECK(msg->findInt32("nameIsType", &nameIsType));
if (nameIsType) {
CHECK(msg->findInt32("encoder", &encoder));
}
sp format = new AMessage;
if (nameIsType) {
format->setString("mime", name.c_str());
format->setInt32("encoder", encoder);
} else {
format->setString("componentName", name.c_str());
}
mCodec->initiateAllocateComponent(format);
break;
}
主要是改变内部状态为INITIALIZING，同时调用了mCodec(此时为ACodec)的initiateAllocateComponent()方法：
void ACodec::initiateAllocateComponent(const sp &msg) {
msg->setWhat(kWhatAllocateComponent);
msg->setTarget(this);
msg->post();
}
同样的按照消息队列机制找到回调
ACodec::UninitializedState::onMessageReceived(const sp &msg)
case ACodec::kWhatAllocateComponent:
{
onAllocateComponent(msg);
handled = true;
break;
}
ACodec::UninitializedState::onAllocateComponent()
bool ACodec::UninitializedState::onAllocateComponent(const sp &msg) {
ALOGV("onAllocateComponent");
CHECK(mCodec->mNode == 0);
OMXClient client;
if (client.connect() != OK) {
mCodec->signalError(OMX_ErrorUndefined, NO_INIT);
return false;
}
sp<IOMX> omx = client.interface();
sp<AMessage> notify = new AMessage(kWhatOMXDied, mCodec);
Vector matchingCodecs;
AString mime;
AString componentName;
uint32_t quirks = 0;
int32_t encoder = false;
if (msg->findString("componentName", &componentName)) {
sp<IMediaCodecList> list = MediaCodecList::getInstance();
if (list != NULL && list->findCodecByName(componentName.c_str()) >= 0) {
matchingCodecs.add(componentName);
}
} else {
CHECK(msg->findString("mime", &mime));
if (!msg->findInt32("encoder", &encoder)) {
encoder = false;
}
MediaCodecList::findMatchingCodecs(
mime.c_str(),
encoder, // createEncoder
0, // flags
&matchingCodecs);
}
sp<CodecObserver> observer = new CodecObserver;
IOMX::node_id node = 0;
status_t err = NAME_NOT_FOUND;
for (size_t matchIndex = 0; matchIndex < matchingCodecs.size();
++matchIndex) {
componentName = matchingCodecs[matchIndex];
quirks = MediaCodecList::getQuirksFor(componentName.c_str());
pid_t tid = gettid();
int prevPriority = androidGetThreadPriority(tid);
androidSetThreadPriority(tid, ANDROID_PRIORITY_FOREGROUND);
err = omx->allocateNode(componentName.c_str(), observer, &mCodec->mNodeBinder, &node);
androidSetThreadPriority(tid, prevPriority);
if (err == OK) {
break;
} else {
ALOGW("Allocating component '%s' failed, try next one.", componentName.c_str());
}
node = 0;
}
if (node == 0) {
if (!mime.empty()) {
ALOGE("Unable to instantiate a %scoder for type '%s' with err %#x.",
encoder ? "en" : "de", mime.c_str(), err);
} else {
ALOGE("Unable to instantiate codec '%s' with err %#x.", componentName.c_str(), err);
}
mCodec->signalError((OMX_ERRORTYPE)err, makeNoSideEffectStatus(err));
return false;
}
mDeathNotifier = new DeathNotifier(notify);
if (mCodec->mNodeBinder == NULL ||
mCodec->mNodeBinder->linkToDeath(mDeathNotifier) != OK) {
// This was a local binder, if it dies so do we, we won't care
// about any notifications in the afterlife.
mDeathNotifier.clear();
}
notify = new AMessage(kWhatOMXMessageList, mCodec);
observer->setNotificationMessage(notify);
mCodec->mComponentName = componentName;
mCodec->mRenderTracker.setComponentName(componentName);
mCodec->mFlags = 0;
if (componentName.endsWith(".secure")) {
mCodec->mFlags |= kFlagIsSecure;
mCodec->mFlags |= kFlagIsGrallocUsageProtected;
mCodec->mFlags |= kFlagPushBlankBuffersToNativeWindowOnShutdown;
}
mCodec->mQuirks = quirks;
mCodec->mOMX = omx;
mCodec->mNode = node;
{
sp notify = mCodec->mNotify->dup();
notify->setInt32("what", CodecBase::kWhatComponentAllocated);
notify->setString("componentName", mCodec->mComponentName.c_str());
notify->post();
}
mCodec->changeState(mCodec->mLoadedState);
return true;
}
这段代码中，我们看到mCodec的mOMX被初始化，而omx对象的初始化过程非常像是C/S架构，那么我们这里插入一段，介绍下Android 中OMX的实现手段了
2.2 OMX创建
先看OMXClient::connect()方法
status_t OMXClient::connect() {
sp<IServiceManager> sm = defaultServiceManager();
sp<IBinder> playerbinder = sm->getService(String16("media.player"));
sp<IMediaPlayerService> mediaservice = interface_cast(playerbinder);
if (mediaservice.get() == NULL) {
ALOGE("Cannot obtain IMediaPlayerService");
return NO_INIT;
}
sp<IOMX> mediaServerOMX = mediaservice->getOMX();
if (mediaServerOMX.get() == NULL) {
ALOGE("Cannot obtain mediaserver IOMX");
return NO_INIT;
}
// If we don't want to use the codec process, and the media server OMX
// is local, use it directly instead of going through MuxOMX
if (!sCodecProcessEnabled &&
mediaServerOMX->livesLocally(0 /* node */, getpid())) {
mOMX = mediaServerOMX;
return OK;
}
sp<IBinder> codecbinder = sm->getService(String16("media.codec"));
sp<IMediaCodecService> codecservice = interface_cast(codecbinder);
if (codecservice.get() == NULL) {
ALOGE("Cannot obtain IMediaCodecService");
return NO_INIT;
}
sp<IOMX> mediaCodecOMX = codecservice->getOMX();
if (mediaCodecOMX.get() == NULL) {
ALOGE("Cannot obtain mediacodec IOMX");
return NO_INIT;
}
mOMX = new MuxOMX(mediaServerOMX, mediaCodecOMX);
return OK;
}
这里毫无疑问是做一次IPC工作，函数通过binder机制获得到MediaPlayerService，然后通过MediaPlayerService来创建OMX的实例。这样OMXClient就获得到了OMX的入口，接下来就可以通过binder机制来获得OMX提供的服务。也就是说OMXClient 是Android中 openmax 的入口。
可以看到这里的mediaServerOMX, mediaCodecOMX都是Binder通信的代理对象，在AOSP官网上对与OMX的支持描述如下:
Application Framework At the application framework level is application code that utilizesandroid.media APIs to interact with the multimedia hardware. Binder IPC The Binder IPC proxies facilitate communication over process boundaries. They are located in the frameworks/av/media/libmedia directory and begin with the letter “I”. Native Multimedia Framework At the native level, Android provides a multimedia framework that utilizes the Stagefright engine for audio and video recording and playback. Stagefright comes with a default list of supported software codecs and you can implement your own hardware codec by using the OpenMax integration layer standard. For more implementation details, see the MediaPlayer and Stagefright components located in frameworks/av/media. OpenMAX Integration Layer (IL) The OpenMAX IL provides a standardized way for Stagefright to recognize and use custom hardware-based multimedia codecs called components. You must provide an OpenMAX plugin in the form of a shared library namedlibstagefrighthw.so. This plugin links Stagefright with your custom codec components, which must be implemented according to the OpenMAX IL component standard.
应用程序框架在应用程序框架级别是利用android.media API 与多媒体硬件交互的应用程序代码。
Binder IPC 代理有助于跨进程边界的通信。它们位于 frameworks/av/media/libmedia 目录中，并以字母“I”开头。
原生多媒体框架 在Native层面，Android 提供了一个多媒体框架，利用 Stagefright 引擎进行音视频录制和播放。
Stagefright 附带支持的软件编解码器的默认列表，您可以使用 OpenMax 集成层标准实现自己的硬件编解码器。
有关更多实现细节，请参阅 frameworks/av/media 中的 MediaPlayer 和 Stagefright 组件。
OpenMAX 集成层 (IL) OpenMAX IL 为 Stagefright 提供了一种标准化的方式来识别和使用称为组件的基于硬件的自定义多媒体编解码器。您必须以名为libstagefrighthw.so 的共享库的形式提供OpenMAX 插件。
此插件将 Stagefright 与您的自定义编解码器组件联系起来，这些组件必须根据 OpenMAX IL 组件标准实现。
我们可以猜测，在Server端真正实现功能时，肯定是和这个 libstagefrighthw.so动态链接库有关。往后看的时候我们再来分析这个东西。这里获取OMX对象是整个MediaCodec功能实现的核心点，后面在configure方法的分析时会重点看这个对象的获取过程。
分析完这个函数后，我们再去看ACodec的构造函数中的一个函数:changeState()
这个函数的实现在frameworks/av/media/libstagefright/foundation/AHierarchicalStateMachine.cpp中
void AHierarchicalStateMachine::changeState(const sp &state) {
if (state == mState) {
// Quick exit for the easy case.
return;
}
Vector > A;
sp cur = mState;
for (;;) {
A.push(cur);
if (cur == NULL) {
break;
}
cur = cur->parentState();
}
Vector > B;
cur = state;
for (;;) {
B.push(cur);
if (cur == NULL) {
break;
}
cur = cur->parentState();
}
// Remove the common tail.
while (A.size() > 0 && B.size() > 0 && A.top() == B.top()) {
A.pop();
B.pop();
}
mState = state;
for (size_t i = 0; i < A.size(); ++i) {
A.editItemAt(i)->stateExited();
}
for (size_t i = B.size(); i > 0;) {
i--;
B.editItemAt(i)->stateEntered();
}
}
同时我们看到ACodec.h中
struct ACodec : public AHierarchicalStateMachine, public CodecBase {
// .....
// AHierarchicalStateMachine implements the message handling
virtual void onMessageReceived(const sp &msg) {
handleMessage(msg);
}
}
CodecBase.h中
struct CodecBase : public AHandler, /* static */ ColorUtils {
}
AHierarchicalStateMachine.cpp中
void AHierarchicalStateMachine::handleMessage(const sp &msg) {
sp save = mState;
sp cur = mState;
while (cur != NULL && !cur->onMessageReceived(msg)) {
// If you claim not to have handled the message you shouldn't
// have called setState...
CHECK(save == mState);
cur = cur->parentState();
}
if (cur != NULL) {
return;
}
ALOGW("Warning message %s unhandled in root state.",
msg->debugString().c_str());
}
也就是说，从Handler传给ACodec的Message经过上面的逻辑都会转发到当前ACodec状态机上状态链上每一个AState::onMessageReceived()方法上，这个很关键，会对后面的MediaCodec#start(), stop(), release()等都有关系。
也就是说，Acodec内部事实上是一个维护了由AState子类构建的状态机，我们现在已经在构造函数中看到了它changeState()到了一个mUninitializedState状态。
从刚才的changeState()方法可以看到，如果状态相同的会被直接移去，而状态不同的状态链，原有的状态链会被逐个调用stateExited()，现在新加入的状态链会翻转过来调用stateEntered()方法，我们接着就来看这个UninitializedState::stateEntered()做了什么：
void ACodec::UninitializedState::stateEntered() {
ALOGV("Now uninitialized");
if (mDeathNotifier != NULL) {
mCodec->mNodeBinder->unlinkToDeath(mDeathNotifier);
mDeathNotifier.clear();
}
mCodec->mUsingNativeWindow = false;
mCodec->mNativeWindow.clear();
mCodec->mNativeWindowUsageBits = 0;
mCodec->mNode = 0;
mCodec->mOMX.clear();
mCodec->mQuirks = 0;
mCodec->mFlags = 0;
mCodec->mInputMetadataType = kMetadataBufferTypeInvalid;
mCodec->mOutputMetadataType = kMetadataBufferTypeInvalid;
mCodec->mConverter[0].clear();
mCodec->mConverter[1].clear();
mCodec->mComponentName.clear();
}
其实就是初始化工作,到这里MediaCodec的构造过程基本上就分析完了，其中省略了大量关于Android消息机制native的逻辑，这一块可以参考一下网上很多的文章，而且这也不是这篇文章的重点。
标签：13,mCodec,MultiMedia,msg,codec,MediaCodec,new,解析,sp 来源： https://blog.csdn.net/dfvbrtdhy/article/details/118915330