编程语言
首页 > 编程语言> > webrtc源码分析(9)-拥塞控制(下)-码率分配

webrtc源码分析(9)-拥塞控制(下)-码率分配

作者：互联网

1. 前言

本文是webrtc拥塞控制的下文，主要介绍的是从cc-controller获取码率之后，如何将码率设置到PacingController控制发送速率，同时如何将码率分配调整到各个stream，各个stream的layer, simulcast，fec中

2. 正文

2.1 整体码控结构

webrtc中是会同时存在多个stream，但所有的stream都会共用一个码率预估和平滑发送，这很符合逻辑(虽然gcc保障带宽公平性)，发送数据的流程如上图数字所标，不同的stream最终发包的时候都是发送到RtpControllerSend的PacketSender中，然后在PacingController进行码率控制发送，发送包经过PacketRouter路由到其所属的RTP_RTCP进行缓存记录以便重传，然后通过RtpSenderEgress将包发送到Transport层，由Transport做Dtls，通过ice发到网络。

在拥塞控制(上)中已经介绍完了整个CcController通过cc-feedback计算目标码率，由cc-controller计算出来的码率最后会被放到PacingController中控制发送速度，同时计算出来的码率也会由RtpControllerSend通知观察者回调至上层做码率调整

整个调节过程如上图所示，会将cc-controller估算出来的码率分配给所有的streams，每个stream有设置好的最小码率(min_bitrate)和最大码率(max_bitrate)通过它们进行分配，本文没有介绍音频部分的分配和处理，只介绍了视频部分的，对于每个VideoStream，码率有两个部分构成，一个是media(媒体)，一个是Protection(抗丢包)

media(媒体码率)由Overhead(Rtp头，UDP头开销), Packetization(编码媒体帧封装开销)，Encoder(编码码率)构成；

而Encoder(编码码率)由可以根据simulcast进行细分，每个simulcast下还有不同的Temporal

Protection(抗丢包码率)主要由Fec码率和Rtx(nack重传码率)构成

下面逐渐对以上内容进行细述

2.2 RtpTransportController 反馈码率结果

在拥塞控制(上)中已经介绍完了整个CcController通过cc-feedback计算目标码率，RtpTransportControllerSend从CcController获取目标码率后会使用PostUpdates()进行更新

void RtpTransportControllerSend::OnTransportFeedback(
    const rtcp::TransportFeedback& feedback) {
  feedback_demuxer_.OnTransportFeedback(feedback);
  auto feedback_time = Timestamp::Millis(clock_->TimeInMilliseconds());
  task_queue_.PostTask([this, feedback, feedback_time]() {
    RTC_DCHECK_RUN_ON(&task_queue_);
    absl::optional<TransportPacketsFeedback> feedback_msg =
        transport_feedback_adapter_.ProcessTransportFeedback(feedback,
                                                             feedback_time);
    if (feedback_msg && controller_) {
      //从cc-controller中获取目标码率进行设置
      PostUpdates(controller_->OnTransportPacketsFeedback(*feedback_msg));
    }
    pacer()->UpdateOutstandingData(
        transport_feedback_adapter_.GetOutstandingData());
  });
}

PostUpdates()的过程如下所示:

void RtpTransportControllerSend::PostUpdates(NetworkControlUpdate update) {
  if (update.congestion_window) {
    // 设置拥塞窗口大小
    pacer()->SetCongestionWindow(*update.congestion_window);
  }
  if (update.pacer_config) {
    // 设置平滑发送速率
    pacer()->SetPacingRates(update.pacer_config->data_rate(),
                            update.pacer_config->pad_rate());
  }
  for (const auto& probe : update.probe_cluster_configs) {
    // 按照probe_cluster_config生成探测簇
    pacer()->CreateProbeCluster(probe.target_data_rate, probe.id);
  }
  if (update.target_rate) {
    // 目标码率更新了，生成目标码率
    control_handler_->SetTargetRate(*update.target_rate);
    // 更新码率分配
    UpdateControlState();
  }
}

RtpTransportControllerSend::PostUpdates()中:

2.3 PacingController 控制发送码率

2.3.1 拥塞窗口

PacingController使用了一个拥塞窗口来辅助平滑发送，这个拥塞窗口的值在拥塞控制(上)-2.8中提到过，其内涵是一个使用目标码率 * rtt计算的值。

PacingController使用该拥塞窗口来控制一个rtt内的数据发送，当检测到当前的发送中的数据(已发送但是未得到ack的数据)大于该窗口值得时候，就处于一个congested状态。

bool PacingController::Congested() const {
  if (congestion_window_size_.IsFinite()) {
    // 发送中的数据(outstanding_data_)大于当前窗口值，当前处于拥塞状态
    return outstanding_data_ >= congestion_window_size_;
  }
  return false;
}

2.3.1.1 outstanding_data的维护

发送中的数据(outstanding_data)大小的更新过程在两个地方:

outstanding_data第一个更新在底层udp socket成功发送数据的时候会执行回调OnPacketSent()通知发送了多少数据

/**
 * @description: packet成功从udp发送出去的回调，包此时未到达接收端
 * @param {packet_size} 发送的数据大小
 * @param {send_time} 发送的时间
 * @return {void}
 */
void PacingController::OnPacketSent(RtpPacketMediaType packet_type,
                                    DataSize packet_size,
                                    Timestamp send_time) {
  if (!first_sent_packet_time_) {
    first_sent_packet_time_ = send_time;
  }
  bool audio_packet = packet_type == RtpPacketMediaType::kAudio;
  if (!audio_packet || account_for_audio_) {
    // 更新已发送的数据大小
    UpdateBudgetWithSentData(packet_size);
  }
  last_send_time_ = send_time;
  last_process_time_ = send_time;
}

其中调用的UpdateBudgetWithSentData()如下：

void PacingController::UpdateBudgetWithSentData(DataSize size) {
  // 增加发送中的数据量
  outstanding_data_ += size;
  if (mode_ == ProcessMode::kPeriodic) {
    // 数据已发送,降低发送budget
    media_budget_.UseBudget(size.bytes());
    padding_budget_.UseBudget(size.bytes());
  } else {
    media_debt_ += size;
    media_debt_ = std::min(media_debt_, media_rate_ * kMaxDebtInTime);
    padding_debt_ += size;
    padding_debt_ = std::min(padding_debt_, padding_rate_ * kMaxDebtInTime);
  }
}

其中:

outstanding_data第二个更新主要是通过PacingController::UpdateOutstandingData()直接做数值上的更新的

void PacingController::UpdateOutstandingData(DataSize outstanding_data) {
  const bool was_congested = Congested();
  outstanding_data_ = outstanding_data;//直接更新
  if (was_congested && !Congested()) {
    TimeDelta elapsed_time = UpdateTimeAndGetElapsed(CurrentTime());
    UpdateBudgetWithElapsedTime(elapsed_time);
  }
}

这个调用会在socket层探测到当前网络的发生变化的时候，会直接对outstand_data重置为0

void RtpTransportControllerSend::OnNetworkRouteChanged(
	....
    task_queue_.PostTask([this, msg, network_route] {
	  ....
      // 网络路由发生了改变，将发送中的数据重设为0  
      pacer()->UpdateOutstandingData(DataSize::Zero());
    });
  }
}
    
void RtpTransportControllerSend::OnNetworkAvailability(bool network_available) {
  ....
  task_queue_.PostTask([this, msg]() {
	....
    // 网络的可用性发生改变，对发送中的数据大小进行重新初始化为0
    pacer()->UpdateOutstandingData(DataSize::Zero());
	....
  });
  ....
}

或者在收到了cc-feedback或者RR-report的时候，在这里它们它们充当了一个ack的作用，通过它们就可以将发送中的数据更新降低, 举cc-feedback为例：

void RtpTransportControllerSend::OnTransportFeedback(
    const rtcp::TransportFeedback& feedback) {
  ....
  task_queue_.PostTask([this, feedback, feedback_time]() {
	// 解析cc-feedback获得feedback_msg, 更新transport_feedback_adapter_
    absl::optional<TransportPacketsFeedback> feedback_msg =
        transport_feedback_adapter_.ProcessTransportFeedback(feedback,
                                                             feedback_time);
    // 更新发送中的数据大小
    pacer()->UpdateOutstandingData(
        transport_feedback_adapter_.GetOutstandingData());
  });
}

其中transport_feedback_adapter使用变量in_filght对发送数据进行统计维护

DataSize TransportFeedbackAdapter::GetOutstandingData() const {
  return in_flight_.GetOutstandingData(network_route_);
}

in_flight详细维护过程如下，发送数据的时候会在TransportFeedbackAdapter::ProcessSentPacket()增加in_filght的大小


absl::optional<SentPacket> TransportFeedbackAdapter::ProcessSentPacket(
    const rtc::SentPacket& sent_packet) {
  ....
  if (sent_packet.info.included_in_feedback || sent_packet.packet_id != -1) {
	  ....
      if (!packet_retransmit) {
        if (it->second.sent.sequence_number > last_ack_seq_num_)
          in_flight_.AddInFlightPacketBytes(it->second);// 新增已发送数据 
      }
    }
  }
  ....
}

收到feedback的时候会调用TransportFeedbackAdapter::ProcessTransportFeedbackInner()，对当前已经ack到的seq从in_flight中删除

TransportFeedbackAdapter::ProcessTransportFeedbackInner(
    const rtcp::TransportFeedback& feedback,
    Timestamp feedback_receive_time) { 

    for (const auto& packet : feedback.GetAllPackets()) {
        int64_t seq_num = seq_num_unwrapper_.Unwrap(packet.sequence_number());

        if (seq_num > last_ack_seq_num_) {
          // Starts at history_.begin() if last_ack_seq_num_ < 0, since any valid
          // sequence number is >= 0.
          for (auto it = history_.upper_bound(last_ack_seq_num_);
               it != history_.upper_bound(seq_num); ++it) {
            // 将上一个包到已经ack的当前包的一个in_flight删除
            in_flight_.RemoveInFlightPacketBytes(it->second);
          }
          last_ack_seq_num_ = seq_num;
        }
}

2.3.1.2 拥塞状态的使用

从congested()获取到的拥塞状态主要在PacingController控制下次发送时间的时候起作用，NextSendTime()是PacingController给PacedSender的建议下次发包时间，还未到这个时间PacedSender陷入睡眠，当到达这个时间后会让PacingController起来工作，处理队列中的包。

其中就看到了当判断到当前处于一个拥塞状态的时候，就会返回拥塞沉睡间隔，暂时不处理队列中的包。

Timestamp PacingController::NextSendTime() const {
    ....
  if (Congested() || packet_counter_ == 0) {
    // We need to at least send keep-alive packets with some interval.
    return last_send_time_ + kCongestedPacketInterval;
  }         
    ....
}

此外在发送的时候，也会判断拥塞窗口是否已经使用完毕，如果是，则不发包，详见2.3.2.2

2.3.2 PacingRate

2.3.2.1 PacingRate的更新

Pacing相关码率的更新会通过PacingController::SetPacingRates()进行

void PacingController::SetPacingRates(DataRate pacing_rate,
                                      DataRate padding_rate) {
  RTC_DCHECK_GT(pacing_rate, DataRate::Zero());
  media_rate_ = pacing_rate;
  padding_rate_ = padding_rate;
  pacing_bitrate_ = pacing_rate;
  padding_budget_.set_target_rate_kbps(padding_rate.kbps());

  RTC_LOG(LS_VERBOSE) << "bwe:pacer_updated pacing_kbps="
                      << pacing_bitrate_.kbps()
                      << " padding_budget_kbps=" << padding_rate.kbps();
}

PacingController::SetPacingRates()中主要设置的是pacing_rate和padding_rate，这两个值在GoogCcNetworkController::GetPacingRates()设置的，前者是cc-controller最终给出的目标码率 * 系数, 而后者是一个设置值，这个值还没细看不知道是否动态更新，但只有OnStreamConfig()的时候才设置了一次，感觉是应该不是动态更新

PacerConfig GoogCcNetworkController::GetPacingRates(Timestamp at_time) const {
  // Pacing rate is based on target rate before congestion window pushback,
  // because we don't want to build queues in the pacer when pushback occurs.
  // 此处的pacing rate使用的是last_loss_based_target_rate_, 这个值没有经过拥塞窗口的更新处理
  // 但是没太看懂注释，"当退避产生的时候不想在pacer创建队列"，因为pacer有两种,一种是有queue的
  // 一种是无queue的，可能想要表达的是congestion push back不应用在有queue的队列上?
  DataRate pacing_rate =
      std::max(min_total_allocated_bitrate_, last_loss_based_target_rate_) *
      pacing_factor_;
  // padding_rate 主要的值还是max_padding_rate_，这是一个来自于外部(bitrateAllocation)计算的一个值
  // 其次，它肯定不能大于窗口控制的码率(last_pushback_target_rate_)
  DataRate padding_rate =
      std::min(max_padding_rate_, last_pushback_target_rate_);
  PacerConfig msg;
  msg.at_time = at_time;
  msg.time_window = TimeDelta::Seconds(1);//1s
  msg.data_window = pacing_rate * msg.time_window;
  msg.pad_window = padding_rate * msg.time_window;
  return msg;
}

2.3.2.2 PacingRate的使用

PacingRate是用来控制发送速度的，但PacingController并没有直接使用它，PacingController中基于PacingRate为了控制方便，封装了两个类用来控制发送速度，一个是(class IntervalBudget)，一个是media_debt(DataSize)，前者使用在PacingController的Periodic模式中，在该模式下，PacingController会周期间隔(5ms)的去处理包队列，后者使用在是Dynamic模式，在该模式下，PacingController处理包队列的间隔是任意的，media_budget和media_debt的值会随着时间的流逝而变化；下面以media_budget为例介绍

PacingRate在PacingController::ProcessPackets()中被设置到media_budget上, 并在下面的while循环中使用GetPendingPacket()从队列获取包去发送的时候会使用到:

void PacingController::ProcessPackets() {
....
    if (elapsed_time > TimeDelta::Zero()) {
        // 获取pacing rate
        DataRate target_rate = pacing_bitrate_;
        
        ....

        if (mode_ == ProcessMode::kPeriodic) {
		  // 将pacing rate设置到media_budget中
          media_budget_.set_target_rate_kbps(target_rate.kbps());
          // 基于流逝的时间更新budget
          UpdateBudgetWithElapsedTime(elapsed_time);
        } else {
          media_rate_ = target_rate;
        }
      }
   ....
       
  while (!paused_) {
    ....
    // 根据budget和发送时间从queue中获取下一个要发的包
    std::unique_ptr<RtpPacketToSend> rtp_packet =
        GetPendingPacket(pacing_info, target_send_time, now);    
    ....  
  }
   ....
}

PacingController::ProcessPackets() 中

而media_budget的减少在之前的PacingController::UpdateBudgetWithSentData()中已经提到过，在 packet成功发送的时候会调用onPacketSent，则会被消耗，最终调用UpdateBudgetWithSentData()去减少

void PacingController::UpdateBudgetWithSentData(DataSize size) {
  outstanding_data_ += size;
  if (mode_ == ProcessMode::kPeriodic) {
    // 数据已发送,降低发送budget
    media_budget_.UseBudget(size.bytes());
    padding_budget_.UseBudget(size.bytes());
  } else {
    media_debt_ += size;
    media_debt_ = std::min(media_debt_, media_rate_ * kMaxDebtInTime);
    padding_debt_ += size;
    padding_debt_ = std::min(padding_debt_, padding_rate_ * kMaxDebtInTime);
  }
}

2.3.3 ProbeCluster

PacingController除了控制发送速率，同时还肩负另一个功能，支持码率探测，其使用了一个叫做BitratePorber 的类以用来响应cc-controller的探测需求，BitratePorber在拥塞控制(上)-码率预估的2.4.2已经提到过了，此处不再赘述。

PacingController部分的内容到此结束了，这是cc-controller估算出来的码率对下游最终发送上的应用，接下来介绍估算出来的码率对上游编码和fec等的影响

2.4 将码率分配到源

void RtpTransportControllerSend::PostUpdates(NetworkControlUpdate update) {
  ....
  if (update.target_rate) {
    // 通知上层
    control_handler_->SetTargetRate(*update.target_rate);
    UpdateControlState();
  }
}

在RtpTransportControllerSend::PostUpdates()的最后：

上层观察者(call)接收码率变换通知的是Call::OnTargetTransferRate()

void Call::OnTargetTransferRate(TargetTransferRate msg) {
  RTC_DCHECK_RUN_ON(send_transport_queue());

  uint32_t target_bitrate_bps = msg.target_rate.bps();
  // For controlling the rate of feedback messages.
  // 控制feedback的开销在5%左右
  receive_side_cc_.OnBitrateChanged(target_bitrate_bps);
  // 重新分配到各个stream中的码流
  bitrate_allocator_->OnNetworkEstimateChanged(msg);

  // 更新统计直方图
  worker_thread_->PostTask(
      ToQueuedTask(task_safety_, [this, target_bitrate_bps]() {
        RTC_DCHECK_RUN_ON(worker_thread_);
        last_bandwidth_bps_ = target_bitrate_bps;

        // Ignore updates if bitrate is zero (the aggregate network state is
        // down) or if we're not sending video.
        if (target_bitrate_bps == 0 || video_send_streams_.empty()) {
          estimated_send_bitrate_kbps_counter_.ProcessAndPause();
          pacer_bitrate_kbps_counter_.ProcessAndPause();
          return;
        }

        estimated_send_bitrate_kbps_counter_.Add(target_bitrate_bps / 1000);
        // Pacer bitrate may be higher than bitrate estimate if enforcing min
        // bitrate.
        uint32_t pacer_bitrate_bps =
            std::max(target_bitrate_bps, min_allocated_send_bitrate_bps_);
        pacer_bitrate_kbps_counter_.Add(pacer_bitrate_bps / 1000);
      }));
}

Call::OnTargetTransferRate()中:

2.5 BitrateAllocator 分配码率

BitrateAllocator是对码率进行重新分配的分配器，逻辑入口为BitrateAllocator::OnNetworkEstimateChanged()

其会把码率分配到不同的stream上

void BitrateAllocator::OnNetworkEstimateChanged(TargetTransferRate msg) {
  RTC_DCHECK_RUN_ON(&sequenced_checker_);
  // 获取目标码率和保守码率
  last_target_bps_ = msg.target_rate.bps();
  last_stable_target_bps_ = msg.stable_target_rate.bps();
  last_non_zero_bitrate_bps_ =
      last_target_bps_ > 0 ? last_target_bps_ : last_non_zero_bitrate_bps_;

  // loss_ration 放大255
  int loss_ratio_255 = msg.network_estimate.loss_rate_ratio * 255;
  last_fraction_loss_ =
      rtc::dchecked_cast<uint8_t>(rtc::SafeClamp(loss_ratio_255, 0, 255));
  last_rtt_ = msg.network_estimate.round_trip_time.ms();
  last_bwe_period_ms_ = msg.network_estimate.bwe_period.ms();

  // Periodically log the incoming BWE.
  int64_t now = msg.at_time.ms();
  if (now > last_bwe_log_time_ + kBweLogIntervalMs) {
    RTC_LOG(LS_INFO) << "Current BWE " << last_target_bps_;
    last_bwe_log_time_ = now;
  }

  // 按照目标码率(target_bitrate)为所有的stream分配码率
  auto allocation = AllocateBitrates(allocatable_tracks_, last_target_bps_);
  // 按照保守目标码率(stable_target)为所有的stream分配码率
  auto stable_bitrate_allocation =
      AllocateBitrates(allocatable_tracks_, last_stable_target_bps_);

  for (auto& config : allocatable_tracks_) {
    uint32_t allocated_bitrate = allocation[config.observer];
    uint32_t allocated_stable_target_rate =
        stable_bitrate_allocation[config.observer];
    BitrateAllocationUpdate update;
    update.target_bitrate = DataRate::BitsPerSec(allocated_bitrate);
    update.stable_target_bitrate =
        DataRate::BitsPerSec(allocated_stable_target_rate);
    update.packet_loss_ratio = last_fraction_loss_ / 256.0;
    update.round_trip_time = TimeDelta::Millis(last_rtt_);
    update.bwe_period = TimeDelta::Millis(last_bwe_period_ms_);
    update.cwnd_reduce_ratio = msg.cwnd_reduce_ratio;
    // 更新流上的码率
    uint32_t protection_bitrate = config.observer->OnBitrateUpdated(update);

    if (allocated_bitrate == 0 && config.allocated_bitrate_bps > 0) {
      if (last_target_bps_ > 0)
        ++num_pause_events_;
      // The protection bitrate is an estimate based on the ratio between media
      // and protection used before this observer was muted.
      uint32_t predicted_protection_bps =
          (1.0 - config.media_ratio) * config.config.min_bitrate_bps;
      RTC_LOG(LS_INFO) << "Pausing observer " << config.observer
                       << " with configured min bitrate "
                       << config.config.min_bitrate_bps
                       << " and current estimate of " << last_target_bps_
                       << " and protection bitrate "
                       << predicted_protection_bps;
    } else if (allocated_bitrate > 0 && config.allocated_bitrate_bps == 0) {
      if (last_target_bps_ > 0)
        ++num_pause_events_;
      RTC_LOG(LS_INFO) << "Resuming observer " << config.observer
                       << ", configured min bitrate "
                       << config.config.min_bitrate_bps
                       << ", current allocation " << allocated_bitrate
                       << " and protection bitrate " << protection_bitrate;
    }

    // Only update the media ratio if the observer got an allocation.
    if (allocated_bitrate > 0)
      config.media_ratio = MediaRatio(allocated_bitrate, protection_bitrate);
    config.allocated_bitrate_bps = allocated_bitrate;
  }
  // 更新limit,编码对一些特地分辨率的设置的预设值，当编码器发生调整的
  // 时候，limit是会发生改变的
  UpdateAllocationLimits();
}

BitrateAllocator::OnNetworkEstimateChanged()中:

2.5.1 AllocateBitrates()

AllocateBitrates()会将码率分配到所有的stream(audio stream, video stream)，主要是根据每个流config中的min_bitrate_bps和max_bitrate_bps进行分配。

std::map<BitrateAllocatorObserver*, int> AllocateBitrates(
    const std::vector<AllocatableTrack>& allocatable_tracks,
    uint32_t bitrate) {
  if (allocatable_tracks.empty())
    return std::map<BitrateAllocatorObserver*, int>();

  if (bitrate == 0)
    return ZeroRateAllocation(allocatable_tracks);

  // 统计所有track设定的最大最小码率
  uint32_t sum_min_bitrates = 0;
  uint32_t sum_max_bitrates = 0;
  for (const auto& observer_config : allocatable_tracks) {
    sum_min_bitrates += observer_config.config.min_bitrate_bps;
    sum_max_bitrates += observer_config.config.max_bitrate_bps;
  }

  // Not enough for all observers to get an allocation, allocate according to:
  // enforced min bitrate -> allocated bitrate previous round -> restart paused
  // streams.
  // 码率不够，按照最低分配的策略，给每个流分配码率
  if (!EnoughBitrateForAllObservers(allocatable_tracks, bitrate,
                                    sum_min_bitrates))
    return LowRateAllocation(allocatable_tracks, bitrate);

  // All observers will get their min bitrate plus a share of the rest. This
  // share is allocated to each observer based on its bitrate_priority.
  if (bitrate <= sum_max_bitrates)
    return NormalRateAllocation(allocatable_tracks, bitrate, sum_min_bitrates);

  // All observers will get up to transmission_max_bitrate_multiplier_ x max.
  return MaxRateAllocation(allocatable_tracks, bitrate, sum_max_bitrates);
}

在AllocateBitrates()中:

EnoughBitrateForAllObservers()如下：

bool EnoughBitrateForAllObservers(
    const std::vector<AllocatableTrack>& allocatable_tracks,
    uint32_t bitrate,
    uint32_t sum_min_bitrates) {
  if (bitrate < sum_min_bitrates)
    return false;

  // 计算出多余的码流可以平均摊给每个track的为多少
  uint32_t extra_bitrate_per_observer =
      (bitrate - sum_min_bitrates) /
      static_cast<uint32_t>(allocatable_tracks.size());

  // 算上多余均摊之后，无法满足每个track所需的码率(考虑上了Hysteresis即暂停滞后过所需要的增益码率),则false
  for (const auto& observer_config : allocatable_tracks) {
    if (observer_config.config.min_bitrate_bps + extra_bitrate_per_observer <
        observer_config.MinBitrateWithHysteresis()) {
      return false;
    }
  }
  return true;
}

EnoughBitrateForAllObservers()中:

码率不足的时候会调用LowRateAllocation()进行码率分配

std::map<BitrateAllocatorObserver*, int> LowRateAllocation(
    const std::vector<AllocatableTrack>& allocatable_tracks,
    uint32_t bitrate) {
  std::map<BitrateAllocatorObserver*, int> allocation;
  // Start by allocating bitrate to observers enforcing a min bitrate, hence
  // remaining_bitrate might turn negative.
  // 先每个流分配一个强制最小码率(enforce_min_bitrate)
  int64_t remaining_bitrate = bitrate;
  for (const auto& observer_config : allocatable_tracks) {
    int32_t allocated_bitrate = 0;
    if (observer_config.config.enforce_min_bitrate)
      allocated_bitrate = observer_config.config.min_bitrate_bps;

    allocation[observer_config.observer] = allocated_bitrate;
    remaining_bitrate -= allocated_bitrate;
  }

  // Allocate bitrate to all previously active streams.
  // 如果还有码率，则为活跃(不暂停)的流分配码率
  if (remaining_bitrate > 0) {
    for (const auto& observer_config : allocatable_tracks) {
      // 配置了enforce_min_bitrate或者当前处于暂停的流，先跳过
      if (observer_config.config.enforce_min_bitrate ||
          observer_config.LastAllocatedBitrate() == 0)
        continue;

      // 获取需要的码率(包含fec)
      uint32_t required_bitrate = observer_config.MinBitrateWithHysteresis();
      if (remaining_bitrate >= required_bitrate) {
        allocation[observer_config.observer] = required_bitrate;
        remaining_bitrate -= required_bitrate;
      }
    }
  }

  // Allocate bitrate to previously paused streams.
  //  如果还有码率，则为暂停的流分配码率
  if (remaining_bitrate > 0) {
    for (const auto& observer_config : allocatable_tracks) {
      if (observer_config.LastAllocatedBitrate() != 0)
        continue;

      // Add a hysteresis to avoid toggling.
      uint32_t required_bitrate = observer_config.MinBitrateWithHysteresis();
      if (remaining_bitrate >= required_bitrate) {
        allocation[observer_config.observer] = required_bitrate;
        remaining_bitrate -= required_bitrate;
      }
    }
  }

  // Split a possible remainder evenly on all streams with an allocation.
  // 如果还有剩余的码率，则均分给所有的流
  if (remaining_bitrate > 0)
    DistributeBitrateEvenly(allocatable_tracks, remaining_bitrate, false, 1,
                            &allocation);

  RTC_DCHECK_EQ(allocation.size(), allocatable_tracks.size());
  return allocation;
}

LowRateAllocation()中：

关于这个策略的第二点要非常注意，这种分配模式下将可能导致码率低的时候大部分流勉强正常工作，而某部分流持续异常，出现这种情况，可以检查一下带宽是否太低了。

码率充足的时候会调用NormalRateAllocation()进行分配

std::map<BitrateAllocatorObserver*, int> NormalRateAllocation(
    const std::vector<AllocatableTrack>& allocatable_tracks,
    uint32_t bitrate,
    uint32_t sum_min_bitrates) {
  // 为每个流分配最小满足码率，并计算剩余容量
  std::map<BitrateAllocatorObserver*, int> allocation;
  std::map<BitrateAllocatorObserver*, int> observers_capacities;
  for (const auto& observer_config : allocatable_tracks) {
    // 分配满足最小码率
    allocation[observer_config.observer] =
        observer_config.config.min_bitrate_bps;
    // 每个stream的剩余能被分配的容量
    observers_capacities[observer_config.observer] =
        observer_config.config.max_bitrate_bps -
        observer_config.config.min_bitrate_bps;
  }
  bitrate -= sum_min_bitrates;

  // TODO(srte): Implement fair sharing between prioritized streams, currently
  // they are treated on a first come first serve basis.
  // 所有的流都会有一个优先分配码率，此处是先将码率分配给有优先分配码率的流
  for (const auto& observer_config : allocatable_tracks) {
    // 计算剩余的码率
    int64_t priority_margin = observer_config.config.priority_bitrate_bps -
                              allocation[observer_config.observer];
    if (priority_margin > 0 && bitrate > 0) {
      int64_t extra_bitrate = std::min<int64_t>(priority_margin, bitrate);
      allocation[observer_config.observer] +=
          rtc::dchecked_cast<int>(extra_bitrate);
      observers_capacities[observer_config.observer] -= extra_bitrate;
      bitrate -= extra_bitrate;
    }
  }

  // From the remaining bitrate, allocate a proportional amount to each observer
  // above the min bitrate already allocated.
  // 基于各个流的优先码率比例对对剩余的码率进行分配
  if (bitrate > 0)
    DistributeBitrateRelatively(allocatable_tracks, bitrate,
                                observers_capacities, &allocation);

  return allocation;
}

NormalRateAllocation()中:

当能被分配的码率超过最大的时候则直接调用MaxRateAllocation()按照最大的分配

std::map<BitrateAllocatorObserver*, int> MaxRateAllocation(
    const std::vector<AllocatableTrack>& allocatable_tracks,
    uint32_t bitrate,
    uint32_t sum_max_bitrates) {
  std::map<BitrateAllocatorObserver*, int> allocation;

  // 按照max_bitrate_bps进行分配
  for (const auto& observer_config : allocatable_tracks) {
    allocation[observer_config.observer] =
        observer_config.config.max_bitrate_bps;
    bitrate -= observer_config.config.max_bitrate_bps;
  }
  // 将剩余得码率均分给所有流，但不超过流最大码率得2倍
  DistributeBitrateEvenly(allocatable_tracks, bitrate, true,
                          kTransmissionMaxBitrateMultiplier, &allocation);
  return allocation;
}

void DistributeBitrateEvenly(
    const std::vector<AllocatableTrack>& allocatable_tracks,
    uint32_t bitrate,
    bool include_zero_allocations,
    int max_multiplier,
    std::map<BitrateAllocatorObserver*, int>* allocation) {
  RTC_DCHECK_EQ(allocation->size(), allocatable_tracks.size());
  // 统计所有的流
  std::multimap<uint32_t, const AllocatableTrack*> list_max_bitrates;
  for (const auto& observer_config : allocatable_tracks) {
    if (include_zero_allocations ||
        allocation->at(observer_config.observer) != 0) {
      list_max_bitrates.insert(
          {observer_config.config.max_bitrate_bps, &observer_config});
    }
  }


  auto it = list_max_bitrates.begin();
  while (it != list_max_bitrates.end()) {
    RTC_DCHECK_GT(bitrate, 0);
    // 对剩余码率进行均分
    uint32_t extra_allocation =
        bitrate / static_cast<uint32_t>(list_max_bitrates.size());
    // 求得均分后得码率
    uint32_t total_allocation =
        extra_allocation + allocation->at(it->second->observer);
    bitrate -= extra_allocation;

    if (total_allocation > max_multiplier * it->first) {
      // There is more than we can fit for this observer, carry over to the
      // remaining observers.
      // 如果均分后得码率大于最大码率得两倍，则最多给某个流分配两倍得最大码率
      bitrate += total_allocation - max_multiplier * it->first;
      total_allocation = max_multiplier * it->first;
    }
    // Finally, update the allocation for this observer.
    allocation->at(it->second->observer) = total_allocation;
    it = list_max_bitrates.erase(it);
  }
}

MaxRateAllocation()中:

2.5.2 视频流码率分配更新

当从BitratteAllocator中计算出每个stream的码率之后，视频流的码率会通过VideoSendStreamImpl::OnBitrateUpdated()进行更新分配

uint32_t VideoSendStreamImpl::OnBitrateUpdated(BitrateAllocationUpdate update) {
  RTC_DCHECK_RUN_ON(worker_queue_);
  RTC_DCHECK(rtp_video_sender_->IsActive())
      << "VideoSendStream::Start has not been called.";

  // When the BWE algorithm doesn't pass a stable estimate, we'll use the
  // unstable one instead.
  if (update.stable_target_bitrate.IsZero()) {
    update.stable_target_bitrate = update.target_bitrate;
  }

  // 通过目标码率，计算编码码率，保护码率(fec, rtx等)分配
  rtp_video_sender_->OnBitrateUpdated(update, stats_proxy_->GetSendFrameRate());
  // 获取更新后的目标编码码率
  encoder_target_rate_bps_ = rtp_video_sender_->GetPayloadBitrateBps();
  // 获取更新后的目标fec码率
  const uint32_t protection_bitrate_bps =
      rtp_video_sender_->GetProtectionBitrateBps();
  // 计算link_allocation
  DataRate link_allocation = DataRate::Zero();
  if (encoder_target_rate_bps_ > protection_bitrate_bps) {
    link_allocation =
        DataRate::BitsPerSec(encoder_target_rate_bps_ - protection_bitrate_bps);//???
  }

  // 计算overhead(一些rtp头什么的)
  DataRate overhead =
      update.target_bitrate - DataRate::BitsPerSec(encoder_target_rate_bps_);
  DataRate encoder_stable_target_rate = update.stable_target_bitrate;
  if (encoder_stable_target_rate > overhead) {
    // 保守码率比overhead大，则保守编码码率为保守码率 - overhead
    encoder_stable_target_rate = encoder_stable_target_rate - overhead;
  } else {
    // 否则，保守编码码率直接设置为保守码率
    encoder_stable_target_rate = DataRate::BitsPerSec(encoder_target_rate_bps_);
  }

  // encoder_target_rate_bps_不得大于上限
  encoder_target_rate_bps_ =
      std::min(encoder_max_bitrate_bps_, encoder_target_rate_bps_);

  encoder_stable_target_rate =
      std::min(DataRate::BitsPerSec(encoder_max_bitrate_bps_),
               encoder_stable_target_rate);

  DataRate encoder_target_rate = DataRate::BitsPerSec(encoder_target_rate_bps_);
  //link_allocation不小于encoder_target_rate
  link_allocation = std::max(encoder_target_rate, link_allocation);
  // 更新编码器
  video_stream_encoder_->OnBitrateUpdated(
      encoder_target_rate, encoder_stable_target_rate, link_allocation,
      rtc::dchecked_cast<uint8_t>(update.packet_loss_ratio * 256),
      update.round_trip_time.ms(), update.cwnd_reduce_ratio);
  stats_proxy_->OnSetEncoderTargetRate(encoder_target_rate_bps_);
  return protection_bitrate_bps;
}

VideoSendStreamImpl::OnBitrateUpdated()中：

2.5.2.1 编码码率和抗丢包码率的更新

RtpVideoSender::OnBitrateUpdated()中会通过分配给当前流的目标码率和当前的帧率计算新的编码码率和抗丢包码率

void RtpVideoSender::OnBitrateUpdated(BitrateAllocationUpdate update,
                                    int framerate) {
// Substract overhead from bitrate.
// 计算simulcast流的平均overhead, overhead:Rtp扩展头 + rtp头开销
MutexLock lock(&mutex_);
size_t num_active_streams = 0;
size_t overhead_bytes_per_packet = 0;
for (const auto& stream : rtp_streams_) {
  if (stream.rtp_rtcp->SendingMedia()) {
    // 统计rtp上的扩展头+rtp头开销
    overhead_bytes_per_packet += stream.rtp_rtcp->ExpectedPerPacketOverhead();
    ++num_active_streams;
  }
}
if (num_active_streams > 1) {
  overhead_bytes_per_packet /= num_active_streams;
}

// media_packet_header + udp header
// 计算每个包的over_head
DataSize packet_overhead = DataSize::Bytes(
    overhead_bytes_per_packet + transport_overhead_bytes_per_packet_);
// 计算包的最大大小
DataSize max_total_packet_size = DataSize::Bytes(
    rtp_config_.max_packet_size + transport_overhead_bytes_per_packet_);
uint32_t payload_bitrate_bps = update.target_bitrate.bps();
if (send_side_bwe_with_overhead_ && has_packet_feedback_) {
  // 如果预估码率是包含over_head_的，则减去overhead这部分
  DataRate overhead_rate =
      CalculateOverheadRate(update.target_bitrate, max_total_packet_size,
                            packet_overhead, Frequency::Hertz(framerate));
  // TODO(srte): We probably should not accept 0 payload bitrate here.
  // 计算payload的码率
  payload_bitrate_bps = rtc::saturated_cast<uint32_t>(payload_bitrate_bps -
                                                      overhead_rate.bps());
}

// Get the encoder target rate. It is the estimated network rate -
// protection overhead.
// TODO(srte): We should multiply with 255 here.
// 此时的payload_bitrate_bps还是包括fec的，要减去fec部分才能是编码码率
// fec_controller根据目标码率和丢包率，更新fec_rate，同时计算fec需要使用的码率
// 和剩下的给编码器的码率
encoder_target_rate_bps_ = fec_controller_->UpdateFecRates(
    payload_bitrate_bps, framerate,
    rtc::saturated_cast<uint8_t>(update.packet_loss_ratio * 256),
    loss_mask_vector_, update.round_trip_time.ms());
if (!fec_allowed_) {
  // 没开启fec，直接设置为encoder_target_bitrate
  encoder_target_rate_bps_ = payload_bitrate_bps;
  // fec_controller_->UpdateFecRates() was still called so as to allow
  // |fec_controller_| to update whatever internal state it might have,
  // since |fec_allowed_| may be toggled back on at any moment.
}

  // Subtract packetization overhead from the encoder target. If target rate
  // is really low, cap the overhead at 50%. This also avoids the case where
  // |encoder_target_rate_bps_| is 0 due to encoder pause event while the
  // packetization rate is positive since packets are still flowing.
// 如果目标码率实在太低，也要把payload打包的开销压在50%以下，payload打包开销
// 主要是发生在h264和av1，h264在做聚合包的时候需要添加nal进行分割等，详情见RtpPacketizerH264::NextPacket()
// 这是为了防止在编码器暂停时仍在打包而导致的目标编码码率被设置成0的情况,
uint32_t packetization_rate_bps =
    std::min(GetPacketizationOverheadRate(), encoder_target_rate_bps_ / 2);
// 编码码率减去rtp打包头
encoder_target_rate_bps_ -= packetization_rate_bps;

loss_mask_vector_.clear();

uint32_t encoder_overhead_rate_bps = 0;
if (send_side_bwe_with_overhead_ && has_packet_feedback_) {
  // TODO(srte): The packet size should probably be the same as in the
  // CalculateOverheadRate call above (just max_total_packet_size), it doesn't
  // make sense to use different packet rates for different overhead
  // calculations.
  // 通过encoder rate重新计算overhead
  DataRate encoder_overhead_rate = CalculateOverheadRate(
      DataRate::BitsPerSec(encoder_target_rate_bps_),
      max_total_packet_size - DataSize::Bytes(overhead_bytes_per_packet),
      packet_overhead, Frequency::Hertz(framerate));
  encoder_overhead_rate_bps = std::min(
      encoder_overhead_rate.bps<uint32_t>(),
      update.target_bitrate.bps<uint32_t>() - encoder_target_rate_bps_);
}
// When the field trial "WebRTC-SendSideBwe-WithOverhead" is enabled
// protection_bitrate includes overhead.
const uint32_t media_rate = encoder_target_rate_bps_ +
                            encoder_overhead_rate_bps +
                            packetization_rate_bps;
RTC_DCHECK_GE(update.target_bitrate, DataRate::BitsPerSec(media_rate));
protection_bitrate_bps_ = update.target_bitrate.bps() - media_rate;
}

RtpVideoSender::OnBitrateUpdated()中:

整个过程可以总结为:

目标码率减去平均额外开销得到有效负载码率

有效负载码率经过FEC得到包含了打包的目标编码码率

打包的目标编码码率减去打包开销得到目标编码码率

基于目标编码码率和包平均大小，帧率重新计算overhead，将编码码率 + overhead + 打包开销就得到了媒体码率(media_rate)

目标码率 - 媒体码率(meida_rate) 得到保护码率(protection_bitrate_bps)

其中FecControllerDefault::UpdateFecRates()计算有效载荷码率的详情如下:

uint32_t FecControllerDefault::UpdateFecRates(
    uint32_t estimated_bitrate_bps,
    int actual_framerate_fps,
    uint8_t fraction_lost,
    std::vector<bool> loss_mask_vector,
    int64_t round_trip_time_ms) {
  float target_bitrate_kbps =
      static_cast<float>(estimated_bitrate_bps) / 1000.0f;
  // Sanity check.
  if (actual_framerate_fps < 1.0) {
    actual_framerate_fps = 1.0;
  }

  // 通过丢包率，码率等更新I帧和P帧的fec_rate
  FecProtectionParams delta_fec_params;
  FecProtectionParams key_fec_params;
  {
    MutexLock lock(&mutex_);
    loss_prot_logic_->UpdateBitRate(target_bitrate_kbps);
    loss_prot_logic_->UpdateRtt(round_trip_time_ms);
    // Update frame rate for the loss protection logic class: frame rate should
    // be the actual/sent rate.
    loss_prot_logic_->UpdateFrameRate(actual_framerate_fps);
    // Returns the filtered packet loss, used for the protection setting.
    // The filtered loss may be the received loss (no filter), or some
    // filtered value (average or max window filter).
    // Use max window filter for now.
    media_optimization::FilterPacketLossMode filter_mode =
        media_optimization::kMaxFilter;
    uint8_t packet_loss_enc = loss_prot_logic_->FilteredLoss(
        clock_->TimeInMilliseconds(), filter_mode, fraction_lost);
    // For now use the filtered loss for computing the robustness settings.
    loss_prot_logic_->UpdateFilteredLossPr(packet_loss_enc);
    if (loss_prot_logic_->SelectedType() == media_optimization::kNone) {
      return estimated_bitrate_bps;
    }
    // Update method will compute the robustness settings for the given
    // protection method and the overhead cost
    // the protection method is set by the user via SetVideoProtection.
    loss_prot_logic_->UpdateMethod();
    // Get the bit cost of protection method, based on the amount of
    // overhead data actually transmitted (including headers) the last
    // second.
    // Get the FEC code rate for Key frames (set to 0 when NA).
    key_fec_params.fec_rate =
        loss_prot_logic_->SelectedMethod()->RequiredProtectionFactorK();
    // Get the FEC code rate for Delta frames (set to 0 when NA).
    delta_fec_params.fec_rate =
        loss_prot_logic_->SelectedMethod()->RequiredProtectionFactorD();
    // The RTP module currently requires the same |max_fec_frames| for both
    // key and delta frames.
    delta_fec_params.max_fec_frames =
        loss_prot_logic_->SelectedMethod()->MaxFramesFec();
    key_fec_params.max_fec_frames =
        loss_prot_logic_->SelectedMethod()->MaxFramesFec();
  }
  // Set the FEC packet mask type. |kFecMaskBursty| is more effective for
  // consecutive losses and little/no packet re-ordering. As we currently
  // do not have feedback data on the degree of correlated losses and packet
  // re-ordering, we keep default setting to |kFecMaskRandom| for now.
  delta_fec_params.fec_mask_type = kFecMaskRandom;
  key_fec_params.fec_mask_type = kFecMaskRandom;
  // Update protection callback with protection settings.
  uint32_t sent_video_rate_bps = 0;
  uint32_t sent_nack_rate_bps = 0;
  uint32_t sent_fec_rate_bps = 0;
  // Rate cost of the protection methods.
  float protection_overhead_rate = 0.0f;
  // TODO(Marco): Pass FEC protection values per layer.
  // 将nack，fec的码率更新, 
  // 获取更新后的sent_video_rate_bps(视频发送码率), nack_rate(nack码率)
  // fec_rate(fec码率)
  protection_callback_->ProtectionRequest(
      &delta_fec_params, &key_fec_params, &sent_video_rate_bps,
      &sent_nack_rate_bps, &sent_fec_rate_bps);
  // 计算总发送码率
  uint32_t sent_total_rate_bps =
      sent_video_rate_bps + sent_nack_rate_bps + sent_fec_rate_bps;
  // Estimate the overhead costs of the next second as staying the same
  // wrt the source bitrate.
  if (sent_total_rate_bps > 0) {
    // 计算protection_rate
    protection_overhead_rate =
        static_cast<float>(sent_nack_rate_bps + sent_fec_rate_bps) /
        sent_total_rate_bps;
  }
  // Cap the overhead estimate to a threshold, default is 50%.
  protection_overhead_rate =
      std::min(protection_overhead_rate, overhead_threshold_);
  // Source coding rate: total rate - protection overhead.
  // 计算media_rate
  return estimated_bitrate_bps * (1.0 - protection_overhead_rate);
}

在FecControllerDefault::UpdateFecRates()中

而使用CalculateOverheadRate()计算overhead的方式如下，原理上很简单，通过目标码率/包大小得到包数量，然后将包数量 * overhead就得到overhead了

/**
 * @description: 求overhead的每秒码率，计算方式是通过目标码率和帧率得到帧大小
 * 计算的方式很奇怪：data_rate / framerate / packet_size * framerate  * overhead_per_packet
 * 明显是可以看到中间的framerate可以消掉更符合逻辑： data_rate / packet_size *     overhead_per_packet, 难道只是为了中间的ceil()操作？
 * @param {data_rate} 分配的码率
 * @param {packet_size} 包的大小
 * @param {overhead_per_packet} 每个包的overhead
 * @param {framerate} 当前帧率
 * @return {*}
 */
DataRate RtpVideoSender::CalculateOverheadRate(DataRate data_rate,
                                               DataSize packet_size,
                                               DataSize overhead_per_packet,
                                               Frequency framerate) const {
  Frequency packet_rate = data_rate / packet_size;
  if (use_frame_rate_for_overhead_) {
    framerate = std::max(framerate, Frequency::Hertz(1));// 获取帧率
    DataSize frame_size = data_rate / framerate;        // 计算帧大小
    int packets_per_frame = ceil(frame_size / packet_size); // 计算每帧有多少包
    packet_rate = packets_per_frame * framerate; // 计算包率
  }

  return packet_rate.RoundUpTo(Frequency::Hertz(1)) * overhead_per_packet;
}

2.5.2.2 更新编码器

VideoStream的分配的码率计算出来后，编码器的会通过VideoStreamEncoder::OnBitrateUpdated()进行编码码率的分配更新，会重新调整simulcast 和 temporal layer各层级的码率，以及帧率大小。

/**
 * @description: 更新编码码率，更新当前stream下分配给各个simulcast和
 * temporal layer的码率，调整帧率
 * @param {target_bitrate} 目标编码码率
 * @param {stable_target_bitrate} 目标保守编码码率
 * @param {link_allocation} 能使用的网络通道容量
 * @param {fraction_lost} 丢包率
 * @param {round_trip_time_ms} rtt
 * @param {cwnd_reduce_ratio}
 * @return {*}
 */
void VideoStreamEncoder::OnBitrateUpdated(DataRate target_bitrate,
                                          DataRate stable_target_bitrate,
                                          DataRate link_allocation,
                                          uint8_t fraction_lost,
                                          int64_t round_trip_time_ms,
                                          double cwnd_reduce_ratio) {
  // 线程检查                                         
  RTC_DCHECK_GE(link_allocation, target_bitrate);
  if (!encoder_queue_.IsCurrent()) {
    encoder_queue_.PostTask([this, target_bitrate, stable_target_bitrate,
                             link_allocation, fraction_lost, round_trip_time_ms,
                             cwnd_reduce_ratio] {
      DataRate updated_target_bitrate =
          UpdateTargetBitrate(target_bitrate, cwnd_reduce_ratio);
      OnBitrateUpdated(updated_target_bitrate, stable_target_bitrate,
                       link_allocation, fraction_lost, round_trip_time_ms,
                       cwnd_reduce_ratio);
    });
    return;
  }
  RTC_DCHECK_RUN_ON(&encoder_queue_);

  // target_bitrate == 0 -> video suspended(暂停)
  const bool video_is_suspended = target_bitrate == DataRate::Zero();
  const bool video_suspension_changed = video_is_suspended != EncoderPaused();

  if (!video_is_suspended && settings_.encoder_switch_request_callback &&
      encoder_selector_) {
    // 竟然还能根据链路容量进行编码器的切换，但是目前没在其中看到encoder_selector的实现
    if (auto encoder = encoder_selector_->OnAvailableBitrate(link_allocation)) {
        
      QueueRequestEncoderSwitch(*encoder);
    }
  }

  RTC_DCHECK(sink_) << "sink_ must be set before the encoder is active.";

  RTC_LOG(LS_VERBOSE) << "OnBitrateUpdated, bitrate " << target_bitrate.bps()
                      << " stable bitrate = " << stable_target_bitrate.bps()
                      << " link allocation bitrate = " << link_allocation.bps()
                      << " packet loss " << static_cast<int>(fraction_lost)
                      << " rtt " << round_trip_time_ms;

  if (encoder_) {
    // 丢包率和rtt更新到encoder中并没有实际使用
    encoder_->OnPacketLossRateUpdate(static_cast<float>(fraction_lost) / 256.f);
    encoder_->OnRttUpdate(round_trip_time_ms);
  }

  uint32_t framerate_fps = GetInputFramerateFps();
  frame_dropper_.SetRates((target_bitrate.bps() + 500) / 1000, framerate_fps);

  // 根据目标码率设置到不同的layer中，然后设置到编码器中，并告诉rtp_video_sender
  EncoderRateSettings new_rate_settings{
      VideoBitrateAllocation(), static_cast<double>(framerate_fps),
      link_allocation, target_bitrate, stable_target_bitrate};
  SetEncoderRates(UpdateBitrateAllocation(new_rate_settings));

  if (target_bitrate.bps() != 0)
    encoder_target_bitrate_bps_ = target_bitrate.bps();

  stream_resource_manager_.SetTargetBitrate(target_bitrate);

  if (video_suspension_changed) {
    RTC_LOG(LS_INFO) << "Video suspend state changed to: "
                     << (video_is_suspended ? "suspended" : "not suspended");
    encoder_stats_observer_->OnSuspendChange(video_is_suspended);
  }
  if (video_suspension_changed && !video_is_suspended && pending_frame_ &&
      !DropDueToSize(pending_frame_->size())) {
    // 此时码率可能，不再suspended了, 把pending_frame放入编码队列
    int64_t pending_time_us =
        clock_->CurrentTime().us() - pending_frame_post_time_us_;
    if (pending_time_us < kPendingFrameTimeoutMs * 1000)
      EncodeVideoFrame(*pending_frame_, pending_frame_post_time_us_);
    pending_frame_.reset();
  }
}

VideoStreamEncoder::OnBitrateUpdated()中:

其中，将码率分配到不同的simulcast和temporal layer的UpdateBitrateAllocation()见下：

VideoStreamEncoder::EncoderRateSettings
VideoStreamEncoder::UpdateBitrateAllocation(
    const EncoderRateSettings& rate_settings) {
  VideoBitrateAllocation new_allocation;
  // Only call allocators if bitrate > 0 (ie, not suspended), otherwise they
  // might cap the bitrate to the min bitrate configured.
  // 通过rate_allocator重新分配码率
  if (rate_allocator_ && rate_settings.encoder_target > DataRate::Zero()) {
    // 重新计算分配到simulcast和temporal下的码率
    // -> SimulcastRateAllocator::Allocate()
    new_allocation = rate_allocator_->Allocate(VideoBitrateAllocationParameters(
        rate_settings.encoder_target, rate_settings.stable_encoder_target,
        rate_settings.rate_control.framerate_fps));
  }

  EncoderRateSettings new_rate_settings = rate_settings;
  new_rate_settings.rate_control.target_bitrate = new_allocation;
  new_rate_settings.rate_control.bitrate = new_allocation;
  // VideoBitrateAllocator subclasses may allocate a bitrate higher than the
  // target in order to sustain the min bitrate of the video codec. In this
  // case, make sure the bandwidth allocation is at least equal the allocation
  // as that is part of the document contract for that field.
  new_rate_settings.rate_control.bandwidth_allocation =
      std::max(new_rate_settings.rate_control.bandwidth_allocation,
               DataRate::BitsPerSec(
                   new_rate_settings.rate_control.bitrate.get_sum_bps()));

  if (bitrate_adjuster_) {
    // 对码率进行再调节，以平滑码率调整的过程
    VideoBitrateAllocation adjusted_allocation =
        bitrate_adjuster_->AdjustRateAllocation(new_rate_settings.rate_control);
    RTC_LOG(LS_VERBOSE) << "Adjusting allocation, fps = "
                        << rate_settings.rate_control.framerate_fps << ", from "
                        << new_allocation.ToString() << ", to "
                        << adjusted_allocation.ToString();
    new_rate_settings.rate_control.bitrate = adjusted_allocation;
  }

  return new_rate_settings;
}

UpdateBitrateAllocation()中:

用以设置新的码率SetEncoderRates()如下:

void VideoStreamEncoder::SetEncoderRates(
    const EncoderRateSettings& rate_settings) {
  RTC_DCHECK_GT(rate_settings.rate_control.framerate_fps, 0.0);

  // 检测(fps, bitrate)是否发生改变
  bool rate_control_changed =
      (!last_encoder_rate_settings_.has_value() ||
       last_encoder_rate_settings_->rate_control != rate_settings.rate_control);

  
  // For layer allocation signal we care only about the target bitrate (not the
  // adjusted one) and the target fps.
  // 对于layer, 只是考虑target_bitrate和framerate_fps
  bool layer_allocation_changed =
      !last_encoder_rate_settings_.has_value() ||
      last_encoder_rate_settings_->rate_control.target_bitrate !=
          rate_settings.rate_control.target_bitrate ||
      last_encoder_rate_settings_->rate_control.framerate_fps !=
          rate_settings.rate_control.framerate_fps;

  if (last_encoder_rate_settings_ != rate_settings) {
    last_encoder_rate_settings_ = rate_settings;
  }

  if (!encoder_) {
    return;
  }

  // |bitrate_allocation| is 0 it means that the network is down or the send
  // pacer is full. We currently only report this if the encoder has an internal
  // source. If the encoder does not have an internal source, higher levels
  // are expected to not call AddVideoFrame. We do this since its unclear
  // how current encoder implementations behave when given a zero target
  // bitrate.
  // TODO(perkj): Make sure all known encoder implementations handle zero
  // target bitrate and remove this check.
  if (!HasInternalSource() &&
      rate_settings.rate_control.bitrate.get_sum_bps() == 0) {
    return;
  }

  //fps和bitrate发生了改变，告知编码器
  if (rate_control_changed) {
    // EncoderSimulcastProxy::SetRates()
    // ->H264EncoderImpl::SetRates()
    encoder_->SetRates(rate_settings.rate_control);

    encoder_stats_observer_->OnBitrateAllocationUpdated(
        send_codec_, rate_settings.rate_control.bitrate);
    frame_encode_metadata_writer_.OnSetRates(
        rate_settings.rate_control.bitrate,
        static_cast<uint32_t>(rate_settings.rate_control.framerate_fps + 0.5));
    stream_resource_manager_.SetEncoderRates(rate_settings.rate_control);
    // 不同layer的流的码率发生了变化，告知rtp_rtcp，在发送关键帧的时候，通过extension把最新的layer
    // 分辨率带上，详见RtpSenderVideo::SetVideoLayersAllocation()和RtpSenderVideo::SetVideoLayersAllocationInternal()
    if (layer_allocation_changed &&
        allocation_cb_type_ ==
            BitrateAllocationCallbackType::kVideoLayersAllocation) {
      sink_->OnVideoLayersAllocationUpdated(CreateVideoLayersAllocation(
          send_codec_, rate_settings.rate_control, encoder_->GetEncoderInfo()));
    }
  }

  
  // 通知rtp_video_sender，由它通知rtp_rtcp等模块，码率是否发生了改变
  if ((allocation_cb_type_ ==
       BitrateAllocationCallbackType::kVideoBitrateAllocation) ||
      (encoder_config_.content_type ==
           VideoEncoderConfig::ContentType::kScreen &&
       allocation_cb_type_ == BitrateAllocationCallbackType::
                                  kVideoBitrateAllocationWhenScreenSharing)) {
    // rtp_rtcp检测到layer发生了变化后
    sink_->OnBitrateAllocationUpdated(
        // Update allocation according to info from encoder. An encoder may
        // choose to not use all layers due to for example HW.
        UpdateAllocationFromEncoderInfo(
            rate_settings.rate_control.target_bitrate,
            encoder_->GetEncoderInfo()));
  }
}

VideoStreamEncoder::SetEncoderRates()中：

2.5.3 UpdateAllocationLimits()

BitrateAllocator::OnNetworkEstimateChanged()中将码率分发各个视频流中并更新编码器码率后，stream的limit(min_allocatable_rate, max_padding_rate,max_allocatable_rate)会发生改变，此时需要将变更后的limit重新统计，告知cc-controller变更后的limit，调用的是BitrateAllocator::UpdateAllocationLimits()

void BitrateAllocator::UpdateAllocationLimits() {
  BitrateAllocationLimits limits;
  for (const auto& config : allocatable_tracks_) {
    uint32_t stream_padding = config.config.pad_up_bitrate_bps;
    if (config.config.enforce_min_bitrate) {
      limits.min_allocatable_rate +=
          DataRate::BitsPerSec(config.config.min_bitrate_bps);
    } else if (config.allocated_bitrate_bps == 0) {
      stream_padding =
          std::max(config.MinBitrateWithHysteresis(), stream_padding);
    }
    limits.max_padding_rate += DataRate::BitsPerSec(stream_padding);
    limits.max_allocatable_rate +=
        DataRate::BitsPerSec(config.config.max_bitrate_bps);
  }

  if (limits.min_allocatable_rate == current_limits_.min_allocatable_rate &&
      limits.max_allocatable_rate == current_limits_.max_allocatable_rate &&
      limits.max_padding_rate == current_limits_.max_padding_rate) {
    return;
  }
  current_limits_ = limits;

  RTC_LOG(LS_INFO) << "UpdateAllocationLimits : total_requested_min_bitrate: "
                   << ToString(limits.min_allocatable_rate)
                   << ", total_requested_padding_bitrate: "
                   << ToString(limits.max_padding_rate)
                   << ", total_requested_max_bitrate: "
                   << ToString(limits.max_allocatable_rate);

  limit_observer_->OnAllocationLimitsChanged(limits);
}

其中limit_observer_->OnAllocationLimitsChanged()会通过 RtpTransportControllerSend::UpdateStreamsConfig()会调用到cc-controller中

void RtpTransportControllerSend::UpdateStreamsConfig() {
  streams_config_.at_time = Timestamp::Millis(clock_->TimeInMilliseconds());
  if (controller_)
    PostUpdates(controller_->OnStreamsConfig(streams_config_));
}


NetworkControlUpdate GoogCcNetworkController::OnStreamsConfig(
    StreamsConfig msg) {
  NetworkControlUpdate update;
  if (msg.requests_alr_probing) {
    probe_controller_->EnablePeriodicAlrProbing(*msg.requests_alr_probing);
  }
  if (msg.max_total_allocated_bitrate &&
      *msg.max_total_allocated_bitrate != max_total_allocated_bitrate_) {
    if (rate_control_settings_.TriggerProbeOnMaxAllocatedBitrateChange()) {
      // 重设probe_controller的max_bitrate
      update.probe_cluster_configs =
          probe_controller_->OnMaxTotalAllocatedBitrate(
              msg.max_total_allocated_bitrate->bps(), msg.at_time.ms());
    } else {
      probe_controller_->SetMaxBitrate(msg.max_total_allocated_bitrate->bps());
    }
    max_total_allocated_bitrate_ = *msg.max_total_allocated_bitrate;
  }
  bool pacing_changed = false;
  if (msg.pacing_factor && *msg.pacing_factor != pacing_factor_) {
    pacing_factor_ = *msg.pacing_factor;
    pacing_changed = true;
  }
  if (msg.min_total_allocated_bitrate &&
      *msg.min_total_allocated_bitrate != min_total_allocated_bitrate_) {
    min_total_allocated_bitrate_ = *msg.min_total_allocated_bitrate;
    pacing_changed = true;

    if (use_min_allocatable_as_lower_bound_) {
      ClampConstraints();
      // 重设delay_based_bwe和bandwidth_estimation的limit
      delay_based_bwe_->SetMinBitrate(min_data_rate_);
      bandwidth_estimation_->SetMinMaxBitrate(min_data_rate_, max_data_rate_);
    }
  }
  if (msg.max_padding_rate && *msg.max_padding_rate != max_padding_rate_) {
    max_padding_rate_ = *msg.max_padding_rate;
    pacing_changed = true;
  }

  if (pacing_changed)
    update.pacer_config = GetPacingRates(msg.at_time);
  return update;
}

在GoogCcNetworkController::OnStreamsConfig()中:

3. Ref

标签：target,码率,rate,encoder,bps,源码,bitrate,webrtc
来源： https://www.cnblogs.com/ishen/p/15333270.html