WebRTC 系列（二、本地 demo，H5、Android、iOS）

WebRTC 系列（一、简介）​​​​​​​ 一、整体流程 有了上一篇 WebRTC 简介的基础，我们知道了 WebRTC 的工作流程，接下来就是需要用代码去实现这个流程了。对于不同端，实现起来的难易程度可能略微不同（实践中我感觉 iOS 端是资料最少的），但是整体流程是一样的。 问：要把大象装冰箱，总共分几步？答：三步。 0 Comments

WebRTC 系列（一、简介）​​​​​​​

一、整体流程

有了上一篇 WebRTC 简介的基础，我们知道了 WebRTC 的工作流程，接下来就是需要用代码去实现这个流程了。对于不同端，实现起来的难易程度可能略微不同（实践中我感觉 iOS 端是资料最少的），但是整体流程是一样的。

问：要把大象装冰箱，总共分几步？答：三步。

  1. 初始化；
  2. 发起呼叫，接收应答；
  3. 挂断，释放资源。

初始化

  1. 初始化 PeerConnectionFactory；
  2. 创建 EglBase；
  3. 创建 PeerConnectionFactory；
  4. 创建音轨；
  5. 创建视轨；
  6. 初始化本地视频渲染控件；
  7. 初始化远端视频渲染控件；
  8. 开始本地渲染。

这些步骤，不同端不一定都需要，有的端的 WebRTC 依赖库中已经做了一些，H5 的初始化操作是最少的。有的步骤的顺序可能会略有不同，Android 和 iOS 就有区别，需要注意的是初始化不当可能导致画面渲染不出来，而且还不易排查，后面我再说明我踩过哪些坑。

发起呼叫，接收应答

  1. A 创建 PeerConnection；
  2. A 为 PeerConnection 添加音轨、视轨；
  3. A 通过 PeerConnection 创建 offer，获取 sdp；
  4. A 将 offer sdp 作为参数 setLocalDescription；
  5. A 发送 offer sdp；
  6. B 收到 offer sdp，创建 PeerConnection；
  7. B 将 offer sdp 作为参数 setRemoteDescription；
  8. B 通过 PeerConnection 创建 answer，获取 sdp；
  9. B 将 answer sdp 作为参数 setLocalDescription；
  10. B 发送 answer sdp；
  11. A 收到 answer sdp，将 answer sdp 作为参数 setRemoteDescription。

这期间，PeerConnection 的 onIceCandidate() 这个回调方法会被调用，我们需要将生成的 IceCandidate 对象传递给对方，收到 IceCandidate 的时候我们需要通过 PeerConnection 的 addIceCandidate() 方法添加。

这期间，PeerConnection 的 onAddStream() 这个回调也会被调用，我们需要将 MediaStream 添加到远端控件渲染器中进行渲染。

挂断，释放资源

  1. 关闭 PeerConnection；
  2. 释放远端视频渲染控件。

释放资源主要是为了能够正常的再次呼叫。

整理完整体流程后，我们可以先来实现一个本地 demo，我是创建了两个 PeerConnection，一个本地 PeerConnection 作为用户 A，一个远端 PeerConnection 作为用户 B，由于大部分 Android 设备和 iOS 设备同时只能开启一个摄像头，所以远端 PeerConnection 不会采集视频数据，只会渲染远端画面。

二、H5

1.添加依赖

上一篇博客已经说到，WebRTC 已经成为 H5 的标准之一，所以 H5 实现它很容易，并不需要额外引入依赖。

2.local_demo.html

H5 的实现相对简单，很多初始化的操作我们不需要去做，浏览器已经帮我们做好了，整体代码如下：

<html><head><title>Local Demo</title><style>body {overflow: hidden;margin: 0px;padding: 0px;}#local_view {width: 100%;height: 100%;}#remote_view {width: 9%;height: 16%;position: absolute;top: 10%;right: 10%;}#left {width: 10%;height: 5%;position: absolute;left: 10%;top: 10%;}.my_button {width: 100%;height: 100%;display: block;margin-bottom: 10%;}</style>
</head><body><video id="local_view" autoplay controls muted></video><video id="remote_view" autoplay controls muted></video><div id="left"><button id="btn_call" class="my_button" onclick="call()">呼叫</button><button id="btn_hang_up" class="my_button" onclick="hangUp()">挂断</button></div>
</body><script type="text/javascript">var localView = document.getElementById("local_view");var remoteView = document.getElementById("remote_view");var localStream;let localPeerConnection;function call() {// 创建 PeerConnectionlocalPeerConnection = new RTCPeerConnection();localPeerConnection.onicecandidate = function (event) {console.log("onicecandidate--->" + event.candidate);let iceCandidate = event.candidate;if (iceCandidate == null) {return;}sendIceCandidate(localPeerConnection, iceCandidate);}// 为 PeerConnection 添加音轨、视轨for (let i = 0; localStream != null && i < localStream.getTracks().length; i++) {const track = localStream.getTracks()[i];localPeerConnection.addTrack(track, localStream);}// 通过 PeerConnection 创建 offer，获取 sdplocalPeerConnection.createOffer().then(function (sessionDescription) {console.log("create offer success.");// 将 offer sdp 作为参数 setLocalDescriptionlocalPeerConnection.setLocalDescription(sessionDescription).then(function () {console.log("set local sdp success.");// 发送 offer sdpsendOffer(sessionDescription)})})}function sendOffer(offer) {receivedOffer(offer);}function receivedOffer(offer) {// 创建 PeerConnectionremotePeerConnection = new RTCPeerConnection();remotePeerConnection.onicecandidate = function (event) {console.log("onicecandidate--->" + event.candidate);let iceCandidate = event.candidate;if (iceCandidate == null) {return;}sendIceCandidate(peerConnection, iceCandidate);}remotePeerConnection.ontrack = function (event) {console.log("ontrack--->" + event.streams);let streams = event.streams;if (streams && streams.length > 0) {remoteView.srcObject = streams[0];}}// 将 offer sdp 作为参数 setRemoteDescriptionremotePeerConnection.setRemoteDescription(offer).then(function () {console.log("set remote sdp success.");// 通过 PeerConnection 创建 answer，获取 sdpremotePeerConnection.createAnswer().then(function (sessionDescription) {console.log("create answer success.");// 将 answer sdp 作为参数 setLocalDescriptionremotePeerConnection.setLocalDescription(sessionDescription).then(function () {console.log("set local sdp success.");// 发送 answer sdpsendAnswer(sessionDescription);})})})}function sendAnswer(answer) {receivedAnswer(answer)}function receivedAnswer(answer) {// 收到 answer sdp，将 answer sdp 作为参数 setRemoteDescriptionlocalPeerConnection.setRemoteDescription(answer).then(function () {console.log("set remote sdp success.");})}function sendIceCandidate(peerConnection, iceCandidate) {receivedCandidate(peerConnection, iceCandidate);}function receivedCandidate(peerConnection, iceCandidate) {if (peerConnection == localPeerConnection) {remotePeerConnection.addIceCandidate(iceCandidate);} else {localPeerConnection.addIceCandidate(iceCandidate);}}function hangUp() {// 关闭 PeerConnectionif (localPeerConnection != null) {localPeerConnection.close();localPeerConnection = null;}if (remotePeerConnection != null) {remotePeerConnection.close();remotePeerConnection = null;}// 释放远端视频渲染控件remoteView.removeAttribute('src');remoteView.load();}navigator.mediaDevices.getUserMedia({ audio: true, video: true }).then(function (mediaStream) {// 初始化 PeerConnectionFactory// 创建 EglBase// 创建 PeerConnectionFactory// 创建音轨// 创建视轨localStream = mediaStream;// 初始化本地视频渲染控件// 初始化远端视频渲染控件// 开始本地渲染localView.srcObject = mediaStream;}).catch(function (error) {console.log("error--->" + error);})
</script></html>

运行起来后点击呼叫，在右上角的远端控件就能收到本地视频了：

三、Android

Android 端流程是最多的，但是由于我是写 Android 的，所以觉得它的流程是最严谨的，也可能是因为 WebRTC 是 Google 开源的缘故，所以 Android 端这方面的资料还是挺好找的。

1.添加依赖

对于 Android 来说，需要在 app.gradle 中引入依赖：

// WebRTC
implementation 'org.webrtc:google-webrtc:1.0.32006'

由于国内众所周知的原因，下载依赖比较慢甚至会失败，所以可能需要使用阿里云 maven 镜像。

gradle 7.0 之前在 build.gradle 中添加阿里云 maven 镜像：

...
allprojects {repositories {...// 阿里云 maven 镜像maven { url 'http://maven.aliyun.com/nexus/content/groups/public/' }}
}
...

gradle 7.0 及之后则是在 setting.gradle 中添加阿里云 maven 镜像：

...
dependencyResolutionManagement {...repositories {...// 阿里云 maven 镜像maven {allowInsecureProtocol = trueurl 'http://maven.aliyun.com/nexus/content/groups/public/'}}
}
...

2.权限

除了依赖之外，Android 还需要配置网络权限、录音权限、摄像头权限、获取网络状态权限、获取 wifi 状态权限，Android 6.0 之后录音权限、摄像头权限需要动态申请，这些不是 WebRTC 的重点，只要不是 Android 新手应该都知道怎么去做，这里就不赘述了。

<!-- 允许程序访问网络的权限 -->
<uses-permission android:name="android.permission.INTERNET" />
<!-- 允许程序录音的权限 -->
<uses-permission android:name="android.permission.RECORD_AUDIO" />
<!-- 允许程序使用摄像头的权限 -->
<uses-permission android:name="android.permission.CAMERA" />
<!-- 允许程序获取网络状态的权限 -->
<uses-permission android:name="android.permission.ACCESS_NETWORK_STATE" />
<!-- 允许程序获取 wifi 状态的权限 -->
<uses-permission android:name="android.permission.ACCESS_WIFI_STATE" />

3.布局

<?xml version="1.0" encoding="utf-8"?>
<androidx.constraintlayout.widget.ConstraintLayout xmlns:android="http://schemas.android.com/apk/res/android"xmlns:app="http://schemas.android.com/apk/res-auto"xmlns:tools="http://schemas.android.com/tools"android:layout_width="match_parent"android:layout_height="match_parent"android:background="#FF000000"android:keepScreenOn="true"tools:context=".LocalDemoActivity"><org.webrtc.SurfaceViewRendererandroid:id="@+id/svr_local"android:layout_width="match_parent"android:layout_height="0dp"app:layout_constraintBottom_toBottomOf="parent"app:layout_constraintDimensionRatio="9:16"app:layout_constraintEnd_toEndOf="parent"app:layout_constraintStart_toStartOf="parent"app:layout_constraintTop_toTopOf="parent" /><org.webrtc.SurfaceViewRendererandroid:id="@+id/svr_remote"android:layout_width="90dp"android:layout_height="0dp"android:layout_marginTop="30dp"android:layout_marginEnd="30dp"app:layout_constraintDimensionRatio="9:16"app:layout_constraintEnd_toEndOf="parent"app:layout_constraintTop_toTopOf="parent" /><androidx.appcompat.widget.LinearLayoutCompatandroid:layout_width="match_parent"android:layout_height="wrap_content"android:layout_marginStart="30dp"android:layout_marginTop="30dp"android:layout_marginEnd="30dp"android:orientation="vertical"app:layout_constraintStart_toStartOf="parent"app:layout_constraintTop_toTopOf="parent"><androidx.appcompat.widget.AppCompatButtonandroid:id="@+id/btn_call"android:layout_width="wrap_content"android:layout_height="wrap_content"android:text="呼叫" /><androidx.appcompat.widget.AppCompatButtonandroid:id="@+id/btn_hang_up"android:layout_width="wrap_content"android:layout_height="wrap_content"android:text="挂断" /></androidx.appcompat.widget.LinearLayoutCompat>
</androidx.constraintlayout.widget.ConstraintLayout>

4.LocalDemoActivity

package com.qinshou.webrtcdemo_android;import android.content.Context;
import android.os.Bundle;
import android.view.View;import androidx.appcompat.app.AppCompatActivity;import org.webrtc.AudioSource;
import org.webrtc.AudioTrack;
import org.webrtc.Camera2Capturer;
import org.webrtc.Camera2Enumerator;
import org.webrtc.CameraEnumerator;
import org.webrtc.DataChannel;
import org.webrtc.DefaultVideoDecoderFactory;
import org.webrtc.DefaultVideoEncoderFactory;
import org.webrtc.EglBase;
import org.webrtc.IceCandidate;
import org.webrtc.MediaConstraints;
import org.webrtc.MediaStream;
import org.webrtc.PeerConnection;
import org.webrtc.PeerConnectionFactory;
import org.webrtc.RtpReceiver;
import org.webrtc.SessionDescription;
import org.webrtc.SurfaceTextureHelper;
import org.webrtc.SurfaceViewRenderer;
import org.webrtc.VideoCapturer;
import org.webrtc.VideoDecoderFactory;
import org.webrtc.VideoEncoderFactory;
import org.webrtc.VideoSource;
import org.webrtc.VideoTrack;import java.util.ArrayList;
import java.util.List;/*** Author: MrQinshou* Email: cqflqinhao@126.com* Date: 2023/3/21 17:22* Description: 本地 demo*/
public class LocalDemoActivity extends AppCompatActivity {private static final String TAG = LocalDemoActivity.class.getSimpleName();private static final String AUDIO_TRACK_ID = "ARDAMSa0";private static final String VIDEO_TRACK_ID = "ARDAMSv0";private static final List<String> STREAM_IDS = new ArrayList<String>() {{add("ARDAMS");}};private static final String SURFACE_TEXTURE_HELPER_THREAD_NAME = "SurfaceTextureHelperThread";private static final int WIDTH = 1280;private static final int HEIGHT = 720;private static final int FPS = 30;private EglBase mEglBase;private PeerConnectionFactory mPeerConnectionFactory;private VideoCapturer mVideoCapturer;private AudioTrack mAudioTrack;private VideoTrack mVideoTrack;private PeerConnection mLocalPeerConnection;private PeerConnection mRemotePeerConnection;@Overrideprotected void onCreate(Bundle savedInstanceState) {super.onCreate(savedInstanceState);setContentView(R.layout.activity_local_demo);findViewById(R.id.btn_call).setOnClickListener(new View.OnClickListener() {@Overridepublic void onClick(View view) {call();}});findViewById(R.id.btn_hang_up).setOnClickListener(new View.OnClickListener() {@Overridepublic void onClick(View view) {hangUp();}});// 初始化 PeerConnectionFactoryinitPeerConnectionFactory(LocalDemoActivity.this);// 创建 EglBasemEglBase = EglBase.create();// 创建 PeerConnectionFactorymPeerConnectionFactory = createPeerConnectionFactory(mEglBase);// 创建音轨mAudioTrack = createAudioTrack(mPeerConnectionFactory);// 创建视轨mVideoCapturer = createVideoCapturer();VideoSource videoSource = createVideoSource(mPeerConnectionFactory, mVideoCapturer);mVideoTrack = createVideoTrack(mPeerConnectionFactory, videoSource);// 初始化本地视频渲染控件，这个方法非常重要，不初始化会黑屏SurfaceViewRenderer svrLocal = findViewById(R.id.svr_local);svrLocal.init(mEglBase.getEglBaseContext(), null);mVideoTrack.addSink(svrLocal);// 初始化远端视频渲染控件，这个方法非常重要，不初始化会黑屏SurfaceViewRenderer svrRemote = findViewById(R.id.svr_remote);svrRemote.init(mEglBase.getEglBaseContext(), null);// 开始本地渲染// 创建 SurfaceTextureHelper，用来表示 camera 初始化的线程SurfaceTextureHelper surfaceTextureHelper = SurfaceTextureHelper.create(SURFACE_TEXTURE_HELPER_THREAD_NAME, mEglBase.getEglBaseContext());// 初始化视频采集器mVideoCapturer.initialize(surfaceTextureHelper, LocalDemoActivity.this, videoSource.getCapturerObserver());mVideoCapturer.startCapture(WIDTH, HEIGHT, FPS);}@Overrideprotected void onDestroy() {super.onDestroy();if (mEglBase != null) {mEglBase.release();mEglBase = null;}if (mVideoCapturer != null) {try {mVideoCapturer.stopCapture();} catch (InterruptedException e) {e.printStackTrace();}mVideoCapturer.dispose();mVideoCapturer = null;}if (mAudioTrack != null) {mAudioTrack.dispose();mAudioTrack = null;}if (mVideoTrack != null) {mVideoTrack.dispose();mVideoTrack = null;}if (mLocalPeerConnection != null) {mLocalPeerConnection.close();mLocalPeerConnection = null;}if (mRemotePeerConnection != null) {mRemotePeerConnection.close();mRemotePeerConnection = null;}SurfaceViewRenderer svrLocal = findViewById(R.id.svr_local);svrLocal.release();SurfaceViewRenderer svrRemote = findViewById(R.id.svr_remote);svrRemote.release();}private void initPeerConnectionFactory(Context context) {PeerConnectionFactory.initialize(PeerConnectionFactory.InitializationOptions.builder(context).createInitializationOptions());}private PeerConnectionFactory createPeerConnectionFactory(EglBase eglBase) {VideoEncoderFactory videoEncoderFactory = new DefaultVideoEncoderFactory(eglBase.getEglBaseContext(), true, true);VideoDecoderFactory videoDecoderFactory = new DefaultVideoDecoderFactory(eglBase.getEglBaseContext());return PeerConnectionFactory.builder().setVideoEncoderFactory(videoEncoderFactory).setVideoDecoderFactory(videoDecoderFactory).createPeerConnectionFactory();}private AudioTrack createAudioTrack(PeerConnectionFactory peerConnectionFactory) {AudioSource audioSource = peerConnectionFactory.createAudioSource(new MediaConstraints());AudioTrack audioTrack = peerConnectionFactory.createAudioTrack(AUDIO_TRACK_ID, audioSource);audioTrack.setEnabled(true);return audioTrack;}private VideoCapturer createVideoCapturer() {VideoCapturer videoCapturer = null;CameraEnumerator cameraEnumerator = new Camera2Enumerator(LocalDemoActivity.this);for (String deviceName : cameraEnumerator.getDeviceNames()) {// 前摄像头
//            if (cameraEnumerator.isFrontFacing(deviceName)) {// 后摄像头if (cameraEnumerator.isBackFacing(deviceName)) {videoCapturer = new Camera2Capturer(LocalDemoActivity.this, deviceName, null);}}return videoCapturer;}private VideoSource createVideoSource(PeerConnectionFactory peerConnectionFactory, VideoCapturer videoCapturer) {// 创建视频源VideoSource videoSource = peerConnectionFactory.createVideoSource(videoCapturer.isScreencast());return videoSource;}private VideoTrack createVideoTrack(PeerConnectionFactory peerConnectionFactory, VideoSource videoSource) {// 创建视轨VideoTrack videoTrack = peerConnectionFactory.createVideoTrack(VIDEO_TRACK_ID, videoSource);videoTrack.setEnabled(true);return videoTrack;}private void call() {// 创建 PeerConnectionPeerConnection.RTCConfiguration rtcConfiguration = new PeerConnection.RTCConfiguration(new ArrayList<>());mLocalPeerConnection = mPeerConnectionFactory.createPeerConnection(rtcConfiguration, new PeerConnection.Observer() {@Overridepublic void onSignalingChange(PeerConnection.SignalingState signalingState) {}@Overridepublic void onIceConnectionChange(PeerConnection.IceConnectionState iceConnectionState) {}@Overridepublic void onIceConnectionReceivingChange(boolean b) {}@Overridepublic void onIceGatheringChange(PeerConnection.IceGatheringState iceGatheringState) {}@Overridepublic void onIceCandidate(IceCandidate iceCandidate) {ShowLogUtil.verbose("onIceCandidate--->" + iceCandidate);sendIceCandidate(mLocalPeerConnection, iceCandidate);}@Overridepublic void onIceCandidatesRemoved(IceCandidate[] iceCandidates) {}@Overridepublic void onAddStream(MediaStream mediaStream) {}@Overridepublic void onRemoveStream(MediaStream mediaStream) {}@Overridepublic void onDataChannel(DataChannel dataChannel) {}@Overridepublic void onRenegotiationNeeded() {}@Overridepublic void onAddTrack(RtpReceiver rtpReceiver, MediaStream[] mediaStreams) {}});// 为 PeerConnection 添加音轨、视轨mLocalPeerConnection.addTrack(mAudioTrack, STREAM_IDS);mLocalPeerConnection.addTrack(mVideoTrack, STREAM_IDS);// 通过 PeerConnection 创建 offer，获取 sdpMediaConstraints mediaConstraints = new MediaConstraints();mLocalPeerConnection.createOffer(new MySdpObserver() {@Overridepublic void onCreateSuccess(SessionDescription sessionDescription) {ShowLogUtil.verbose("create offer success.");// 将 offer sdp 作为参数 setLocalDescriptionmLocalPeerConnection.setLocalDescription(new MySdpObserver() {@Overridepublic void onCreateSuccess(SessionDescription sessionDescription) {}@Overridepublic void onSetSuccess() {ShowLogUtil.verbose("set local sdp success.");// 发送 offer sdpsendOffer(sessionDescription);}}, sessionDescription);}@Overridepublic void onSetSuccess() {}}, mediaConstraints);}private void sendOffer(SessionDescription offer) {receivedOffer(offer);}private void receivedOffer(SessionDescription offer) {// 创建 PeerConnectionPeerConnection.RTCConfiguration rtcConfiguration = new PeerConnection.RTCConfiguration(new ArrayList<>());mRemotePeerConnection = mPeerConnectionFactory.createPeerConnection(rtcConfiguration, new PeerConnection.Observer() {@Overridepublic void onSignalingChange(PeerConnection.SignalingState signalingState) {}@Overridepublic void onIceConnectionChange(PeerConnection.IceConnectionState iceConnectionState) {}@Overridepublic void onIceConnectionReceivingChange(boolean b) {}@Overridepublic void onIceGatheringChange(PeerConnection.IceGatheringState iceGatheringState) {}@Overridepublic void onIceCandidate(IceCandidate iceCandidate) {ShowLogUtil.verbose("onIceCandidate--->" + iceCandidate);sendIceCandidate(mRemotePeerConnection, iceCandidate);}@Overridepublic void onIceCandidatesRemoved(IceCandidate[] iceCandidates) {}@Overridepublic void onAddStream(MediaStream mediaStream) {ShowLogUtil.verbose("onAddStream--->" + mediaStream);if (mediaStream == null || mediaStream.videoTracks == null || mediaStream.videoTracks.isEmpty()) {return;}runOnUiThread(new Runnable() {@Overridepublic void run() {SurfaceViewRenderer svrRemote = findViewById(R.id.svr_remote);mediaStream.videoTracks.get(0).addSink(svrRemote);}});}@Overridepublic void onRemoveStream(MediaStream mediaStream) {}@Overridepublic void onDataChannel(DataChannel dataChannel) {}@Overridepublic void onRenegotiationNeeded() {}@Overridepublic void onAddTrack(RtpReceiver rtpReceiver, MediaStream[] mediaStreams) {}});// 将 offer sdp 作为参数 setRemoteDescriptionmRemotePeerConnection.setRemoteDescription(new MySdpObserver() {@Overridepublic void onCreateSuccess(SessionDescription sessionDescription) {}@Overridepublic void onSetSuccess() {ShowLogUtil.verbose("set remote sdp success.");// 通过 PeerConnection 创建 answer，获取 sdpMediaConstraints mediaConstraints = new MediaConstraints();mRemotePeerConnection.createAnswer(new MySdpObserver() {@Overridepublic void onCreateSuccess(SessionDescription sessionDescription) {ShowLogUtil.verbose("create answer success.");// 将 answer sdp 作为参数 setLocalDescriptionmRemotePeerConnection.setLocalDescription(new MySdpObserver() {@Overridepublic void onCreateSuccess(SessionDescription sessionDescription) {}@Overridepublic void onSetSuccess() {ShowLogUtil.verbose("set local sdp success.");// 发送 answer sdpsendAnswer(sessionDescription);}}, sessionDescription);}@Overridepublic void onSetSuccess() {}}, mediaConstraints);}}, offer);}private void sendAnswer(SessionDescription answer) {receivedAnswer(answer);}private void receivedAnswer(SessionDescription answer) {// 收到 answer sdp，将 answer sdp 作为参数 setRemoteDescriptionmLocalPeerConnection.setRemoteDescription(new MySdpObserver() {@Overridepublic void onCreateSuccess(SessionDescription sessionDescription) {}@Overridepublic void onSetSuccess() {ShowLogUtil.verbose("set remote sdp success.");}}, answer);}private void sendIceCandidate(PeerConnection peerConnection, IceCandidate iceCandidate) {receivedCandidate(peerConnection, iceCandidate);}private void receivedCandidate(PeerConnection peerConnection, IceCandidate iceCandidate) {if (peerConnection == mLocalPeerConnection) {mRemotePeerConnection.addIceCandidate(iceCandidate);} else {mLocalPeerConnection.addIceCandidate(iceCandidate);}}private void hangUp() {// 关闭 PeerConnectionif (mLocalPeerConnection != null) {mLocalPeerConnection.close();mLocalPeerConnection.dispose();mLocalPeerConnection = null;}if (mRemotePeerConnection != null) {mRemotePeerConnection.close();mRemotePeerConnection.dispose();mRemotePeerConnection = null;}// 释放远端视频渲染控件SurfaceViewRenderer svrRemote = findViewById(R.id.svr_remote);svrRemote.clearImage();}
}

其中 MySdpObserver 只是 SdpObserver 的一个自定义子接口，默认实现了失败回调：

package com.qinshou.webrtcdemo_android;import org.webrtc.SdpObserver;/*** Author: MrQinshou* Email: cqflqinhao@126.com* Date: 2023/3/20 18:46* Description: 类描述*/
public interface MySdpObserver extends SdpObserver {@Overridedefault void onCreateFailure(String s) {}@Overridedefault void onSetFailure(String s) {}
}

运行起来后点击呼叫，在右上角的远端控件就能收到本地视频了：

四、iOS

相较于 Android，iOS 端的资料相对较少，且资料大同小异，我也是踩了好多坑才跑通了 demo。

1.添加依赖

对于 iOS 来说，也需要额外引入依赖，由于我是做 Android 的，对 iOS 的其他引入依赖方式不太了解，下面是使用 Cocopods 引入 WebRTC 依赖：

...
target 'WebRTCDemo-iOS' do...pod 'GoogleWebRTC','~>1.1.31999'
end
...

2.权限

除了依赖之外，iOS 还需要配置录音权限、摄像头权限，也需要动态申请，这些不是 WebRTC 的重点，只要不是 iOS 新手应该都知道怎么去做，这里就不赘述了。

<key>NSMicrophoneUsageDescription</key>
<string>允许程序使用麦克风的权限</string>
<key>NSCameraUsageDescription</key>
<string>允许程序使用摄像头的权限</string>

3.LocalDemoViewController

//
//  LocalDemoViewController.swift
//  WebRTCDemo
//
//  Created by 覃浩 on 2023/3/21.
//import UIKit
import WebRTC
import SnapKitclass LocalDemoViewController: UIViewController {private static let AUDIO_TRACK_ID = "ARDAMSa0"private static let VIDEO_TRACK_ID = "ARDAMSv0"private static let STREAM_IDS = ["ARDAMS"]private static let WIDTH = 1280private static let HEIGHT = 720private static let FPS = 30private var localView: RTCEAGLVideoView!private var remoteView: RTCEAGLVideoView!private var peerConnectionFactory: RTCPeerConnectionFactory!private var audioTrack: RTCAudioTrack?private var videoTrack: RTCVideoTrack?/**iOS 需要将 Capturer 保存为全局变量，否则无法渲染本地画面*/private var videoCapturer: RTCVideoCapturer?/**iOS 需要将远端流保存为全局变量，否则无法渲染远端画面*/private var remoteStream: RTCMediaStream?private var localPeerConnection: RTCPeerConnection?private var remotePeerConnection: RTCPeerConnection?override func viewDidLoad() {super.viewDidLoad()// 表明 View 不要扩展到整个屏幕，而是在 NavigationBar 下的区域edgesForExtendedLayout = UIRectEdge()let btnCall = UIButton()btnCall.backgroundColor = UIColor.lightGraybtnCall.setTitle("呼叫", for: .normal)btnCall.setTitleColor(UIColor.black, for: .normal)btnCall.addTarget(self, action: #selector(call), for: .touchUpInside)self.view.addSubview(btnCall)btnCall.snp.makeConstraints({ maker inmaker.left.equalToSuperview().offset(30)maker.width.equalTo(60)maker.height.equalTo(40)})let btnHangUp = UIButton()btnHangUp.backgroundColor = UIColor.lightGraybtnHangUp.setTitle("挂断", for: .normal)btnHangUp.setTitleColor(UIColor.black, for: .normal)btnHangUp.addTarget(self, action: #selector(hangUp), for: .touchUpInside)self.view.addSubview(btnHangUp)btnHangUp.snp.makeConstraints({ maker inmaker.left.equalToSuperview().offset(30)maker.width.equalTo(60)maker.height.equalTo(40)maker.top.equalTo(btnCall.snp.bottom).offset(10)})// 初始化 PeerConnectionFactoryinitPeerConnectionFactory()// 创建 EglBase// 创建 PeerConnectionFactorypeerConnectionFactory = createPeerConnectionFactory()// 创建音轨audioTrack = createAudioTrack(peerConnectionFactory: peerConnectionFactory)// 创建视轨videoTrack = createVideoTrack(peerConnectionFactory: peerConnectionFactory)let tuple = createVideoCapturer(videoSource: videoTrack!.source)let captureDevice = tuple.captureDevicevideoCapturer = tuple.videoCapture// 初始化本地视频渲染控件localView = RTCEAGLVideoView()localView.delegate = selfself.view.insertSubview(localView,at: 0)localView.snp.makeConstraints({ maker inmaker.width.equalToSuperview()maker.height.equalTo(localView.snp.width).multipliedBy(16.0/9.0)maker.centerY.equalToSuperview()})videoTrack?.add(localView!)// 初始化远端视频渲染控件remoteView = RTCEAGLVideoView()remoteView.delegate = selfself.view.insertSubview(remoteView, aboveSubview: localView)remoteView.snp.makeConstraints({ maker inmaker.width.equalTo(90)maker.height.equalTo(160)maker.top.equalToSuperview().offset(30)maker.right.equalToSuperview().offset(-30)})// 开始本地渲染(videoCapturer as? RTCCameraVideoCapturer)?.startCapture(with: captureDevice!, format: captureDevice!.activeFormat, fps: LocalDemoViewController.FPS)}override func viewDidDisappear(_ animated: Bool) {(videoCapturer as? RTCCameraVideoCapturer)?.stopCapture()videoCapturer = nillocalPeerConnection?.close()localPeerConnection = nilremotePeerConnection?.close()remotePeerConnection = nil}private func initPeerConnectionFactory() {RTCPeerConnectionFactory.initialize()}private func createPeerConnectionFactory() -> RTCPeerConnectionFactory {var videoEncoderFactory = RTCDefaultVideoEncoderFactory()var videoDecoderFactory = RTCDefaultVideoDecoderFactory()if TARGET_OS_SIMULATOR != 0 {videoEncoderFactory = RTCSimluatorVideoEncoderFactory()videoDecoderFactory = RTCSimulatorVideoDecoderFactory()}return RTCPeerConnectionFactory(encoderFactory: videoEncoderFactory, decoderFactory: videoDecoderFactory)}private func createAudioTrack(peerConnectionFactory: RTCPeerConnectionFactory) -> RTCAudioTrack {let mandatoryConstraints : [String : String] = [:]let optionalConstraints : [String : String] = [:]let audioSource = peerConnectionFactory.audioSource(with: RTCMediaConstraints(mandatoryConstraints: mandatoryConstraints, optionalConstraints: optionalConstraints))let audioTrack = peerConnectionFactory.audioTrack(with: audioSource, trackId: LocalDemoViewController.AUDIO_TRACK_ID)audioTrack.isEnabled = truereturn audioTrack}private func createVideoTrack(peerConnectionFactory: RTCPeerConnectionFactory) -> RTCVideoTrack? {let videoSource = peerConnectionFactory.videoSource()let videoTrack = peerConnectionFactory.videoTrack(with: videoSource, trackId: LocalDemoViewController.VIDEO_TRACK_ID)videoTrack.isEnabled = truereturn videoTrack}private func createVideoCapturer(videoSource: RTCVideoSource) -> (captureDevice: AVCaptureDevice?, videoCapture: RTCVideoCapturer?) {let videoCapturer = RTCCameraVideoCapturer(delegate: videoSource)let captureDevices = RTCCameraVideoCapturer.captureDevices()if (captureDevices.count == 0) {return (nil, nil)}var captureDevice: AVCaptureDevice?for c in captureDevices {// 前摄像头// if (c.position == .front) {// 后摄像头if (c.position == .back) {captureDevice = cbreak}}if (captureDevice == nil) {return (nil, nil)}return (captureDevice, videoCapturer)}@objc private func call() {// 创建 PeerConnectionlet rtcConfiguration = RTCConfiguration()var mandatoryConstraints : [String : String] = [:]var optionalConstraints : [String : String] = [:]var mediaConstraints = RTCMediaConstraints(mandatoryConstraints: mandatoryConstraints, optionalConstraints: optionalConstraints)localPeerConnection = peerConnectionFactory.peerConnection(with: rtcConfiguration, constraints: mediaConstraints, delegate: self)// 为 PeerConnection 添加音轨、视轨localPeerConnection?.add(audioTrack!, streamIds: LocalDemoViewController.STREAM_IDS)localPeerConnection?.add(videoTrack!, streamIds: LocalDemoViewController.STREAM_IDS)// 通过 PeerConnection 创建 offer，获取 sdpmandatoryConstraints = [:]optionalConstraints = [:]mediaConstraints = RTCMediaConstraints(mandatoryConstraints: mandatoryConstraints, optionalConstraints: optionalConstraints)localPeerConnection?.offer(for: mediaConstraints, completionHandler: { sessionDescription, error inShowLogUtil.verbose("create offer success.")// 将 offer sdp 作为参数 setLocalDescriptionself.localPeerConnection?.setLocalDescription(sessionDescription!, completionHandler: { _ inShowLogUtil.verbose("set local sdp success.")// 发送 offer sdpself.sendOffer(offer: sessionDescription!)})})}private func sendOffer(offer: RTCSessionDescription) {receivedOffer(offer: offer)}private func receivedOffer(offer: RTCSessionDescription) {// 创建 PeerConnectionlet rtcConfiguration = RTCConfiguration()let mandatoryConstraints : [String : String] = [:]let optionalConstraints : [String : String] = [:]var mediaConstraints = RTCMediaConstraints(mandatoryConstraints: mandatoryConstraints, optionalConstraints: optionalConstraints)remotePeerConnection = peerConnectionFactory.peerConnection(with: rtcConfiguration, constraints: mediaConstraints, delegate: self)// 将 offer sdp 作为参数 setRemoteDescriptionremotePeerConnection?.setRemoteDescription(offer, completionHandler: { _ inShowLogUtil.verbose("set remote sdp success.")// 通过 PeerConnection 创建 answer，获取 sdplet mandatoryConstraints : [String : String] = [:]let optionalConstraints : [String : String] = [:]let mediaConstraints = RTCMediaConstraints(mandatoryConstraints: mandatoryConstraints, optionalConstraints: optionalConstraints)self.remotePeerConnection?.answer(for: mediaConstraints, completionHandler: { sessionDescription, error inShowLogUtil.verbose("create answer success.")// 将 answer sdp 作为参数 setLocalDescriptionself.remotePeerConnection?.setLocalDescription(sessionDescription!, completionHandler: { _ inShowLogUtil.verbose("set local sdp success.")// 发送 answer sdpself.sendAnswer(answer: sessionDescription!)})})})}private func sendAnswer(answer: RTCSessionDescription) {receivedAnswer(answer: answer)}private func receivedAnswer(answer: RTCSessionDescription) {// 收到 answer sdp，将 answer sdp 作为参数 setRemoteDescriptionlocalPeerConnection?.setRemoteDescription(answer, completionHandler: { _ in   ShowLogUtil.verbose("set remote sdp success.")})}private func sendIceCandidate(peerConnection: RTCPeerConnection, iceCandidate: RTCIceCandidate)  {receivedCandidate(peerConnection: peerConnection,iceCandidate: iceCandidate)}private func receivedCandidate(peerConnection: RTCPeerConnection, iceCandidate: RTCIceCandidate) {if (peerConnection == localPeerConnection) {remotePeerConnection?.add(iceCandidate)} else {localPeerConnection?.add(iceCandidate)}}@objc private func hangUp() {// 关闭 PeerConnectionlocalPeerConnection?.close()localPeerConnection = nilremotePeerConnection?.close()remotePeerConnection = nil// 释放远端视频渲染控件if let track = remoteStream?.videoTracks.first {track.remove(remoteView!)}}
}// MARK: - RTCVideoViewDelegate
extension LocalDemoViewController: RTCVideoViewDelegate {func videoView(_ videoView: RTCVideoRenderer, didChangeVideoSize size: CGSize) {}
}// MARK: - RTCPeerConnectionDelegate
extension LocalDemoViewController: RTCPeerConnectionDelegate {func peerConnection(_ peerConnection: RTCPeerConnection, didChange stateChanged: RTCSignalingState) {}func peerConnection(_ peerConnection: RTCPeerConnection, didAdd stream: RTCMediaStream) {ShowLogUtil.verbose("peerConnection didAdd stream--->\(stream)")if (peerConnection == self.localPeerConnection) {} else if (peerConnection == self.remotePeerConnection) {self.remoteStream = streamif let track = stream.videoTracks.first {track.add(remoteView!)}if let audioTrack = stream.audioTracks.first{audioTrack.source.volume = 8}}}func peerConnection(_ peerConnection: RTCPeerConnection, didRemove stream: RTCMediaStream) {}func peerConnectionShouldNegotiate(_ peerConnection: RTCPeerConnection) {}func peerConnection(_ peerConnection: RTCPeerConnection, didChange newState: RTCIceConnectionState) {}func peerConnection(_ peerConnection: RTCPeerConnection, didChange newState: RTCIceGatheringState) {}func peerConnection(_ peerConnection: RTCPeerConnection, didGenerate candidate: RTCIceCandidate) {ShowLogUtil.verbose("didGenerate candidate--->\(candidate)")self.sendIceCandidate(peerConnection: peerConnection, iceCandidate: candidate)}func peerConnection(_ peerConnection: RTCPeerConnection, didRemove candidates: [RTCIceCandidate]) {}func peerConnection(_ peerConnection: RTCPeerConnection, didOpen dataChannel: RTCDataChannel) {}
}

运行起来后点击呼叫，在右上角的远端控件就能收到本地视频了：

Gif 的质量就包涵一下，压缩后实在是失真有点严重。大家可能会看到 iOS 中间会卡了一下，那其实不是卡了，因为 iOS 录屏录不了点击事件，实际上是跟 H5 和 Android 一样，我中间点击了一下挂断，但是 iOS 挂断后释放远端渲染控件时，控件会保留显示最后一帧的画面，所以大家如果在实际做的时候想要效果好一点的话，可以在渲染控件上再放一个黑色遮罩，挂断时显示遮罩。

五、总结

踩过的坑，其实在代码中已经有注释了，大家留意一下。

Android 端的坑：

  1. SurfaceViewRender 对象需要调用 init() 方法初始化，否则会黑屏。

iOS 端的坑：

  1. 本地画面渲染控件就老老实实的用 RTCEAGLVideoView 就好，不要用 RTCCameraPreviewView，跟远端渲染控件保持一致，代码更整齐，使用其他视频源时还容易切换。
  2. 本地视频捕获对象（RTCVideoCapturer）、视轨对象（RTCVideoTrack）、远端视频流对象（RTCMediaStream）要声明成全局变量，我也不知道为什么，但是如果声明成局部变量就会渲染不出来画面。
  3. demo 阶段不要去设置花里胡哨的参数，（DtlsSrtpKeyAgreement、kRTCMediaConstraintsOfferToReceiveAudio、kRTCMediaConstraintsOfferToReceiveVideo 等），均使用默认配置就好，设置参数越多越不容易排查问题。

通过这三端的本地 demo，大家应该还是可以看到一定区别，H5 其实很少的代码量就能实现 WebRTC 功能，而 Android 相对来说需要初始化的地方更多，但可能因为是我的本行的缘故，所以我在实现 Android 端的时候并没有遇到什么坑。

其实一端与另一端通话是只需要创建一个 PeerConnection ，这一次本地 demo 中我们创建了两个 PeerConnection，这是为了不搭建信令服务器就能看到效果，所以创建了一个 RemotePeerConnection 模拟远端，下一次我们就会通过 WebSocket 搭建一个简单的信令服务器来实现点对点通话。

六、Demo

Demo 传送门

本文链接：https://my.lmcjl.com/post/5614.html

展开阅读全文

相关文章

4 评论

留下您的评论.