WebRTC 系列(二、本地 demo,H5、Android、iOS)

WebRTC 系列(一、简介)​​​​​​​

一、整体流程

有了上一篇 WebRTC 简介的基础,我们知道了 WebRTC 的工作流程,接下来就是需要用代码去实现这个流程了。对于不同端,实现起来的难易程度可能略微不同(实践中我感觉 iOS 端是资料最少的),但是整体流程是一样的。

问:要把大象装冰箱,总共分几步?答:三步。

  1. 初始化;
  2. 发起呼叫,接收应答;
  3. 挂断,释放资源。

初始化

  1. 初始化 PeerConnectionFactory;
  2. 创建 EglBase;
  3. 创建 PeerConnectionFactory;
  4. 创建音轨;
  5. 创建视轨;
  6. 初始化本地视频渲染控件;
  7. 初始化远端视频渲染控件;
  8. 开始本地渲染。

这些步骤,不同端不一定都需要,有的端的 WebRTC 依赖库中已经做了一些,H5 的初始化操作是最少的。有的步骤的顺序可能会略有不同,Android 和 iOS 就有区别,需要注意的是初始化不当可能导致画面渲染不出来,而且还不易排查,后面我再说明我踩过哪些坑。

发起呼叫,接收应答

  1. A 创建 PeerConnection;
  2. A 为 PeerConnection 添加音轨、视轨;
  3. A 通过 PeerConnection 创建 offer,获取 sdp;
  4. A 将 offer sdp 作为参数 setLocalDescription;
  5. A 发送 offer sdp;
  6. B 收到 offer sdp,创建 PeerConnection;
  7. B 将 offer sdp 作为参数 setRemoteDescription;
  8. B 通过 PeerConnection 创建 answer,获取 sdp;
  9. B 将 answer sdp 作为参数 setLocalDescription;
  10. B 发送 answer sdp;
  11. A 收到 answer sdp,将 answer sdp 作为参数 setRemoteDescription。

这期间,PeerConnection 的 onIceCandidate() 这个回调方法会被调用,我们需要将生成的 IceCandidate 对象传递给对方,收到 IceCandidate 的时候我们需要通过 PeerConnection 的 addIceCandidate() 方法添加。

这期间,PeerConnection 的 onAddStream() 这个回调也会被调用,我们需要将 MediaStream 添加到远端控件渲染器中进行渲染。

挂断,释放资源

  1. 关闭 PeerConnection;
  2. 释放远端视频渲染控件。

释放资源主要是为了能够正常的再次呼叫。

整理完整体流程后,我们可以先来实现一个本地 demo,我是创建了两个 PeerConnection,一个本地 PeerConnection 作为用户 A,一个远端 PeerConnection 作为用户 B,由于大部分 Android 设备和 iOS 设备同时只能开启一个摄像头,所以远端 PeerConnection 不会采集视频数据,只会渲染远端画面。

二、H5

1.添加依赖

上一篇博客已经说到,WebRTC 已经成为 H5 的标准之一,所以 H5 实现它很容易,并不需要额外引入依赖。

2.local_demo.html

H5 的实现相对简单,很多初始化的操作我们不需要去做,浏览器已经帮我们做好了,整体代码如下:

<html><head><title>Local Demo</title><style>body {overflow: hidden;margin: 0px;padding: 0px;}#local_view {width: 100%;height: 100%;}#remote_view {width: 9%;height: 16%;position: absolute;top: 10%;right: 10%;}#left {width: 10%;height: 5%;position: absolute;left: 10%;top: 10%;}.my_button {width: 100%;height: 100%;display: block;margin-bottom: 10%;}</style>
</head><body><video id="local_view" autoplay controls muted></video><video id="remote_view" autoplay controls muted></video><div id="left"><button id="btn_call" class="my_button" onclick="call()">呼叫</button><button id="btn_hang_up" class="my_button" onclick="hangUp()">挂断</button></div>
</body><script type="text/javascript">var localView = document.getElementById("local_view");var remoteView = document.getElementById("remote_view");var localStream;let localPeerConnection;function call() {// 创建 PeerConnectionlocalPeerConnection = new RTCPeerConnection();localPeerConnection.onicecandidate = function (event) {console.log("onicecandidate--->" + event.candidate);let iceCandidate = event.candidate;if (iceCandidate == null) {return;}sendIceCandidate(localPeerConnection, iceCandidate);}// 为 PeerConnection 添加音轨、视轨for (let i = 0; localStream != null && i < localStream.getTracks().length; i++) {const track = localStream.getTracks()[i];localPeerConnection.addTrack(track, localStream);}// 通过 PeerConnection 创建 offer,获取 sdplocalPeerConnection.createOffer().then(function (sessionDescription) {console.log("create offer success.");// 将 offer sdp 作为参数 setLocalDescriptionlocalPeerConnection.setLocalDescription(sessionDescription).then(function () {console.log("set local sdp success.");// 发送 offer sdpsendOffer(sessionDescription)})})}function sendOffer(offer) {receivedOffer(offer);}function receivedOffer(offer) {// 创建 PeerConnectionremotePeerConnection = new RTCPeerConnection();remotePeerConnection.onicecandidate = function (event) {console.log("onicecandidate--->" + event.candidate);let iceCandidate = event.candidate;if (iceCandidate == null) {return;}sendIceCandidate(peerConnection, iceCandidate);}remotePeerConnection.ontrack = function (event) {console.log("ontrack--->" + event.streams);let streams = event.streams;if (streams && streams.length > 0) {remoteView.srcObject = streams[0];}}// 将 offer sdp 作为参数 setRemoteDescriptionremotePeerConnection.setRemoteDescription(offer).then(function () {console.log("set remote sdp success.");// 通过 PeerConnection 创建 answer,获取 sdpremotePeerConnection.createAnswer().then(function (sessionDescription) {console.log("create answer success.");// 将 answer sdp 作为参数 setLocalDescriptionremotePeerConnection.setLocalDescription(sessionDescription).then(function () {console.log("set local sdp success.");// 发送 answer sdpsendAnswer(sessionDescription);})})})}function sendAnswer(answer) {receivedAnswer(answer)}function receivedAnswer(answer) {// 收到 answer sdp,将 answer sdp 作为参数 setRemoteDescriptionlocalPeerConnection.setRemoteDescription(answer).then(function () {console.log("set remote sdp success.");})}function sendIceCandidate(peerConnection, iceCandidate) {receivedCandidate(peerConnection, iceCandidate);}function receivedCandidate(peerConnection, iceCandidate) {if (peerConnection == localPeerConnection) {remotePeerConnection.addIceCandidate(iceCandidate);} else {localPeerConnection.addIceCandidate(iceCandidate);}}function hangUp() {// 关闭 PeerConnectionif (localPeerConnection != null) {localPeerConnection.close();localPeerConnection = null;}if (remotePeerConnection != null) {remotePeerConnection.close();remotePeerConnection = null;}// 释放远端视频渲染控件remoteView.removeAttribute('src');remoteView.load();}navigator.mediaDevices.getUserMedia({ audio: true, video: true }).then(function (mediaStream) {// 初始化 PeerConnectionFactory// 创建 EglBase// 创建 PeerConnectionFactory// 创建音轨// 创建视轨localStream = mediaStream;// 初始化本地视频渲染控件// 初始化远端视频渲染控件// 开始本地渲染localView.srcObject = mediaStream;}).catch(function (error) {console.log("error--->" + error);})
</script></html>

运行起来后点击呼叫,在右上角的远端控件就能收到本地视频了:

三、Android

Android 端流程是最多的,但是由于我是写 Android 的,所以觉得它的流程是最严谨的,也可能是因为 WebRTC 是 Google 开源的缘故,所以 Android 端这方面的资料还是挺好找的。

1.添加依赖

对于 Android 来说,需要在 app.gradle 中引入依赖:

// WebRTC
implementation 'org.webrtc:google-webrtc:1.0.32006'

由于国内众所周知的原因,下载依赖比较慢甚至会失败,所以可能需要使用阿里云 maven 镜像。

gradle 7.0 之前在 build.gradle 中添加阿里云 maven 镜像:

...
allprojects {repositories {...// 阿里云 maven 镜像maven { url 'http://maven.aliyun.com/nexus/content/groups/public/' }}
}
...

gradle 7.0 及之后则是在 setting.gradle 中添加阿里云 maven 镜像:

...
dependencyResolutionManagement {...repositories {...// 阿里云 maven 镜像maven {allowInsecureProtocol = trueurl 'http://maven.aliyun.com/nexus/content/groups/public/'}}
}
...

2.权限

除了依赖之外,Android 还需要配置网络权限、录音权限、摄像头权限、获取网络状态权限、获取 wifi 状态权限,Android 6.0 之后录音权限、摄像头权限需要动态申请,这些不是 WebRTC 的重点,只要不是 Android 新手应该都知道怎么去做,这里就不赘述了。

<!-- 允许程序访问网络的权限 -->
<uses-permission android:name="android.permission.INTERNET" />
<!-- 允许程序录音的权限 -->
<uses-permission android:name="android.permission.RECORD_AUDIO" />
<!-- 允许程序使用摄像头的权限 -->
<uses-permission android:name="android.permission.CAMERA" />
<!-- 允许程序获取网络状态的权限 -->
<uses-permission android:name="android.permission.ACCESS_NETWORK_STATE" />
<!-- 允许程序获取 wifi 状态的权限 -->
<uses-permission android:name="android.permission.ACCESS_WIFI_STATE" />

3.布局

<?xml version="1.0" encoding="utf-8"?>
<androidx.constraintlayout.widget.ConstraintLayout xmlns:android="http://schemas.android.com/apk/res/android"xmlns:app="http://schemas.android.com/apk/res-auto"xmlns:tools="http://schemas.android.com/tools"android:layout_width="match_parent"android:layout_height="match_parent"android:background="#FF000000"android:keepScreenOn="true"tools:context=".LocalDemoActivity"><org.webrtc.SurfaceViewRendererandroid:id="@+id/svr_local"android:layout_width="match_parent"android:layout_height="0dp"app:layout_constraintBottom_toBottomOf="parent"app:layout_constraintDimensionRatio="9:16"app:layout_constraintEnd_toEndOf="parent"app:layout_constraintStart_toStartOf="parent"app:layout_constraintTop_toTopOf="parent" /><org.webrtc.SurfaceViewRendererandroid:id="@+id/svr_remote"android:layout_width="90dp"android:layout_height="0dp"android:layout_marginTop="30dp"android:layout_marginEnd="30dp"app:layout_constraintDimensionRatio="9:16"app:layout_constraintEnd_toEndOf="parent"app:layout_constraintTop_toTopOf="parent" /><androidx.appcompat.widget.LinearLayoutCompatandroid:layout_width="match_parent"android:layout_height="wrap_content"android:layout_marginStart="30dp"android:layout_marginTop="30dp"android:layout_marginEnd="30dp"android:orientation="vertical"app:layout_constraintStart_toStartOf="parent"app:layout_constraintTop_toTopOf="parent"><androidx.appcompat.widget.AppCompatButtonandroid:id="@+id/btn_call"android:layout_width="wrap_content"android:layout_height="wrap_content"android:text="呼叫" /><androidx.appcompat.widget.AppCompatButtonandroid:id="@+id/btn_hang_up"android:layout_width="wrap_content"android:layout_height="wrap_content"android:text="挂断" /></androidx.appcompat.widget.LinearLayoutCompat>
</androidx.constraintlayout.widget.ConstraintLayout>

4.LocalDemoActivity

package com.qinshou.webrtcdemo_android;import android.content.Context;
import android.os.Bundle;
import android.view.View;import androidx.appcompat.app.AppCompatActivity;import org.webrtc.AudioSource;
import org.webrtc.AudioTrack;
import org.webrtc.Camera2Capturer;
import org.webrtc.Camera2Enumerator;
import org.webrtc.CameraEnumerator;
import org.webrtc.DataChannel;
import org.webrtc.DefaultVideoDecoderFactory;
import org.webrtc.DefaultVideoEncoderFactory;
import org.webrtc.EglBase;
import org.webrtc.IceCandidate;
import org.webrtc.MediaConstraints;
import org.webrtc.MediaStream;
import org.webrtc.PeerConnection;
import org.webrtc.PeerConnectionFactory;
import org.webrtc.RtpReceiver;
import org.webrtc.SessionDescription;
import org.webrtc.SurfaceTextureHelper;
import org.webrtc.SurfaceViewRenderer;
import org.webrtc.VideoCapturer;
import org.webrtc.VideoDecoderFactory;
import org.webrtc.VideoEncoderFactory;
import org.webrtc.VideoSource;
import org.webrtc.VideoTrack;import java.util.ArrayList;
import java.util.List;/*** Author: MrQinshou* Email: cqflqinhao@126.com* Date: 2023/3/21 17:22* Description: 本地 demo*/
public class LocalDemoActivity extends AppCompatActivity {private static final String TAG = LocalDemoActivity.class.getSimpleName();private static final String AUDIO_TRACK_ID = "ARDAMSa0";private static final String VIDEO_TRACK_ID = "ARDAMSv0";private static final List<String> STREAM_IDS = new ArrayList<String>() {{add("ARDAMS");}};private static final String SURFACE_TEXTURE_HELPER_THREAD_NAME = "SurfaceTextureHelperThread";private static final int WIDTH = 1280;private static final int HEIGHT = 720;private static final int FPS = 30;private EglBase mEglBase;private PeerConnectionFactory mPeerConnectionFactory;private VideoCapturer mVideoCapturer;private AudioTrack mAudioTrack;private VideoTrack mVideoTrack;private PeerConnection mLocalPeerConnection;private PeerConnection mRemotePeerConnection;@Overrideprotected void onCreate(Bundle savedInstanceState) {super.onCreate(savedInstanceState);setContentView(R.layout.activity_local_demo);findViewById(R.id.btn_call).setOnClickListener(new View.OnClickListener() {@Overridepublic void onClick(View view) {call();}});findViewById(R.id.btn_hang_up).setOnClickListener(new View.OnClickListener() {@Overridepublic void onClick(View view) {hangUp();}});// 初始化 PeerConnectionFactoryinitPeerConnectionFactory(LocalDemoActivity.this);// 创建 EglBasemEglBase = EglBase.create();// 创建 PeerConnectionFactorymPeerConnectionFactory = createPeerConnectionFactory(mEglBase);// 创建音轨mAudioTrack = createAudioTrack(mPeerConnectionFactory);// 创建视轨mVideoCapturer = createVideoCapturer();VideoSource videoSource = createVideoSource(mPeerConnectionFactory, mVideoCapturer);mVideoTrack = createVideoTrack(mPeerConnectionFactory, videoSource);// 初始化本地视频渲染控件,这个方法非常重要,不初始化会黑屏SurfaceViewRenderer svrLocal = findViewById(R.id.svr_local);svrLocal.init(mEglBase.getEglBaseContext(), null);mVideoTrack.addSink(svrLocal);// 初始化远端视频渲染控件,这个方法非常重要,不初始化会黑屏SurfaceViewRenderer svrRemote = findViewById(R.id.svr_remote);svrRemote.init(mEglBase.getEglBaseContext(), null);// 开始本地渲染// 创建 SurfaceTextureHelper,用来表示 camera 初始化的线程SurfaceTextureHelper surfaceTextureHelper = SurfaceTextureHelper.create(SURFACE_TEXTURE_HELPER_THREAD_NAME, mEglBase.getEglBaseContext());// 初始化视频采集器mVideoCapturer.initialize(surfaceTextureHelper, LocalDemoActivity.this, videoSource.getCapturerObserver());mVideoCapturer.startCapture(WIDTH, HEIGHT, FPS);}@Overrideprotected void onDestroy() {super.onDestroy();if (mEglBase != null) {mEglBase.release();mEglBase = null;}if (mVideoCapturer != null) {try {mVideoCapturer.stopCapture();} catch (InterruptedException e) {e.printStackTrace();}mVideoCapturer.dispose();mVideoCapturer = null;}if (mAudioTrack != null) {mAudioTrack.dispose();mAudioTrack = null;}if (mVideoTrack != null) {mVideoTrack.dispose();mVideoTrack = null;}if (mLocalPeerConnection != null) {mLocalPeerConnection.close();mLocalPeerConnection = null;}if (mRemotePeerConnection != null) {mRemotePeerConnection.close();mRemotePeerConnection = null;}SurfaceViewRenderer svrLocal = findViewById(R.id.svr_local);svrLocal.release();SurfaceViewRenderer svrRemote = findViewById(R.id.svr_remote);svrRemote.release();}private void initPeerConnectionFactory(Context context) {PeerConnectionFactory.initialize(PeerConnectionFactory.InitializationOptions.builder(context).createInitializationOptions());}private PeerConnectionFactory createPeerConnectionFactory(EglBase eglBase) {VideoEncoderFactory videoEncoderFactory = new DefaultVideoEncoderFactory(eglBase.getEglBaseContext(), true, true);VideoDecoderFactory videoDecoderFactory = new DefaultVideoDecoderFactory(eglBase.getEglBaseContext());return PeerConnectionFactory.builder().setVideoEncoderFactory(videoEncoderFactory).setVideoDecoderFactory(videoDecoderFactory).createPeerConnectionFactory();}private AudioTrack createAudioTrack(PeerConnectionFactory peerConnectionFactory) {AudioSource audioSource = peerConnectionFactory.createAudioSource(new MediaConstraints());AudioTrack audioTrack = peerConnectionFactory.createAudioTrack(AUDIO_TRACK_ID, audioSource);audioTrack.setEnabled(true);return audioTrack;}private VideoCapturer createVideoCapturer() {VideoCapturer videoCapturer = null;CameraEnumerator cameraEnumerator = new Camera2Enumerator(LocalDemoActivity.this);for (String deviceName : cameraEnumerator.getDeviceNames()) {// 前摄像头
//            if (cameraEnumerator.isFrontFacing(deviceName)) {// 后摄像头if (cameraEnumerator.isBackFacing(deviceName)) {videoCapturer = new Camera2Capturer(LocalDemoActivity.this, deviceName, null);}}return videoCapturer;}private VideoSource createVideoSource(PeerConnectionFactory peerConnectionFactory, VideoCapturer videoCapturer) {// 创建视频源VideoSource videoSource = peerConnectionFactory.createVideoSource(videoCapturer.isScreencast());return videoSource;}private VideoTrack createVideoTrack(PeerConnectionFactory peerConnectionFactory, VideoSource videoSource) {// 创建视轨VideoTrack videoTrack = peerConnectionFactory.createVideoTrack(VIDEO_TRACK_ID, videoSource);videoTrack.setEnabled(true);return videoTrack;}private void call() {// 创建 PeerConnectionPeerConnection.RTCConfiguration rtcConfiguration = new PeerConnection.RTCConfiguration(new ArrayList<>());mLocalPeerConnection = mPeerConnectionFactory.createPeerConnection(rtcConfiguration, new PeerConnection.Observer() {@Overridepublic void onSignalingChange(PeerConnection.SignalingState signalingState) {}@Overridepublic void onIceConnectionChange(PeerConnection.IceConnectionState iceConnectionState) {}@Overridepublic void onIceConnectionReceivingChange(boolean b) {}@Overridepublic void onIceGatheringChange(PeerConnection.IceGatheringState iceGatheringState) {}@Overridepublic void onIceCandidate(IceCandidate iceCandidate) {ShowLogUtil.verbose("onIceCandidate--->" + iceCandidate);sendIceCandidate(mLocalPeerConnection, iceCandidate);}@Overridepublic void onIceCandidatesRemoved(IceCandidate[] iceCandidates) {}@Overridepublic void onAddStream(MediaStream mediaStream) {}@Overridepublic void onRemoveStream(MediaStream mediaStream) {}@Overridepublic void onDataChannel(DataChannel dataChannel) {}@Overridepublic void onRenegotiationNeeded() {}@Overridepublic void onAddTrack(RtpReceiver rtpReceiver, MediaStream[] mediaStreams) {}});// 为 PeerConnection 添加音轨、视轨mLocalPeerConnection.addTrack(mAudioTrack, STREAM_IDS);mLocalPeerConnection.addTrack(mVideoTrack, STREAM_IDS);// 通过 PeerConnection 创建 offer,获取 sdpMediaConstraints mediaConstraints = new MediaConstraints();mLocalPeerConnection.createOffer(new MySdpObserver() {@Overridepublic void onCreateSuccess(SessionDescription sessionDescription) {ShowLogUtil.verbose("create offer success.");// 将 offer sdp 作为参数 setLocalDescriptionmLocalPeerConnection.setLocalDescription(new MySdpObserver() {@Overridepublic void onCreateSuccess(SessionDescription sessionDescription) {}@Overridepublic void onSetSuccess() {ShowLogUtil.verbose("set local sdp success.");// 发送 offer sdpsendOffer(sessionDescription);}}, sessionDescription);}@Overridepublic void onSetSuccess() {}}, mediaConstraints);}private void sendOffer(SessionDescription offer) {receivedOffer(offer);}private void receivedOffer(SessionDescription offer) {// 创建 PeerConnectionPeerConnection.RTCConfiguration rtcConfiguration = new PeerConnection.RTCConfiguration(new ArrayList<>());mRemotePeerConnection = mPeerConnectionFactory.createPeerConnection(rtcConfiguration, new PeerConnection.Observer() {@Overridepublic void onSignalingChange(PeerConnection.SignalingState signalingState) {}@Overridepublic void onIceConnectionChange(PeerConnection.IceConnectionState iceConnectionState) {}@Overridepublic void onIceConnectionReceivingChange(boolean b) {}@Overridepublic void onIceGatheringChange(PeerConnection.IceGatheringState iceGatheringState) {}@Overridepublic void onIceCandidate(IceCandidate iceCandidate) {ShowLogUtil.verbose("onIceCandidate--->" + iceCandidate);sendIceCandidate(mRemotePeerConnection, iceCandidate);}@Overridepublic void onIceCandidatesRemoved(IceCandidate[] iceCandidates) {}@Overridepublic void onAddStream(MediaStream mediaStream) {ShowLogUtil.verbose("onAddStream--->" + mediaStream);if (mediaStream == null || mediaStream.videoTracks == null || mediaStream.videoTracks.isEmpty()) {return;}runOnUiThread(new Runnable() {@Overridepublic void run() {SurfaceViewRenderer svrRemote = findViewById(R.id.svr_remote);mediaStream.videoTracks.get(0).addSink(svrRemote);}});}@Overridepublic void onRemoveStream(MediaStream mediaStream) {}@Overridepublic void onDataChannel(DataChannel dataChannel) {}@Overridepublic void onRenegotiationNeeded() {}@Overridepublic void onAddTrack(RtpReceiver rtpReceiver, MediaStream[] mediaStreams) {}});// 将 offer sdp 作为参数 setRemoteDescriptionmRemotePeerConnection.setRemoteDescription(new MySdpObserver() {@Overridepublic void onCreateSuccess(SessionDescription sessionDescription) {}@Overridepublic void onSetSuccess() {ShowLogUtil.verbose("set remote sdp success.");// 通过 PeerConnection 创建 answer,获取 sdpMediaConstraints mediaConstraints = new MediaConstraints();mRemotePeerConnection.createAnswer(new MySdpObserver() {@Overridepublic void onCreateSuccess(SessionDescription sessionDescription) {ShowLogUtil.verbose("create answer success.");// 将 answer sdp 作为参数 setLocalDescriptionmRemotePeerConnection.setLocalDescription(new MySdpObserver() {@Overridepublic void onCreateSuccess(SessionDescription sessionDescription) {}@Overridepublic void onSetSuccess() {ShowLogUtil.verbose("set local sdp success.");// 发送 answer sdpsendAnswer(sessionDescription);}}, sessionDescription);}@Overridepublic void onSetSuccess() {}}, mediaConstraints);}}, offer);}private void sendAnswer(SessionDescription answer) {receivedAnswer(answer);}private void receivedAnswer(SessionDescription answer) {// 收到 answer sdp,将 answer sdp 作为参数 setRemoteDescriptionmLocalPeerConnection.setRemoteDescription(new MySdpObserver() {@Overridepublic void onCreateSuccess(SessionDescription sessionDescription) {}@Overridepublic void onSetSuccess() {ShowLogUtil.verbose("set remote sdp success.");}}, answer);}private void sendIceCandidate(PeerConnection peerConnection, IceCandidate iceCandidate) {receivedCandidate(peerConnection, iceCandidate);}private void receivedCandidate(PeerConnection peerConnection, IceCandidate iceCandidate) {if (peerConnection == mLocalPeerConnection) {mRemotePeerConnection.addIceCandidate(iceCandidate);} else {mLocalPeerConnection.addIceCandidate(iceCandidate);}}private void hangUp() {// 关闭 PeerConnectionif (mLocalPeerConnection != null) {mLocalPeerConnection.close();mLocalPeerConnection.dispose();mLocalPeerConnection = null;}if (mRemotePeerConnection != null) {mRemotePeerConnection.close();mRemotePeerConnection.dispose();mRemotePeerConnection = null;}// 释放远端视频渲染控件SurfaceViewRenderer svrRemote = findViewById(R.id.svr_remote);svrRemote.clearImage();}
}

其中 MySdpObserver 只是 SdpObserver 的一个自定义子接口,默认实现了失败回调:

package com.qinshou.webrtcdemo_android;import org.webrtc.SdpObserver;/*** Author: MrQinshou* Email: cqflqinhao@126.com* Date: 2023/3/20 18:46* Description: 类描述*/
public interface MySdpObserver extends SdpObserver {@Overridedefault void onCreateFailure(String s) {}@Overridedefault void onSetFailure(String s) {}
}

运行起来后点击呼叫,在右上角的远端控件就能收到本地视频了:

四、iOS

相较于 Android,iOS 端的资料相对较少,且资料大同小异,我也是踩了好多坑才跑通了 demo。

1.添加依赖

对于 iOS 来说,也需要额外引入依赖,由于我是做 Android 的,对 iOS 的其他引入依赖方式不太了解,下面是使用 Cocopods 引入 WebRTC 依赖:

...
target 'WebRTCDemo-iOS' do...pod 'GoogleWebRTC','~>1.1.31999'
end
...

2.权限

除了依赖之外,iOS 还需要配置录音权限、摄像头权限,也需要动态申请,这些不是 WebRTC 的重点,只要不是 iOS 新手应该都知道怎么去做,这里就不赘述了。

<key>NSMicrophoneUsageDescription</key>
<string>允许程序使用麦克风的权限</string>
<key>NSCameraUsageDescription</key>
<string>允许程序使用摄像头的权限</string>

3.LocalDemoViewController

//
//  LocalDemoViewController.swift
//  WebRTCDemo
//
//  Created by 覃浩 on 2023/3/21.
//import UIKit
import WebRTC
import SnapKitclass LocalDemoViewController: UIViewController {private static let AUDIO_TRACK_ID = "ARDAMSa0"private static let VIDEO_TRACK_ID = "ARDAMSv0"private static let STREAM_IDS = ["ARDAMS"]private static let WIDTH = 1280private static let HEIGHT = 720private static let FPS = 30private var localView: RTCEAGLVideoView!private var remoteView: RTCEAGLVideoView!private var peerConnectionFactory: RTCPeerConnectionFactory!private var audioTrack: RTCAudioTrack?private var videoTrack: RTCVideoTrack?/**iOS 需要将 Capturer 保存为全局变量,否则无法渲染本地画面*/private var videoCapturer: RTCVideoCapturer?/**iOS 需要将远端流保存为全局变量,否则无法渲染远端画面*/private var remoteStream: RTCMediaStream?private var localPeerConnection: RTCPeerConnection?private var remotePeerConnection: RTCPeerConnection?override func viewDidLoad() {super.viewDidLoad()// 表明 View 不要扩展到整个屏幕,而是在 NavigationBar 下的区域edgesForExtendedLayout = UIRectEdge()let btnCall = UIButton()btnCall.backgroundColor = UIColor.lightGraybtnCall.setTitle("呼叫", for: .normal)btnCall.setTitleColor(UIColor.black, for: .normal)btnCall.addTarget(self, action: #selector(call), for: .touchUpInside)self.view.addSubview(btnCall)btnCall.snp.makeConstraints({ maker inmaker.left.equalToSuperview().offset(30)maker.width.equalTo(60)maker.height.equalTo(40)})let btnHangUp = UIButton()btnHangUp.backgroundColor = UIColor.lightGraybtnHangUp.setTitle("挂断", for: .normal)btnHangUp.setTitleColor(UIColor.black, for: .normal)btnHangUp.addTarget(self, action: #selector(hangUp), for: .touchUpInside)self.view.addSubview(btnHangUp)btnHangUp.snp.makeConstraints({ maker inmaker.left.equalToSuperview().offset(30)maker.width.equalTo(60)maker.height.equalTo(40)maker.top.equalTo(btnCall.snp.bottom).offset(10)})// 初始化 PeerConnectionFactoryinitPeerConnectionFactory()// 创建 EglBase// 创建 PeerConnectionFactorypeerConnectionFactory = createPeerConnectionFactory()// 创建音轨audioTrack = createAudioTrack(peerConnectionFactory: peerConnectionFactory)// 创建视轨videoTrack = createVideoTrack(peerConnectionFactory: peerConnectionFactory)let tuple = createVideoCapturer(videoSource: videoTrack!.source)let captureDevice = tuple.captureDevicevideoCapturer = tuple.videoCapture// 初始化本地视频渲染控件localView = RTCEAGLVideoView()localView.delegate = selfself.view.insertSubview(localView,at: 0)localView.snp.makeConstraints({ maker inmaker.width.equalToSuperview()maker.height.equalTo(localView.snp.width).multipliedBy(16.0/9.0)maker.centerY.equalToSuperview()})videoTrack?.add(localView!)// 初始化远端视频渲染控件remoteView = RTCEAGLVideoView()remoteView.delegate = selfself.view.insertSubview(remoteView, aboveSubview: localView)remoteView.snp.makeConstraints({ maker inmaker.width.equalTo(90)maker.height.equalTo(160)maker.top.equalToSuperview().offset(30)maker.right.equalToSuperview().offset(-30)})// 开始本地渲染(videoCapturer as? RTCCameraVideoCapturer)?.startCapture(with: captureDevice!, format: captureDevice!.activeFormat, fps: LocalDemoViewController.FPS)}override func viewDidDisappear(_ animated: Bool) {(videoCapturer as? RTCCameraVideoCapturer)?.stopCapture()videoCapturer = nillocalPeerConnection?.close()localPeerConnection = nilremotePeerConnection?.close()remotePeerConnection = nil}private func initPeerConnectionFactory() {RTCPeerConnectionFactory.initialize()}private func createPeerConnectionFactory() -> RTCPeerConnectionFactory {var videoEncoderFactory = RTCDefaultVideoEncoderFactory()var videoDecoderFactory = RTCDefaultVideoDecoderFactory()if TARGET_OS_SIMULATOR != 0 {videoEncoderFactory = RTCSimluatorVideoEncoderFactory()videoDecoderFactory = RTCSimulatorVideoDecoderFactory()}return RTCPeerConnectionFactory(encoderFactory: videoEncoderFactory, decoderFactory: videoDecoderFactory)}private func createAudioTrack(peerConnectionFactory: RTCPeerConnectionFactory) -> RTCAudioTrack {let mandatoryConstraints : [String : String] = [:]let optionalConstraints : [String : String] = [:]let audioSource = peerConnectionFactory.audioSource(with: RTCMediaConstraints(mandatoryConstraints: mandatoryConstraints, optionalConstraints: optionalConstraints))let audioTrack = peerConnectionFactory.audioTrack(with: audioSource, trackId: LocalDemoViewController.AUDIO_TRACK_ID)audioTrack.isEnabled = truereturn audioTrack}private func createVideoTrack(peerConnectionFactory: RTCPeerConnectionFactory) -> RTCVideoTrack? {let videoSource = peerConnectionFactory.videoSource()let videoTrack = peerConnectionFactory.videoTrack(with: videoSource, trackId: LocalDemoViewController.VIDEO_TRACK_ID)videoTrack.isEnabled = truereturn videoTrack}private func createVideoCapturer(videoSource: RTCVideoSource) -> (captureDevice: AVCaptureDevice?, videoCapture: RTCVideoCapturer?) {let videoCapturer = RTCCameraVideoCapturer(delegate: videoSource)let captureDevices = RTCCameraVideoCapturer.captureDevices()if (captureDevices.count == 0) {return (nil, nil)}var captureDevice: AVCaptureDevice?for c in captureDevices {// 前摄像头// if (c.position == .front) {// 后摄像头if (c.position == .back) {captureDevice = cbreak}}if (captureDevice == nil) {return (nil, nil)}return (captureDevice, videoCapturer)}@objc private func call() {// 创建 PeerConnectionlet rtcConfiguration = RTCConfiguration()var mandatoryConstraints : [String : String] = [:]var optionalConstraints : [String : String] = [:]var mediaConstraints = RTCMediaConstraints(mandatoryConstraints: mandatoryConstraints, optionalConstraints: optionalConstraints)localPeerConnection = peerConnectionFactory.peerConnection(with: rtcConfiguration, constraints: mediaConstraints, delegate: self)// 为 PeerConnection 添加音轨、视轨localPeerConnection?.add(audioTrack!, streamIds: LocalDemoViewController.STREAM_IDS)localPeerConnection?.add(videoTrack!, streamIds: LocalDemoViewController.STREAM_IDS)// 通过 PeerConnection 创建 offer,获取 sdpmandatoryConstraints = [:]optionalConstraints = [:]mediaConstraints = RTCMediaConstraints(mandatoryConstraints: mandatoryConstraints, optionalConstraints: optionalConstraints)localPeerConnection?.offer(for: mediaConstraints, completionHandler: { sessionDescription, error inShowLogUtil.verbose("create offer success.")// 将 offer sdp 作为参数 setLocalDescriptionself.localPeerConnection?.setLocalDescription(sessionDescription!, completionHandler: { _ inShowLogUtil.verbose("set local sdp success.")// 发送 offer sdpself.sendOffer(offer: sessionDescription!)})})}private func sendOffer(offer: RTCSessionDescription) {receivedOffer(offer: offer)}private func receivedOffer(offer: RTCSessionDescription) {// 创建 PeerConnectionlet rtcConfiguration = RTCConfiguration()let mandatoryConstraints : [String : String] = [:]let optionalConstraints : [String : String] = [:]var mediaConstraints = RTCMediaConstraints(mandatoryConstraints: mandatoryConstraints, optionalConstraints: optionalConstraints)remotePeerConnection = peerConnectionFactory.peerConnection(with: rtcConfiguration, constraints: mediaConstraints, delegate: self)// 将 offer sdp 作为参数 setRemoteDescriptionremotePeerConnection?.setRemoteDescription(offer, completionHandler: { _ inShowLogUtil.verbose("set remote sdp success.")// 通过 PeerConnection 创建 answer,获取 sdplet mandatoryConstraints : [String : String] = [:]let optionalConstraints : [String : String] = [:]let mediaConstraints = RTCMediaConstraints(mandatoryConstraints: mandatoryConstraints, optionalConstraints: optionalConstraints)self.remotePeerConnection?.answer(for: mediaConstraints, completionHandler: { sessionDescription, error inShowLogUtil.verbose("create answer success.")// 将 answer sdp 作为参数 setLocalDescriptionself.remotePeerConnection?.setLocalDescription(sessionDescription!, completionHandler: { _ inShowLogUtil.verbose("set local sdp success.")// 发送 answer sdpself.sendAnswer(answer: sessionDescription!)})})})}private func sendAnswer(answer: RTCSessionDescription) {receivedAnswer(answer: answer)}private func receivedAnswer(answer: RTCSessionDescription) {// 收到 answer sdp,将 answer sdp 作为参数 setRemoteDescriptionlocalPeerConnection?.setRemoteDescription(answer, completionHandler: { _ in   ShowLogUtil.verbose("set remote sdp success.")})}private func sendIceCandidate(peerConnection: RTCPeerConnection, iceCandidate: RTCIceCandidate)  {receivedCandidate(peerConnection: peerConnection,iceCandidate: iceCandidate)}private func receivedCandidate(peerConnection: RTCPeerConnection, iceCandidate: RTCIceCandidate) {if (peerConnection == localPeerConnection) {remotePeerConnection?.add(iceCandidate)} else {localPeerConnection?.add(iceCandidate)}}@objc private func hangUp() {// 关闭 PeerConnectionlocalPeerConnection?.close()localPeerConnection = nilremotePeerConnection?.close()remotePeerConnection = nil// 释放远端视频渲染控件if let track = remoteStream?.videoTracks.first {track.remove(remoteView!)}}
}// MARK: - RTCVideoViewDelegate
extension LocalDemoViewController: RTCVideoViewDelegate {func videoView(_ videoView: RTCVideoRenderer, didChangeVideoSize size: CGSize) {}
}// MARK: - RTCPeerConnectionDelegate
extension LocalDemoViewController: RTCPeerConnectionDelegate {func peerConnection(_ peerConnection: RTCPeerConnection, didChange stateChanged: RTCSignalingState) {}func peerConnection(_ peerConnection: RTCPeerConnection, didAdd stream: RTCMediaStream) {ShowLogUtil.verbose("peerConnection didAdd stream--->\(stream)")if (peerConnection == self.localPeerConnection) {} else if (peerConnection == self.remotePeerConnection) {self.remoteStream = streamif let track = stream.videoTracks.first {track.add(remoteView!)}if let audioTrack = stream.audioTracks.first{audioTrack.source.volume = 8}}}func peerConnection(_ peerConnection: RTCPeerConnection, didRemove stream: RTCMediaStream) {}func peerConnectionShouldNegotiate(_ peerConnection: RTCPeerConnection) {}func peerConnection(_ peerConnection: RTCPeerConnection, didChange newState: RTCIceConnectionState) {}func peerConnection(_ peerConnection: RTCPeerConnection, didChange newState: RTCIceGatheringState) {}func peerConnection(_ peerConnection: RTCPeerConnection, didGenerate candidate: RTCIceCandidate) {ShowLogUtil.verbose("didGenerate candidate--->\(candidate)")self.sendIceCandidate(peerConnection: peerConnection, iceCandidate: candidate)}func peerConnection(_ peerConnection: RTCPeerConnection, didRemove candidates: [RTCIceCandidate]) {}func peerConnection(_ peerConnection: RTCPeerConnection, didOpen dataChannel: RTCDataChannel) {}
}

运行起来后点击呼叫,在右上角的远端控件就能收到本地视频了:

Gif 的质量就包涵一下,压缩后实在是失真有点严重。大家可能会看到 iOS 中间会卡了一下,那其实不是卡了,因为 iOS 录屏录不了点击事件,实际上是跟 H5 和 Android 一样,我中间点击了一下挂断,但是 iOS 挂断后释放远端渲染控件时,控件会保留显示最后一帧的画面,所以大家如果在实际做的时候想要效果好一点的话,可以在渲染控件上再放一个黑色遮罩,挂断时显示遮罩。

五、总结

踩过的坑,其实在代码中已经有注释了,大家留意一下。

Android 端的坑:

  1. SurfaceViewRender 对象需要调用 init() 方法初始化,否则会黑屏。

iOS 端的坑:

  1. 本地画面渲染控件就老老实实的用 RTCEAGLVideoView 就好,不要用 RTCCameraPreviewView,跟远端渲染控件保持一致,代码更整齐,使用其他视频源时还容易切换。
  2. 本地视频捕获对象(RTCVideoCapturer)、视轨对象(RTCVideoTrack)、远端视频流对象(RTCMediaStream)要声明成全局变量,我也不知道为什么,但是如果声明成局部变量就会渲染不出来画面。
  3. demo 阶段不要去设置花里胡哨的参数,(DtlsSrtpKeyAgreement、kRTCMediaConstraintsOfferToReceiveAudio、kRTCMediaConstraintsOfferToReceiveVideo 等),均使用默认配置就好,设置参数越多越不容易排查问题。

通过这三端的本地 demo,大家应该还是可以看到一定区别,H5 其实很少的代码量就能实现 WebRTC 功能,而 Android 相对来说需要初始化的地方更多,但可能因为是我的本行的缘故,所以我在实现 Android 端的时候并没有遇到什么坑。

其实一端与另一端通话是只需要创建一个 PeerConnection ,这一次本地 demo 中我们创建了两个 PeerConnection,这是为了不搭建信令服务器就能看到效果,所以创建了一个 RemotePeerConnection 模拟远端,下一次我们就会通过 WebSocket 搭建一个简单的信令服务器来实现点对点通话。

六、Demo

Demo 传送门

本文链接:https://my.lmcjl.com/post/5614.html

展开阅读全文

4 评论

留下您的评论.