经验首页 前端设计 程序设计 Java相关 移动开发 数据库/运维 软件/图像 大数据/云计算 其他经验
当前位置:技术经验 » JS/JS库/框架 » JavaScript » 查看文章
WebRTC入门
来源:cnblogs  作者:ggtc  时间:2024/7/8 9:59:42  对本文有异议

效果展示

image

基础概念

  • WebRTC指的是基于web的实时视频通话,其实就相当于A->B发直播画面,同时B->A发送直播画面,这样就是视频聊天了
  • WebRTC的视频通话是A和B两两之间进行的
  • WebRTC通话双方通过一个公共的中心服务器找到对方,就像聊天室一样
  • WebRTC的连接过程一般是
    1. A通过websocket连接下中心服务器,B通过websocket连接下中心服务器。每次有人加入或退出中心服务器,中心服务器就把为维护的连接广播给A和B
    2. A接到广播知道了B的存在,A发起提案,传递视频编码器等参数,让中心服务器转发给B。B收到中心服务器转发的A的提案,创建回答,传递视频编码器等参数,让中心服务器转发给A
    3. A收到回答,发起交互式连接,包括自己的地址,端口等,让中心服务器转发给B。B收到连接,回答交互式连接,包括自己的地址,端口等,让中心服务器转发给A。
    4. 至此A知道了B的地址,B知道了A的地址,连接建立,中心服务器退出整个过程
    5. A给B推视频流,同时B给A推视频流。双方同时用video元素把对方的视频流播放出来

API

  • WebSokcet 和中心服务器的连接,中心服务器也叫信令服务器,用来建立连接前中转消息,相当于相亲前的媒人

  • RTCPeerConnection 视频通话连接

  • rc.createOffer 发起方创建本地提案,获得SDP描述

  • rc.createAnswer 接收方创建本地回答,获得SDP描述

  • rc.setLocalDescription 设置本地创建的SDP描述

  • rc.setRemoteDescription 设置对方传递过来的SDP描述

  • rc.onicecandidate 在创建本地提案会本地回答时触发此事件,获得交互式连接对象,用于发送给对方

  • rc.addIceCandidate 设置中心服务器转发过来IceCandidate

  • rc.addStream 向连接中添加媒体流

  • rc.addTrack 向媒体流中添加轨道

  • rc.ontrack 在此事件中接受来自对方的媒体流

其实两个人通信只需要一个RTCPeerConnection,A和B各持一端,不需要两个RTCPeerConnection,这点容易被误导

媒体流

获取

这里我获取的是窗口视频流,而不是摄像头视频流

  1. navigator.mediaDevices.getDisplayMedia()
  2. .then(meStream => {
  3. //在本地显示预览
  4. document.getElementById("local").srcObject = meStream;
  5. })

传输

  1. //给对方发送视频流
  2. other.stream = meStream;
  3. const videoTracks = meStream.getVideoTracks();
  4. const audioTracks = meStream.getAudioTracks();
  5. //log("推流")
  6. other.peerConnection.addStream(meStream);
  7. meStream.getVideoTracks().forEach(track => {
  8. other.peerConnection.addTrack(track, meStream);
  9. });

接收

  1. other.peerConnection.addEventListener("track", event => {
  2. //log("拉流")
  3. document.getElementById("remote").srcObject = event.streams[0];
  4. })

连接

WebSocet连接

这是最开始需要建立的和信令服务器的连接,用于点对点连接建立前转发消息,这算是最重要的逻辑了

  1. ws = new WebSocket('/sdp');
  2. ws.addEventListener("message", event => {
  3. var msg = JSON.parse(event.data);
  4. if (msg.type == "connect") {
  5. //log("接到提案");
  6. var other = remotes.find(r => r.name != myName);
  7. onReciveOffer(msg.data.description, msg.data.candidate, other);
  8. }
  9. else if (msg.type == "connected") {
  10. //log("接到回答");
  11. var other = remotes.find(r => r.name != myName);
  12. onReciveAnwer(msg.data.description, msg.data.candidate, other);
  13. }
  14. //获取自己在房间中的临时名字
  15. else if (msg.type == "id") {
  16. myName = msg.data;
  17. }
  18. //有人加入或退出房间时
  19. else if (msg.type == "join") {
  20. //成员列表
  21. for (var i = 0; i < msg.data.length; i++) {
  22. var other = remotes.find(r => r.name == msg.data[i]);
  23. if (other == null) {
  24. remotes.push({
  25. stream: null,
  26. peerConnection: new RTCPeerConnection(null),
  27. description: null,
  28. candidate: null,
  29. video: null,
  30. name: msg.data[i]
  31. });
  32. }
  33. }
  34. //过滤已经离开的人
  35. remotes = remotes.filter(r => msg.data.find(x => x == r.name) != null);
  36. //...
  37. }
  38. });

RTCPeerConnection连接

在都已经加入聊天室后就可以开始建立点对点连接了

  1. //对某人创建提案
  2. other.peerConnection.createOffer({ offerToReceiveVideo: 1 })
  3. .then(description => {
  4. //设置成自己的本地描述
  5. other.description = description;
  6. other.peerConnection.setLocalDescription(description);
  7. });

在创建提案后会触发此事件,然后把提案和交互式连接消息一起发送出去

  1. //交互式连接候选项
  2. other.peerConnection.addEventListener("icecandidate", event => {
  3. other.candidate = event.candidate;
  4. //log("发起提案");
  5. //发送提案到中心服务器
  6. ws.send(JSON.stringify({
  7. type: "connect",
  8. data: {
  9. name: other.name,
  10. description: other.description,
  11. candidate: other.candidate
  12. }
  13. }));
  14. })

对方收到提案后按照同样的流程创建回答和响应

  1. /**接收到提案 */
  2. function onReciveOffer(description, iceCandidate,other) {
  3. //交互式连接候选者
  4. other.peerConnection.addEventListener("icecandidate", event => {
  5. other.candidate = event.candidate;
  6. //log("发起回答");
  7. //回答信令到中心服务器
  8. ws.send(JSON.stringify({
  9. type: "connected",
  10. data: {
  11. name: other.name,
  12. description: other.description,
  13. candidate: other.candidate
  14. }
  15. }));
  16. })
  17. //设置来自对方的远程描述
  18. other.peerConnection.setRemoteDescription(description);
  19. other.peerConnection.addIceCandidate(new RTCIceCandidate(iceCandidate));
  20. other.peerConnection.createAnswer()
  21. .then(answerDescription => {
  22. other.description = answerDescription;
  23. other.peerConnection.setLocalDescription(answerDescription);
  24. })
  25. }

发起方收到回答后,点对点连接建立,双方都能看到画面了,至此已经不需要中心服务器了

  1. /**接收到回答 */
  2. function onReciveAnwer(description, iceCandidate,other) {
  3. //收到回答后设置接收方的描述
  4. other.peerConnection.setRemoteDescription(description);
  5. other.peerConnection.addIceCandidate(new RTCIceCandidate(iceCandidate));
  6. }

完整代码

SDPController.cs
  1. [ApiController]
  2. [Route("sdp")]
  3. public class SDPController : Controller
  4. {
  5. public static List<(string name, WebSocket ws)> clients = new List<(string, WebSocket)>();
  6. private List<string> names = new List<string>() { "张三", "李四", "王五","钟鸣" };
  7. [HttpGet("")]
  8. public async Task Index()
  9. {
  10. WebSocket client = await HttpContext.WebSockets.AcceptWebSocketAsync();
  11. var ws = (name:names[clients.Count], client);
  12. clients.Add(ws);
  13. await client.SendAsync(UTF8Encoding.UTF8.GetBytes(JsonConvert.SerializeObject(new {type="id",data=ws.name})), WebSocketMessageType.Text, true, CancellationToken.None);
  14. List<string> list = new List<string>();
  15. foreach (var person in clients)
  16. {
  17. list.Add(person.name);
  18. }
  19. var join = new
  20. {
  21. type = "join",
  22. data = list,
  23. };
  24. foreach (var item in clients)
  25. {
  26. await item.ws.SendAsync(UTF8Encoding.UTF8.GetBytes(JsonConvert.SerializeObject(join)), WebSocketMessageType.Text, true, CancellationToken.None);
  27. }
  28. var defaultBuffer = new byte[40000];
  29. try
  30. {
  31. while (!client.CloseStatus.HasValue)
  32. {
  33. //接受信令
  34. var result = await client.ReceiveAsync(defaultBuffer, CancellationToken.None);
  35. JObject obj=JsonConvert.DeserializeObject<JObject>(UTF8Encoding.UTF8.GetString(defaultBuffer,0,result.Count));
  36. if (obj.Value<string>("type")=="connect" || obj.Value<string>("type") == "connected")
  37. {
  38. var another = clients.FirstOrDefault(r => r.name == obj["data"].Value<string>("name"));
  39. await another.ws.SendAsync(new ArraySegment<byte>(defaultBuffer, 0, result.Count), result.MessageType, result.EndOfMessage, CancellationToken.None);
  40. }
  41. }
  42. }
  43. catch (Exception e)
  44. {
  45. }
  46. Console.WriteLine("退出");
  47. clients.Remove(ws);
  48. list = new List<string>();
  49. foreach (var person in clients)
  50. {
  51. list.Add(person.name);
  52. }
  53. join = new
  54. {
  55. type = "join",
  56. data = list
  57. };
  58. foreach (var item in clients)
  59. {
  60. await item.ws.SendAsync(UTF8Encoding.UTF8.GetBytes(JsonConvert.SerializeObject(join)), WebSocketMessageType.Text, true, CancellationToken.None);
  61. }
  62. }
  63. }
home.html
  1. <!DOCTYPE html>
  2. <html>
  3. <head>
  4. <meta charset="utf-8" />
  5. <title></title>
  6. <style>
  7. html,body{
  8. height:100%;
  9. margin:0;
  10. }
  11. .container{
  12. display:grid;
  13. grid-template:auto 1fr 1fr/1fr 200px;
  14. height:100%;
  15. grid-gap:8px;
  16. justify-content:center;
  17. align-items:center;
  18. }
  19. .video {
  20. background-color: black;
  21. height:calc(100% - 1px);
  22. overflow:auto;
  23. }
  24. #local {
  25. grid-area:2/1/3/2;
  26. }
  27. #remote {
  28. grid-area: 3/1/4/2;
  29. }
  30. .list{
  31. grid-area:1/2/4/3;
  32. background-color:#eeeeee;
  33. height:100%;
  34. overflow:auto;
  35. }
  36. #persons{
  37. text-align:center;
  38. }
  39. .person{
  40. padding:5px;
  41. }
  42. </style>
  43. </head>
  44. <body>
  45. <div class="container">
  46. <div style="grid-area:1/1/2/2;padding:8px;">
  47. <button id="start">录制本地窗口</button>
  48. <button id="call">发起远程</button>
  49. <button id="hangup">挂断远程</button>
  50. </div>
  51. <video autoplay id="local" class="video"></video>
  52. <video autoplay id="remote" class="video"></video>
  53. <div class="list">
  54. <div style="text-align:center;background-color:white;padding:8px;">
  55. <button id="join">加入</button>
  56. <button id="exit">退出</button>
  57. </div>
  58. <div id="persons">
  59. </div>
  60. <div id="log">
  61. </div>
  62. </div>
  63. </div>
  64. <script>
  65. /**在屏幕顶部显示一条消息,3秒后消失 */
  66. function layerMsg(msg) {
  67. // 创建一个新的div元素作为消息层
  68. var msgDiv = document.createElement('div');
  69. msgDiv.textContent = msg;
  70. // 设置消息层的样式
  71. msgDiv.style.position = 'fixed';
  72. msgDiv.style.top = '0';
  73. msgDiv.style.left = '50%';
  74. msgDiv.style.transform = 'translateX(-50%)';
  75. msgDiv.style.background = '#f2f2f2';
  76. msgDiv.style.color = '#333';
  77. msgDiv.style.padding = '10px';
  78. msgDiv.style.borderBottom = '2px solid #ccc';
  79. msgDiv.style.width = '100%';
  80. msgDiv.style.textAlign = 'center';
  81. msgDiv.style.zIndex = '9999'; // 确保消息层显示在最顶层
  82. // 将消息层添加到文档的body中
  83. document.body.appendChild(msgDiv);
  84. // 使用setTimeout函数,在3秒后移除消息层
  85. setTimeout(function () {
  86. document.body.removeChild(msgDiv);
  87. }, 3000);
  88. }
  89. function log(msg) {
  90. document.getElementById("log").innerHTML += `<div>${msg}</div>`;
  91. }
  92. </script>
  93. <script>
  94. var myName = null;
  95. // 服务器配置
  96. const servers = null;
  97. var remotes = [];
  98. var startButton = document.getElementById("start");
  99. var callButton = document.getElementById("call");
  100. var hangupButton = document.getElementById("hangup");
  101. var joinButton = document.getElementById("join");
  102. var exitButton = document.getElementById("exit");
  103. startButton.disabled = false;
  104. callButton.disabled = false;
  105. hangupButton.disabled = true;
  106. joinButton.disabled = false;
  107. exitButton.disabled = true;
  108. /**和中心服务器的连接,用于交换信令 */
  109. var ws;
  110. //加入房间
  111. document.getElementById("join").onclick = function () {
  112. ws = new WebSocket('/sdp');
  113. ws.addEventListener("message", event => {
  114. var msg = JSON.parse(event.data);
  115. if (msg.type == "offer") {
  116. log("接收到offer");
  117. onReciveOffer(msg);
  118. }
  119. else if (msg.type == "answer") {
  120. log("接收到answer");
  121. onReciveAnwer(msg);
  122. }
  123. else if (msg.candidate != undefined) {
  124. layerMsg("接收到candidate");
  125. onReciveIceCandidate(msg);
  126. }
  127. else if (msg.type == "connect") {
  128. log("接到提案");
  129. var other = remotes.find(r => r.name != myName);
  130. onReciveOffer(msg.data.description, msg.data.candidate, other);
  131. }
  132. else if (msg.type == "connected") {
  133. log("接到回答");
  134. var other = remotes.find(r => r.name != myName);
  135. onReciveAnwer(msg.data.description, msg.data.candidate, other);
  136. }
  137. else if (msg.type == "id") {
  138. myName = msg.data;
  139. }
  140. else if (msg.type == "join") {
  141. //新增
  142. for (var i = 0; i < msg.data.length; i++) {
  143. var other = remotes.find(r => r.name == msg.data[i]);
  144. if (other == null) {
  145. remotes.push({
  146. stream: null,
  147. peerConnection: new RTCPeerConnection(servers),
  148. description: null,
  149. candidate: null,
  150. video: null,
  151. name: msg.data[i]
  152. });
  153. }
  154. }
  155. //过滤已经离开的人
  156. remotes = remotes.filter(r => msg.data.find(x => x == r.name) != null);
  157. document.getElementById("persons").innerHTML = "";
  158. for (var i = 0; i < remotes.length; i++) {
  159. var div = document.createElement("div");
  160. div.classList.add("person")
  161. var btn = document.createElement("button");
  162. btn.innerText = remotes[i].name;
  163. if (remotes[i].name == myName) {
  164. btn.innerText += "(我)";
  165. }
  166. div.appendChild(btn);
  167. document.getElementById("persons").appendChild(div);
  168. }
  169. }
  170. });
  171. startButton.disabled = false;
  172. joinButton.disabled = true;
  173. exitButton.disabled = false;
  174. }
  175. //退出房间
  176. document.getElementById("exit").onclick = function () {
  177. if (ws != null) {
  178. ws.close();
  179. ws = null;
  180. startButton.disabled = true;
  181. callButton.disabled = true;
  182. hangupButton.disabled = true;
  183. joinButton.disabled = false;
  184. exitButton.disabled = true;
  185. document.getElementById("persons").innerHTML = "";
  186. remotes = [];
  187. local.peerConnection = null;
  188. local.candidate = null;
  189. local.description = null;
  190. local.stream = null;
  191. local.video = null;
  192. }
  193. }
  194. //推流
  195. startButton.onclick = function () {
  196. var local = remotes.find(r => r.name == myName);
  197. var other = remotes.find(r => r.name != myName);
  198. if (other == null) {
  199. return;
  200. }
  201. navigator.mediaDevices.getDisplayMedia()
  202. .then(meStream => {
  203. //在本地显示预览
  204. document.getElementById("local").srcObject = meStream;
  205. //给对方发送视频流
  206. other.stream = meStream;
  207. const videoTracks = meStream.getVideoTracks();
  208. const audioTracks = meStream.getAudioTracks();
  209. log("推流")
  210. other.peerConnection.addStream(meStream);
  211. meStream.getVideoTracks().forEach(track => {
  212. other.peerConnection.addTrack(track, meStream);
  213. });
  214. })
  215. }
  216. callButton.onclick = function () {
  217. callButton.disabled = true;
  218. hangupButton.disabled = false;
  219. var other = remotes.find(r => r.name != myName);
  220. //交互式连接候选者
  221. other.peerConnection.addEventListener("icecandidate", event => {
  222. if (event.candidate == null) {
  223. return;
  224. }
  225. other.candidate = event.candidate;
  226. log("发起提案");
  227. //发送提案到中心服务器
  228. ws.send(JSON.stringify({
  229. type: "connect",
  230. data: {
  231. name: other.name,
  232. description: other.description,
  233. candidate: other.candidate
  234. }
  235. }));
  236. })
  237. other.peerConnection.addEventListener("track", event => {
  238. log("拉流")
  239. document.getElementById("remote").srcObject = event.streams[0];
  240. })
  241. //对某人创建信令
  242. other.peerConnection.createOffer({ offerToReceiveVideo: 1 })
  243. .then(description => {
  244. //设置成自己的本地描述
  245. other.description = description;
  246. other.peerConnection.setLocalDescription(description);
  247. })
  248. .catch(e => {
  249. debugger
  250. });
  251. }
  252. //挂断给对方的流
  253. hangupButton.onclick = function () {
  254. callButton.disabled = false;
  255. hangupButton.disabled = true;
  256. var local = remotes.find(r => r.name == myName);
  257. var other = remotes.find(r => r.name != myName);
  258. other.peerConnection = new RTCPeerConnection(servers);
  259. other.description = null;
  260. other.candidate = null;
  261. other.stream = null;
  262. }
  263. /**接收到回答 */
  264. function onReciveAnwer(description, iceCandidate,other) {
  265. if (other == null) {
  266. return;
  267. }
  268. //收到回答后设置接收方的描述
  269. other.peerConnection.setRemoteDescription(description)
  270. .catch(e => {
  271. debugger
  272. });
  273. other.peerConnection.addIceCandidate(new RTCIceCandidate(iceCandidate));
  274. }
  275. /**接收到提案 */
  276. function onReciveOffer(description, iceCandidate,other) {
  277. //交互式连接候选者
  278. other.peerConnection.addEventListener("icecandidate", event => {
  279. if (event.candidate == null) {
  280. return;
  281. }
  282. other.candidate = event.candidate;
  283. log("发起回答");
  284. //回答信令到中心服务器
  285. ws.send(JSON.stringify({
  286. type: "connected",
  287. data: {
  288. name: other.name,
  289. description: other.description,
  290. candidate: other.candidate
  291. }
  292. }));
  293. })
  294. other.peerConnection.addEventListener("track", event => {
  295. log("拉流")
  296. document.getElementById("remote").srcObject = event.streams[0];
  297. })
  298. //设置来自对方的远程描述
  299. other.peerConnection.setRemoteDescription(description)
  300. .catch(e => {
  301. debugger
  302. });
  303. other.peerConnection.addIceCandidate(new RTCIceCandidate(iceCandidate));
  304. other.peerConnection.createAnswer()
  305. .then(answerDescription => {
  306. other.description = answerDescription;
  307. other.peerConnection.setLocalDescription(answerDescription);
  308. })
  309. }
  310. function onReciveIceCandidate(iceCandidate) {
  311. if (remotePeerConnection == null) {
  312. return;
  313. }
  314. remotePeerConnection.addIceCandidate(new RTCIceCandidate(iceCandidate));
  315. }
  316. </script>
  317. </body>
  318. </html>

原文链接:https://www.cnblogs.com/ggtc/p/18287925

 友情链接:直通硅谷  点职佳  北美留学生论坛

本站QQ群:前端 618073944 | Java 606181507 | Python 626812652 | C/C++ 612253063 | 微信 634508462 | 苹果 692586424 | C#/.net 182808419 | PHP 305140648 | 运维 608723728

W3xue 的所有内容仅供测试,对任何法律问题及风险不承担任何责任。通过使用本站内容随之而来的风险与本站无关。
关于我们  |  意见建议  |  捐助我们  |  报错有奖  |  广告合作、友情链接(目前9元/月)请联系QQ:27243702 沸活量
皖ICP备17017327号-2 皖公网安备34020702000426号