「Unity/Agora/voicechat community/デモ画面共有」の版間の差分
提供: 初心者エンジニアの簡易メモ
(→デモ画面共有コード) |
(→左右反転対応) |
||
(同じ利用者による、間の5版が非表示) | |||
行66: | 行66: | ||
エラー詳細 | エラー詳細 | ||
AgoraRTCException: AgoraRTCError UNEXPECTED_ERROR: SecurityError: Failed to execute 'getDisplayMedia' on 'MediaDevices': Access to the feature "display-capture" is disallowed by permission policy. | AgoraRTCException: AgoraRTCError UNEXPECTED_ERROR: SecurityError: Failed to execute 'getDisplayMedia' on 'MediaDevices': Access to the feature "display-capture" is disallowed by permission policy. | ||
− | chrome | + | chrome ver94以降だとこのエラーが出るっぽい。iframeを使用しないようにするか、iframeタグないに、以下属性をつけるかすればよいっぽい |
+ | |||
+ | <iframe frameborder="0" allow="camera *; geolocation *; microphone *; autoplay *" class="openctiSoftPhone" dataauraclass="openctiSoftPhone"></iframe> | ||
参考:https://trailblazer.salesforce.com/issues_view?id=a1p4V0000029erUQAQ&title=domexception-failed-to-execute-getdisplaymedia-on-mediadevices-access-to-the-feature-display-capture-is-disallowed-by-permission-policy | 参考:https://trailblazer.salesforce.com/issues_view?id=a1p4V0000029erUQAQ&title=domexception-failed-to-execute-getdisplaymedia-on-mediadevices-access-to-the-feature-display-capture-is-disallowed-by-permission-policy | ||
行481: | 行483: | ||
} | } | ||
</pre> | </pre> | ||
+ | =====動画が出ない場合===== | ||
+ | 以下のコードを使ってる場合は、"VideoCanvas"名のGameObjectをヒエラルキーに設置してるか確認。 | ||
+ | GameObject.Find("VideoCanvas"); | ||
+ | |||
+ | ====チャンネルのClientRoleコールバック==== | ||
+ | <pre> | ||
+ | channel1.ChannelOnClientRoleChanged = HandleOnClientRoleChanged; | ||
+ | channel1.ChannelOnClientRoleChangeFailed = OnClientRoleChangeFailedHandler; | ||
+ | void HandleOnClientRoleChanged(string channelId, CLIENT_ROLE_TYPE oldRole, CLIENT_ROLE_TYPE newRole) | ||
+ | { | ||
+ | Debug.Log("Engine OnClientRoleChanged: " + oldRole + " -> " + newRole); | ||
+ | } | ||
+ | void OnClientRoleChangeFailedHandler(string channelId, CLIENT_ROLE_CHANGE_FAILED_REASON reason, CLIENT_ROLE_TYPE currentRole) | ||
+ | { | ||
+ | Debug.Log("Engine OnClientRoleChangeFaile: " + reason + " c-> " + currentRole); | ||
+ | } | ||
+ | </pre> | ||
+ | ====左右反転対応==== | ||
+ | <pre> | ||
+ | VideoSurface videoSurface = makeImageSurface(objName); | ||
+ | videoSurface.EnableFilpTextureApply(true, false); | ||
+ | </pre> | ||
+ | ==画面共有の音声がオフにできない問題== | ||
+ | コミュニティの質問 | ||
+ | |||
+ | https://github.com/AgoraIO-Community/Agora_Unity_WebGL/discussions/205 |
2023年1月23日 (月) 02:29時点における最新版
目次
デモから画面共有サンプル作成
DevDemo側サンプルソース解析
- SceneFuncTests.unityのApi_testHelperのGameObjectのFunctionalTest/DevDemo/Test/DVC_ShareScreen.csが、画面共有ソース
- TestHome.csのonJoinButtonClicked()からシーン移動している
- FunctionalTest/DevDemo/SceneHome2のボタンで、移動できる。
以下"SceneFuncTests"がロードできないというエラーとなる場合
以下エラーとなる場合
'SceneFuncTests' couldn't be loaded because it has not been added to the build settings or the AssetBundle has not been loaded.
To add a scene to the build settings use the menu File->Build Settings... PlayerSettingのSceneInBuildにSceneFuncTests.unityを追加すると良い。
以下"SceneFuncTests"がロードできないというエラーとなる場合
以下エラーとなる場合
clientmanager.js:862 Note this API should be replaced by startScreenCaptureForWeb instead.
Agora_Unity_WebGL/[build_path]/AgoraWebSDK/libs/clientmanager.js
startScreenCaptureByDisplayId()はNGで、startScreenCaptureForWebに置き換えるようなエラーが出る。
以下の方だとstartScreenCaptureForWebを、実行するサンプルがあるのでそちらを確認する。
- Assets/FunctionalTest/NewScreenShareClientManager/AgoraClientManager.cs
- Assets/FunctionalTest/NewScreenShareMChannel/AgoraMultiChannel2.cs
- StartScreenButtonを押す
- ClientManagerTestシーンは、背景のぼかしオン・オフができる。配信前に画面確認ができない。
- MainScreenNewシーンは、背景のぼかしオンができる。オフができない。配信前に画面確認ができる。
- NewScreenShareClientチェックは、画面共有時に、動画配信とは別のwindowを開く感じ。その際、画面共有接続には、SCREEN_SHARE_IDが使われる。
- ドラッグ移動するコード。makeVideoView()とmakeImageSurface()ののどちらかが.AddComponent<UIElementDragger>();されてること。
"PERMISSION_DENIED"エラーが出た場合
エラー詳細
AgoraRTCException: AgoraRTCError PERMISSION_DENIED: NotAllowedError: Permission denied by system
- macの場合は、macの設定/セキュリティとプライバシー/プライバシー/画面収録/Chromeをonに
配信元画面の大きさを変更
videoSurface.GetComponent<RawImage>().rectTransform.sizeDelta = v2 * 2f;
配信先画面の大きさを変更
if (remote) { Vector2 v2 = AgoraUIUtils.GetScaledDimension(640, 360, EnforcingViewLength); videoSurface.GetComponent<RawImage>().rectTransform.sizeDelta = v2 * 2f; remoteUserDisplays.Add(videoSurface.gameObject); UserVideoDict[uid] = videoSurface; }
ハンドラ
channel1.ChannelOnJoinChannelSuccess = Channel1OnJoinChannelSuccessHandler; // 自分が接続開始 channel1.ChannelOnLeaveChannel = Channel1OnLeaveChannelHandler; // 自分が退出時 channel1.ChannelOnUserJoined = Channel1OnUserJoinedHandler; // 別ユーザが接続してきたとき channel1.ChannelOnError = Channel1OnErrorHandler; // エラー時 channel1.ChannelOnUserOffLine = ChannelOnUserOfflineHandler; // 別ユーザーの接続が終了したとき channel1.ChannelOnScreenShareStarted = screenShareStartedHandler_MC; // 自分が映像開始時 channel1.ChannelOnScreenShareStopped = screenShareStoppedHandler_MC; // 自分が映像停止時 channel1.ChannelOnScreenShareCanceled = screenShareCanceledHandler_MC; // 自分が映像キャンセル時 channel1.ChannelOnVideoSizeChanged = onVideoSizeChanged_MCHandler; // 映像サイズ変更時
映像の背景ぼやかす
AgoraMultiChannel2のenableVirtualBackground()を動作させる。
"getDisplayMedia"のセキュリティエラーが出る場合
エラー詳細
AgoraRTCException: AgoraRTCError UNEXPECTED_ERROR: SecurityError: Failed to execute 'getDisplayMedia' on 'MediaDevices': Access to the feature "display-capture" is disallowed by permission policy.
chrome ver94以降だとこのエラーが出るっぽい。iframeを使用しないようにするか、iframeタグないに、以下属性をつけるかすればよいっぽい
<iframe frameborder="0" allow="camera *; geolocation *; microphone *; autoplay *" class="openctiSoftPhone" dataauraclass="openctiSoftPhone"></iframe>
デモ画面共有コード
MainSceneNew.sceneを適当なsceneへコピーし、JoinChannel、LeaveChannel、StartScreenShare、StopScreenShareだけのこして、以下のcsをVideoCanvasに設置
using System.Collections; using System.Collections.Generic; using UnityEngine; using UnityEngine.UI; using agora_gaming_rtc; using agora_utilities; public class DisplayShareScene : MonoBehaviour { [SerializeField] private string APP_ID = "YOUR_APPID"; [SerializeField] private string TOKEN_1 = ""; [SerializeField] private string CHANNEL_NAME_1 = "YOUR_CHANNEL_NAME_1"; public Text logText; private Logger logger; private IRtcEngine mRtcEngine = null; private AgoraChannel channel1 = null; public Button startScreenShareButton, stopScreenShareButton; public bool useNewScreenShare = false; public bool useScreenShareAudio = false; public Button[] joinChannelButtons, leaveChannelButtons; protected Dictionary<uint, VideoSurface> UserVideoDict = new Dictionary<uint, VideoSurface>(); private List<GameObject> remoteUserDisplays = new List<GameObject>(); public VirtualBackgroundSource myVirtualBackground; public int blurDegrees = 2; public string hexColor = "#00FF00"; public string imgFile = "seinfeld.jpg"; public string videoFile = "movie.mp4"; // Use this for initialization void Start() { if (!CheckAppId()) { return; } joinChannelButtons[0].onClick.AddListener(JoinChannel1); leaveChannelButtons[0].onClick.AddListener(LeaveChannel1); InitEngine(); //channel setup. updateScreenShareNew(); } public void updateScreenShareNew() { startScreenShareButton.onClick.AddListener(delegate { startScreenShare2(useScreenShareAudio); }); stopScreenShareButton.onClick.AddListener(delegate { stopScreenShare2(); }); } void Update() { PermissionHelper.RequestMicrophontPermission(); PermissionHelper.RequestCameraPermission(); } bool CheckAppId() { logger = new Logger(logText); logger.DebugAssert(APP_ID.Length > 10, "Please fill in your appId in VideoCanvas!!!!!"); return (APP_ID.Length > 10); } //for starting/stopping a screen share through AgoraChannel class. public void startScreenShare2(bool audioEnabled) { channel1.StartScreenCaptureForWeb(audioEnabled); } public void stopScreenShare2() { channel1.StopScreenCapture(); } public void enableVirtualBackground() { channel1.enableVirtualBackground(true, myVirtualBackground); } public void setVirtualBackgroundBlur() { mRtcEngine.SetVirtualBackgroundBlur_MC(blurDegrees); } public void setVirtualBackgroundColor() { mRtcEngine.SetVirtualBackgroundColor_MC(hexColor); } public void setVirtualBackgroundImage() { mRtcEngine.SetVirtualBackgroundImage_MC(imgFile); } public void setVirtualBackgroundVideo() { mRtcEngine.SetVirtualBackgroundVideo_MC(videoFile); } void InitEngine() { mRtcEngine = IRtcEngine.GetEngine(APP_ID); mRtcEngine.SetChannelProfile(CHANNEL_PROFILE.CHANNEL_PROFILE_LIVE_BROADCASTING); // If you want to user Multi Channel Video, please call "SetMultiChannleWant to true" mRtcEngine.SetMultiChannelWant(true); mRtcEngine.EnableAudio(); mRtcEngine.EnableVideo(); mRtcEngine.EnableVideoObserver(); mRtcEngine.SetClientRole(CLIENT_ROLE_TYPE.CLIENT_ROLE_BROADCASTER); channel1 = mRtcEngine.CreateChannel(CHANNEL_NAME_1); channel1.SetClientRole(CLIENT_ROLE_TYPE.CLIENT_ROLE_BROADCASTER); channel1.ChannelOnJoinChannelSuccess = Channel1OnJoinChannelSuccessHandler; // 自分が接続開始 channel1.ChannelOnLeaveChannel = Channel1OnLeaveChannelHandler; // 自分が退出時 channel1.ChannelOnUserJoined = Channel1OnUserJoinedHandler; // 別ユーザが接続してきたとき channel1.ChannelOnError = Channel1OnErrorHandler; // エラー時 channel1.ChannelOnUserOffLine = ChannelOnUserOfflineHandler; // 別ユーザーの接続が終了したとき channel1.ChannelOnScreenShareStarted = screenShareStartedHandler_MC; // 自分が映像開始時 channel1.ChannelOnScreenShareStopped = screenShareStoppedHandler_MC; // 自分が映像停止時 channel1.ChannelOnScreenShareCanceled = screenShareCanceledHandler_MC; // 自分が映像キャンセル時 channel1.ChannelOnVideoSizeChanged = onVideoSizeChanged_MCHandler; // 映像サイズ変更時 } public void JoinChannel1() { channel1.JoinChannel(TOKEN_1, "", 0, new ChannelMediaOptions(true, true)); if (joinChannelButtons.Length > 0 && joinChannelButtons[0]) { joinChannelButtons[0].interactable = false; leaveChannelButtons[0].interactable = true; } } public void LeaveChannel1() { channel1.LeaveChannel(); if (joinChannelButtons.Length > 0 && joinChannelButtons[0]) { joinChannelButtons[0].interactable = true; leaveChannelButtons[0].interactable = false; } } void JoinChannel() { mRtcEngine.JoinChannel(TOKEN_1, CHANNEL_NAME_1, "", 0, new ChannelMediaOptions(true, true, true, true)); } void OnApplicationQuit() { Debug.Log("OnApplicationQuit"); if (mRtcEngine != null) { channel1.LeaveChannel(); channel1.ReleaseChannel(); mRtcEngine.DisableVideoObserver(); IRtcEngine.Destroy(); } } float EnforcingViewLength = 180f; // 映像サイズ変更時 void onVideoSizeChanged_MCHandler(string channelID, uint uid, int width, int height, int rotation) { logger.UpdateLog(string.Format("channelOnVideoSizeChanged channelID: {3}, uid: {0}, width: {1}, height: {2}", uid, width, height, channelID)); if (UserVideoDict.ContainsKey(uid)) { GameObject go = UserVideoDict[uid].gameObject; Vector2 v2 = new Vector2(width, height); RawImage image = go.GetComponent<RawImage>(); v2 = AgoraUIUtils.GetScaledDimension(width, height, EnforcingViewLength); if (rotation == 90 || rotation == 270) { v2 = new Vector2(v2.y, v2.x); } image.rectTransform.sizeDelta = v2; } } void screenShareStartedHandler(string channelId, uint uid, int elapsed) { logger.UpdateLog(string.Format("onScreenShareStarted channelId: {0}, uid: {1}, elapsed: {2}", channelId, uid, elapsed)); } void screenShareStoppedHandler(string channelId, uint uid, int elapsed) { logger.UpdateLog(string.Format("onScreenShareStopped channelId: {0}, uid: {1}, elapsed: {2}", channelId, uid, elapsed)); } void screenShareCanceledHandler(string channelId, uint uid, int elapsed) { logger.UpdateLog(string.Format("onScreenShareCanceled channelId: {0}, uid: {1}, elapsed: {2}", channelId, uid, elapsed)); } // 自分が映像開始時 void screenShareStartedHandler_MC(string channelId, uint uid, int elapsed) { logger.UpdateLog(string.Format("onScreenShareStartedMC channelId: {0}, uid: {1}, elapsed: {2}", channelId, uid, elapsed)); } // 自分が映像停止時 void screenShareStoppedHandler_MC(string channelId, uint uid, int elapsed) { logger.UpdateLog(string.Format("onScreenShareStoppedMC channelId: {0}, uid: {1}, elapsed: {2}", channelId, uid, elapsed)); } // 自分が映像キャンセル時 void screenShareCanceledHandler_MC(string channelId, uint uid, int elapsed) { logger.UpdateLog(string.Format("onScreenShareCanceledMC channelId: {0}, uid: {1}, elapsed: {2}", channelId, uid, elapsed)); } // 自分が接続開始 void Channel1OnJoinChannelSuccessHandler(string channelId, uint uid, int elapsed) { logger.UpdateLog(string.Format("sdk version: ${0}", IRtcEngine.GetSdkVersion())); logger.UpdateLog(string.Format("onJoinChannelSuccess channelId: {0}, uid: {1}, elapsed: {2}", channelId, uid, elapsed)); makeVideoView(channelId, 0); } void EngineOnJoinChannelSuccessHandler(string channelId, uint uid, int elapsed) { logger.UpdateLog(string.Format("sdk version: ${0}", IRtcEngine.GetSdkVersion())); logger.UpdateLog(string.Format("EngineOnJoinChannelSuccess channelId: {0}, uid: {1}, elapsed: {2}", CHANNEL_NAME_1, uid, elapsed)); makeVideoView(channelId, 0); } // 自分が退出時 void Channel1OnLeaveChannelHandler(string channelId, RtcStats rtcStats) { logger.UpdateLog(string.Format("Channel1OnLeaveChannelHandler channelId: {0}", channelId)); } void EngineOnLeaveChannelHandler(RtcStats rtcStats) { logger.UpdateLog(string.Format("EngineOnLeaveChannelHandler channelId: {0}", CHANNEL_NAME_1)); } void Channel1OnErrorHandler(string channelId, int err, string message) { logger.UpdateLog(string.Format("Channel1OnErrorHandler channelId: {0}, err: {1}, message: {2}", channelId, err, message)); } void EngineOnErrorHandler(int err, string message) { logger.UpdateLog(string.Format("EngineOnErrorHandler channelId: {0}, err: {1}, message: {2}", CHANNEL_NAME_1, err, message)); } // 別ユーザが接続してきたとき void Channel1OnUserJoinedHandler(string channelId, uint uid, int elapsed) { logger.UpdateLog(string.Format("Channel1OnUserJoinedHandler channelId: {0} uid: ${1} elapsed: ${2}", channelId, uid, elapsed)); makeVideoView(channelId, uid, true); } void EngineOnUserJoinedHandler(uint uid, int elapsed) { logger.UpdateLog(string.Format("EngineOnUserJoinedHandler channelId: {0} uid: ${1} elapsed: ${2}", CHANNEL_NAME_1, uid, elapsed)); makeVideoView(CHANNEL_NAME_1, uid); } // 別ユーザーの接続が終了したとき void ChannelOnUserOfflineHandler(string channelId, uint uid, USER_OFFLINE_REASON reason) { logger.UpdateLog(string.Format("OnUserOffLine uid: ${0}, reason: ${1}", uid, (int)reason)); DestroyVideoView(channelId, uid); } void EngineOnUserOfflineHandler(uint uid, USER_OFFLINE_REASON reason) { logger.UpdateLog(string.Format("OnUserOffLine uid: ${0}, reason: ${1}", uid, (int)reason)); DestroyVideoView(CHANNEL_NAME_1, uid); } public void RespawnLocal(string channelName) { GameObject go = GameObject.Find(channelName + "_0"); if (go != null) { go.name = "Destroying"; Destroy(go); makeVideoView(channelName, 0); } } public void RespawnRemote() { if (LastRemote != null) { string[] strs = LastRemote.name.Split('_'); string channel = strs[0]; uint uid = uint.Parse(strs[1]); LastRemote.name = "_Destroyer"; Destroy(LastRemote); Debug.LogWarningFormat("Remaking video surface for uid:{0} channel:{1}", uid, channel); remoteUserDisplays.Remove(LastRemote); makeVideoView(channel, uid, true); } } GameObject LastRemote = null; private void makeVideoView(string channelId, uint uid, bool remote = false) { string objName = channelId + "_" + uid.ToString(); GameObject go = GameObject.Find(objName); if (!ReferenceEquals(go, null)) { return; // reuse } // create a GameObject and assign to this new user VideoSurface videoSurface = makeImageSurface(objName); if (!ReferenceEquals(videoSurface, null)) { // configure videoSurface videoSurface.SetForMultiChannelUser(channelId, uid); videoSurface.SetEnable(true); videoSurface.SetVideoSurfaceType(AgoraVideoSurfaceType.RawImage); // make the object draggable videoSurface.gameObject.AddComponent<UIElementDragger>(); if (uid != 0) { LastRemote = videoSurface.gameObject; } if (remote) { Debug.Log("is remote " + uid.ToString() + remote.ToString()); Vector2 v2 = AgoraUIUtils.GetScaledDimension(640, 360, EnforcingViewLength); videoSurface.GetComponent<RawImage>().rectTransform.sizeDelta = v2 * 3f; remoteUserDisplays.Add(videoSurface.gameObject); UserVideoDict[uid] = videoSurface; } else { Vector2 v2 = AgoraUIUtils.GetScaledDimension(640, 360, EnforcingViewLength); videoSurface.GetComponent<RawImage>().rectTransform.sizeDelta = v2 * 3f; } } } public VideoSurface makeImageSurface(string goName) { GameObject go = new GameObject(); if (go == null) { return null; } go.name = goName; // to be renderered onto go.AddComponent<RawImage>(); // make the object draggable go.AddComponent<UIElementDragger>(); GameObject canvas = GameObject.Find("VideoCanvas"); if (canvas != null) { go.transform.SetParent(canvas.transform); } // set up transform go.transform.Rotate(0f, 0.0f, 180.0f); float xPos = Random.Range(-Screen.width / 5f, Screen.width / 5f); float yPos = Random.Range(-Screen.height / 5f, Screen.height / 5f); go.transform.localPosition = new Vector3(xPos, yPos, 0f); // configure videoSurface VideoSurface videoSurface = go.AddComponent<VideoSurface>(); return videoSurface; } private void DestroyVideoView(string channelId, uint uid) { string objName = channelId + "_" + uid.ToString(); GameObject go = GameObject.Find(objName); if (!ReferenceEquals(go, null)) { Object.Destroy(go); } } }
動画が出ない場合
以下のコードを使ってる場合は、"VideoCanvas"名のGameObjectをヒエラルキーに設置してるか確認。
GameObject.Find("VideoCanvas");
チャンネルのClientRoleコールバック
channel1.ChannelOnClientRoleChanged = HandleOnClientRoleChanged; channel1.ChannelOnClientRoleChangeFailed = OnClientRoleChangeFailedHandler; void HandleOnClientRoleChanged(string channelId, CLIENT_ROLE_TYPE oldRole, CLIENT_ROLE_TYPE newRole) { Debug.Log("Engine OnClientRoleChanged: " + oldRole + " -> " + newRole); } void OnClientRoleChangeFailedHandler(string channelId, CLIENT_ROLE_CHANGE_FAILED_REASON reason, CLIENT_ROLE_TYPE currentRole) { Debug.Log("Engine OnClientRoleChangeFaile: " + reason + " c-> " + currentRole); }
左右反転対応
VideoSurface videoSurface = makeImageSurface(objName); videoSurface.EnableFilpTextureApply(true, false);
画面共有の音声がオフにできない問題
コミュニティの質問
https://github.com/AgoraIO-Community/Agora_Unity_WebGL/discussions/205