Unity/Agora/voicechat community/デモ画面共有
提供: 初心者エンジニアの簡易メモ
目次
デモから画面共有サンプル作成
DevDemo側サンプルソース解析
- SceneFuncTests.unityのApi_testHelperのGameObjectのFunctionalTest/DevDemo/Test/DVC_ShareScreen.csが、画面共有ソース
- TestHome.csのonJoinButtonClicked()からシーン移動している
- FunctionalTest/DevDemo/SceneHome2のボタンで、移動できる。
以下"SceneFuncTests"がロードできないというエラーとなる場合
以下エラーとなる場合
'SceneFuncTests' couldn't be loaded because it has not been added to the build settings or the AssetBundle has not been loaded.
To add a scene to the build settings use the menu File->Build Settings... PlayerSettingのSceneInBuildにSceneFuncTests.unityを追加すると良い。
以下"SceneFuncTests"がロードできないというエラーとなる場合
以下エラーとなる場合
clientmanager.js:862 Note this API should be replaced by startScreenCaptureForWeb instead.
Agora_Unity_WebGL/[build_path]/AgoraWebSDK/libs/clientmanager.js
startScreenCaptureByDisplayId()はNGで、startScreenCaptureForWebに置き換えるようなエラーが出る。
以下の方だとstartScreenCaptureForWebを、実行するサンプルがあるのでそちらを確認する。
- Assets/FunctionalTest/NewScreenShareClientManager/AgoraClientManager.cs
- Assets/FunctionalTest/NewScreenShareMChannel/AgoraMultiChannel2.cs
- StartScreenButtonを押す
- ClientManagerTestシーンは、背景のぼかしオン・オフができる。配信前に画面確認ができない。
- MainScreenNewシーンは、背景のぼかしオンができる。オフができない。配信前に画面確認ができる。
- NewScreenShareClientチェックは、画面共有時に、動画配信とは別のwindowを開く感じ。その際、画面共有接続には、SCREEN_SHARE_IDが使われる。
- ドラッグ移動するコード。makeVideoView()とmakeImageSurface()ののどちらかが.AddComponent<UIElementDragger>();されてること。
"PERMISSION_DENIED"エラーが出た場合
エラー詳細
AgoraRTCException: AgoraRTCError PERMISSION_DENIED: NotAllowedError: Permission denied by system
- macの場合は、macの設定/セキュリティとプライバシー/プライバシー/画面収録/Chromeをonに
配信元画面の大きさを変更
videoSurface.GetComponent<RawImage>().rectTransform.sizeDelta = v2 * 2f;
配信先画面の大きさを変更
if (remote)
{
Vector2 v2 = AgoraUIUtils.GetScaledDimension(640, 360, EnforcingViewLength);
videoSurface.GetComponent<RawImage>().rectTransform.sizeDelta = v2 * 2f;
remoteUserDisplays.Add(videoSurface.gameObject);
UserVideoDict[uid] = videoSurface;
}
ハンドラ
channel1.ChannelOnJoinChannelSuccess = Channel1OnJoinChannelSuccessHandler; // 自分が接続開始 channel1.ChannelOnLeaveChannel = Channel1OnLeaveChannelHandler; // 自分が退出時 channel1.ChannelOnUserJoined = Channel1OnUserJoinedHandler; // 別ユーザが接続してきたとき channel1.ChannelOnError = Channel1OnErrorHandler; // エラー時 channel1.ChannelOnUserOffLine = ChannelOnUserOfflineHandler; // 別ユーザーの接続が終了したとき channel1.ChannelOnScreenShareStarted = screenShareStartedHandler_MC; // 自分が映像開始時 channel1.ChannelOnScreenShareStopped = screenShareStoppedHandler_MC; // 自分が映像停止時 channel1.ChannelOnScreenShareCanceled = screenShareCanceledHandler_MC; // 自分が映像キャンセル時 channel1.ChannelOnVideoSizeChanged = onVideoSizeChanged_MCHandler; // 映像サイズ変更時
映像の背景ぼやかす
AgoraMultiChannel2のenableVirtualBackground()を動作させる。
"getDisplayMedia"のセキュリティエラーが出る場合
エラー詳細
AgoraRTCException: AgoraRTCError UNEXPECTED_ERROR: SecurityError: Failed to execute 'getDisplayMedia' on 'MediaDevices': Access to the feature "display-capture" is disallowed by permission policy.
chrome ver94以降だとこのエラーが出るっぽい。iframeを使用しないようにするか、iframeタグないに、以下属性をつけるかすればよいっぽい
<iframe frameborder="0" allow="camera *; geolocation *; microphone *; autoplay *" class="openctiSoftPhone" dataauraclass="openctiSoftPhone"></iframe>
デモ画面共有コード
MainSceneNew.sceneを適当なsceneへコピーし、JoinChannel、LeaveChannel、StartScreenShare、StopScreenShareだけのこして、以下のcsをVideoCanvasに設置
using System.Collections;
using System.Collections.Generic;
using UnityEngine;
using UnityEngine.UI;
using agora_gaming_rtc;
using agora_utilities;
public class DisplayShareScene : MonoBehaviour
{
[SerializeField] private string APP_ID = "YOUR_APPID";
[SerializeField] private string TOKEN_1 = "";
[SerializeField] private string CHANNEL_NAME_1 = "YOUR_CHANNEL_NAME_1";
public Text logText;
private Logger logger;
private IRtcEngine mRtcEngine = null;
private AgoraChannel channel1 = null;
public Button startScreenShareButton, stopScreenShareButton;
public bool useNewScreenShare = false;
public bool useScreenShareAudio = false;
public Button[] joinChannelButtons, leaveChannelButtons;
protected Dictionary<uint, VideoSurface> UserVideoDict = new Dictionary<uint, VideoSurface>();
private List<GameObject> remoteUserDisplays = new List<GameObject>();
public VirtualBackgroundSource myVirtualBackground;
public int blurDegrees = 2;
public string hexColor = "#00FF00";
public string imgFile = "seinfeld.jpg";
public string videoFile = "movie.mp4";
// Use this for initialization
void Start()
{
if (!CheckAppId())
{
return;
}
joinChannelButtons[0].onClick.AddListener(JoinChannel1);
leaveChannelButtons[0].onClick.AddListener(LeaveChannel1);
InitEngine();
//channel setup.
updateScreenShareNew();
}
public void updateScreenShareNew()
{
startScreenShareButton.onClick.AddListener(delegate { startScreenShare2(useScreenShareAudio); });
stopScreenShareButton.onClick.AddListener(delegate { stopScreenShare2(); });
}
void Update()
{
PermissionHelper.RequestMicrophontPermission();
PermissionHelper.RequestCameraPermission();
}
bool CheckAppId()
{
logger = new Logger(logText);
logger.DebugAssert(APP_ID.Length > 10, "Please fill in your appId in VideoCanvas!!!!!");
return (APP_ID.Length > 10);
}
//for starting/stopping a screen share through AgoraChannel class.
public void startScreenShare2(bool audioEnabled)
{
channel1.StartScreenCaptureForWeb(audioEnabled);
}
public void stopScreenShare2()
{
channel1.StopScreenCapture();
}
public void enableVirtualBackground()
{
channel1.enableVirtualBackground(true, myVirtualBackground);
}
public void setVirtualBackgroundBlur()
{
mRtcEngine.SetVirtualBackgroundBlur_MC(blurDegrees);
}
public void setVirtualBackgroundColor()
{
mRtcEngine.SetVirtualBackgroundColor_MC(hexColor);
}
public void setVirtualBackgroundImage()
{
mRtcEngine.SetVirtualBackgroundImage_MC(imgFile);
}
public void setVirtualBackgroundVideo()
{
mRtcEngine.SetVirtualBackgroundVideo_MC(videoFile);
}
void InitEngine()
{
mRtcEngine = IRtcEngine.GetEngine(APP_ID);
mRtcEngine.SetChannelProfile(CHANNEL_PROFILE.CHANNEL_PROFILE_LIVE_BROADCASTING);
// If you want to user Multi Channel Video, please call "SetMultiChannleWant to true"
mRtcEngine.SetMultiChannelWant(true);
mRtcEngine.EnableAudio();
mRtcEngine.EnableVideo();
mRtcEngine.EnableVideoObserver();
mRtcEngine.SetClientRole(CLIENT_ROLE_TYPE.CLIENT_ROLE_BROADCASTER);
channel1 = mRtcEngine.CreateChannel(CHANNEL_NAME_1);
channel1.SetClientRole(CLIENT_ROLE_TYPE.CLIENT_ROLE_BROADCASTER);
channel1.ChannelOnJoinChannelSuccess = Channel1OnJoinChannelSuccessHandler; // 自分が接続開始
channel1.ChannelOnLeaveChannel = Channel1OnLeaveChannelHandler; // 自分が退出時
channel1.ChannelOnUserJoined = Channel1OnUserJoinedHandler; // 別ユーザが接続してきたとき
channel1.ChannelOnError = Channel1OnErrorHandler; // エラー時
channel1.ChannelOnUserOffLine = ChannelOnUserOfflineHandler; // 別ユーザーの接続が終了したとき
channel1.ChannelOnScreenShareStarted = screenShareStartedHandler_MC; // 自分が映像開始時
channel1.ChannelOnScreenShareStopped = screenShareStoppedHandler_MC; // 自分が映像停止時
channel1.ChannelOnScreenShareCanceled = screenShareCanceledHandler_MC; // 自分が映像キャンセル時
channel1.ChannelOnVideoSizeChanged = onVideoSizeChanged_MCHandler; // 映像サイズ変更時
}
public void JoinChannel1()
{
channel1.JoinChannel(TOKEN_1, "", 0, new ChannelMediaOptions(true, true));
if (joinChannelButtons.Length > 0 && joinChannelButtons[0])
{
joinChannelButtons[0].interactable = false;
leaveChannelButtons[0].interactable = true;
}
}
public void LeaveChannel1()
{
channel1.LeaveChannel();
if (joinChannelButtons.Length > 0 && joinChannelButtons[0])
{
joinChannelButtons[0].interactable = true;
leaveChannelButtons[0].interactable = false;
}
}
void JoinChannel()
{
mRtcEngine.JoinChannel(TOKEN_1, CHANNEL_NAME_1, "", 0, new ChannelMediaOptions(true, true, true, true));
}
void OnApplicationQuit()
{
Debug.Log("OnApplicationQuit");
if (mRtcEngine != null)
{
channel1.LeaveChannel();
channel1.ReleaseChannel();
mRtcEngine.DisableVideoObserver();
IRtcEngine.Destroy();
}
}
float EnforcingViewLength = 180f;
// 映像サイズ変更時
void onVideoSizeChanged_MCHandler(string channelID, uint uid, int width, int height, int rotation)
{
logger.UpdateLog(string.Format("channelOnVideoSizeChanged channelID: {3}, uid: {0}, width: {1}, height: {2}", uid,
width, height, channelID));
if (UserVideoDict.ContainsKey(uid))
{
GameObject go = UserVideoDict[uid].gameObject;
Vector2 v2 = new Vector2(width, height);
RawImage image = go.GetComponent<RawImage>();
v2 = AgoraUIUtils.GetScaledDimension(width, height, EnforcingViewLength);
if (rotation == 90 || rotation == 270)
{
v2 = new Vector2(v2.y, v2.x);
}
image.rectTransform.sizeDelta = v2;
}
}
void screenShareStartedHandler(string channelId, uint uid, int elapsed)
{
logger.UpdateLog(string.Format("onScreenShareStarted channelId: {0}, uid: {1}, elapsed: {2}", channelId, uid,
elapsed));
}
void screenShareStoppedHandler(string channelId, uint uid, int elapsed)
{
logger.UpdateLog(string.Format("onScreenShareStopped channelId: {0}, uid: {1}, elapsed: {2}", channelId, uid,
elapsed));
}
void screenShareCanceledHandler(string channelId, uint uid, int elapsed)
{
logger.UpdateLog(string.Format("onScreenShareCanceled channelId: {0}, uid: {1}, elapsed: {2}", channelId, uid,
elapsed));
}
// 自分が映像開始時
void screenShareStartedHandler_MC(string channelId, uint uid, int elapsed)
{
logger.UpdateLog(string.Format("onScreenShareStartedMC channelId: {0}, uid: {1}, elapsed: {2}", channelId, uid,
elapsed));
}
// 自分が映像停止時
void screenShareStoppedHandler_MC(string channelId, uint uid, int elapsed)
{
logger.UpdateLog(string.Format("onScreenShareStoppedMC channelId: {0}, uid: {1}, elapsed: {2}", channelId, uid,
elapsed));
}
// 自分が映像キャンセル時
void screenShareCanceledHandler_MC(string channelId, uint uid, int elapsed)
{
logger.UpdateLog(string.Format("onScreenShareCanceledMC channelId: {0}, uid: {1}, elapsed: {2}", channelId, uid,
elapsed));
}
// 自分が接続開始
void Channel1OnJoinChannelSuccessHandler(string channelId, uint uid, int elapsed)
{
logger.UpdateLog(string.Format("sdk version: ${0}", IRtcEngine.GetSdkVersion()));
logger.UpdateLog(string.Format("onJoinChannelSuccess channelId: {0}, uid: {1}, elapsed: {2}", channelId, uid,
elapsed));
makeVideoView(channelId, 0);
}
void EngineOnJoinChannelSuccessHandler(string channelId, uint uid, int elapsed)
{
logger.UpdateLog(string.Format("sdk version: ${0}", IRtcEngine.GetSdkVersion()));
logger.UpdateLog(string.Format("EngineOnJoinChannelSuccess channelId: {0}, uid: {1}, elapsed: {2}", CHANNEL_NAME_1, uid,
elapsed));
makeVideoView(channelId, 0);
}
// 自分が退出時
void Channel1OnLeaveChannelHandler(string channelId, RtcStats rtcStats)
{
logger.UpdateLog(string.Format("Channel1OnLeaveChannelHandler channelId: {0}", channelId));
}
void EngineOnLeaveChannelHandler(RtcStats rtcStats)
{
logger.UpdateLog(string.Format("EngineOnLeaveChannelHandler channelId: {0}", CHANNEL_NAME_1));
}
void Channel1OnErrorHandler(string channelId, int err, string message)
{
logger.UpdateLog(string.Format("Channel1OnErrorHandler channelId: {0}, err: {1}, message: {2}", channelId, err,
message));
}
void EngineOnErrorHandler(int err, string message)
{
logger.UpdateLog(string.Format("EngineOnErrorHandler channelId: {0}, err: {1}, message: {2}", CHANNEL_NAME_1, err,
message));
}
// 別ユーザが接続してきたとき
void Channel1OnUserJoinedHandler(string channelId, uint uid, int elapsed)
{
logger.UpdateLog(string.Format("Channel1OnUserJoinedHandler channelId: {0} uid: ${1} elapsed: ${2}", channelId,
uid, elapsed));
makeVideoView(channelId, uid, true);
}
void EngineOnUserJoinedHandler(uint uid, int elapsed)
{
logger.UpdateLog(string.Format("EngineOnUserJoinedHandler channelId: {0} uid: ${1} elapsed: ${2}", CHANNEL_NAME_1,
uid, elapsed));
makeVideoView(CHANNEL_NAME_1, uid);
}
// 別ユーザーの接続が終了したとき
void ChannelOnUserOfflineHandler(string channelId, uint uid, USER_OFFLINE_REASON reason)
{
logger.UpdateLog(string.Format("OnUserOffLine uid: ${0}, reason: ${1}", uid, (int)reason));
DestroyVideoView(channelId, uid);
}
void EngineOnUserOfflineHandler(uint uid, USER_OFFLINE_REASON reason)
{
logger.UpdateLog(string.Format("OnUserOffLine uid: ${0}, reason: ${1}", uid, (int)reason));
DestroyVideoView(CHANNEL_NAME_1, uid);
}
public void RespawnLocal(string channelName)
{
GameObject go = GameObject.Find(channelName + "_0");
if (go != null)
{
go.name = "Destroying";
Destroy(go);
makeVideoView(channelName, 0);
}
}
public void RespawnRemote()
{
if (LastRemote != null)
{
string[] strs = LastRemote.name.Split('_');
string channel = strs[0];
uint uid = uint.Parse(strs[1]);
LastRemote.name = "_Destroyer";
Destroy(LastRemote);
Debug.LogWarningFormat("Remaking video surface for uid:{0} channel:{1}", uid, channel);
remoteUserDisplays.Remove(LastRemote);
makeVideoView(channel, uid, true);
}
}
GameObject LastRemote = null;
private void makeVideoView(string channelId, uint uid, bool remote = false)
{
string objName = channelId + "_" + uid.ToString();
GameObject go = GameObject.Find(objName);
if (!ReferenceEquals(go, null))
{
return; // reuse
}
// create a GameObject and assign to this new user
VideoSurface videoSurface = makeImageSurface(objName);
if (!ReferenceEquals(videoSurface, null))
{
// configure videoSurface
videoSurface.SetForMultiChannelUser(channelId, uid);
videoSurface.SetEnable(true);
videoSurface.SetVideoSurfaceType(AgoraVideoSurfaceType.RawImage);
// make the object draggable
videoSurface.gameObject.AddComponent<UIElementDragger>();
if (uid != 0)
{
LastRemote = videoSurface.gameObject;
}
if (remote)
{
Debug.Log("is remote " + uid.ToString() + remote.ToString());
Vector2 v2 = AgoraUIUtils.GetScaledDimension(640, 360, EnforcingViewLength);
videoSurface.GetComponent<RawImage>().rectTransform.sizeDelta = v2 * 3f;
remoteUserDisplays.Add(videoSurface.gameObject);
UserVideoDict[uid] = videoSurface;
}
else
{
Vector2 v2 = AgoraUIUtils.GetScaledDimension(640, 360, EnforcingViewLength);
videoSurface.GetComponent<RawImage>().rectTransform.sizeDelta = v2 * 3f;
}
}
}
public VideoSurface makeImageSurface(string goName)
{
GameObject go = new GameObject();
if (go == null)
{
return null;
}
go.name = goName;
// to be renderered onto
go.AddComponent<RawImage>();
// make the object draggable
go.AddComponent<UIElementDragger>();
GameObject canvas = GameObject.Find("VideoCanvas");
if (canvas != null)
{
go.transform.SetParent(canvas.transform);
}
// set up transform
go.transform.Rotate(0f, 0.0f, 180.0f);
float xPos = Random.Range(-Screen.width / 5f, Screen.width / 5f);
float yPos = Random.Range(-Screen.height / 5f, Screen.height / 5f);
go.transform.localPosition = new Vector3(xPos, yPos, 0f);
// configure videoSurface
VideoSurface videoSurface = go.AddComponent<VideoSurface>();
return videoSurface;
}
private void DestroyVideoView(string channelId, uint uid)
{
string objName = channelId + "_" + uid.ToString();
GameObject go = GameObject.Find(objName);
if (!ReferenceEquals(go, null))
{
Object.Destroy(go);
}
}
}
動画が出ない場合
以下のコードを使ってる場合は、"VideoCanvas"名のGameObjectをヒエラルキーに設置してるか確認。
GameObject.Find("VideoCanvas");
チャンネルのClientRoleコールバック
channel1.ChannelOnClientRoleChanged = HandleOnClientRoleChanged;
channel1.ChannelOnClientRoleChangeFailed = OnClientRoleChangeFailedHandler;
void HandleOnClientRoleChanged(string channelId, CLIENT_ROLE_TYPE oldRole, CLIENT_ROLE_TYPE newRole)
{
Debug.Log("Engine OnClientRoleChanged: " + oldRole + " -> " + newRole);
}
void OnClientRoleChangeFailedHandler(string channelId, CLIENT_ROLE_CHANGE_FAILED_REASON reason, CLIENT_ROLE_TYPE currentRole)
{
Debug.Log("Engine OnClientRoleChangeFaile: " + reason + " c-> " + currentRole);
}
左右反転対応
VideoSurface videoSurface = makeImageSurface(objName); videoSurface.EnableFilpTextureApply(true, false);
画面共有の音声がオフにできない問題
コミュニティの質問
https://github.com/AgoraIO-Community/Agora_Unity_WebGL/discussions/205
