facebook twitter hatena line email

「Unity/Agora/voicechat community/デモ画面共有」の版間の差分

提供: 初心者エンジニアの簡易メモ
移動: 案内、 検索
(→‎"getDisplayMedia"のセキュリティエラーが出る場合)
(→‎左右反転対応)
 
(同じ利用者による、間の4版が非表示)
行483: 行483:
 
}
 
}
 
</pre>
 
</pre>
 +
=====動画が出ない場合=====
 +
以下のコードを使ってる場合は、"VideoCanvas"名のGameObjectをヒエラルキーに設置してるか確認。
 +
GameObject.Find("VideoCanvas");
 +
 +
====チャンネルのClientRoleコールバック====
 +
<pre>
 +
channel1.ChannelOnClientRoleChanged = HandleOnClientRoleChanged;
 +
channel1.ChannelOnClientRoleChangeFailed = OnClientRoleChangeFailedHandler;
 +
void HandleOnClientRoleChanged(string channelId, CLIENT_ROLE_TYPE oldRole, CLIENT_ROLE_TYPE newRole)
 +
{
 +
    Debug.Log("Engine OnClientRoleChanged: " + oldRole + " -> " + newRole);
 +
}
 +
void OnClientRoleChangeFailedHandler(string channelId, CLIENT_ROLE_CHANGE_FAILED_REASON reason, CLIENT_ROLE_TYPE currentRole)
 +
{
 +
    Debug.Log("Engine OnClientRoleChangeFaile: " + reason + " c-> " + currentRole);
 +
}
 +
</pre>
 +
====左右反転対応====
 +
<pre>
 +
VideoSurface videoSurface = makeImageSurface(objName);
 +
videoSurface.EnableFilpTextureApply(true, false);
 +
</pre>
 +
==画面共有の音声がオフにできない問題==
 +
コミュニティの質問
 +
 +
https://github.com/AgoraIO-Community/Agora_Unity_WebGL/discussions/205

2023年1月23日 (月) 02:29時点における最新版

デモから画面共有サンプル作成

DevDemo側サンプルソース解析

  • SceneFuncTests.unityのApi_testHelperのGameObjectのFunctionalTest/DevDemo/Test/DVC_ShareScreen.csが、画面共有ソース
  • TestHome.csのonJoinButtonClicked()からシーン移動している
  • FunctionalTest/DevDemo/SceneHome2のボタンで、移動できる。

以下"SceneFuncTests"がロードできないというエラーとなる場合

以下エラーとなる場合 

'SceneFuncTests' couldn't be loaded because it has not been added to the build settings or the AssetBundle has not been loaded.

To add a scene to the build settings use the menu File->Build Settings... PlayerSettingのSceneInBuildにSceneFuncTests.unityを追加すると良い。

以下"SceneFuncTests"がロードできないというエラーとなる場合

以下エラーとなる場合 

clientmanager.js:862 Note this API should be replaced by startScreenCaptureForWeb instead.

Agora_Unity_WebGL/[build_path]/AgoraWebSDK/libs/clientmanager.js

startScreenCaptureByDisplayId()はNGで、startScreenCaptureForWebに置き換えるようなエラーが出る。

以下の方だとstartScreenCaptureForWebを、実行するサンプルがあるのでそちらを確認する。

  • Assets/FunctionalTest/NewScreenShareClientManager/AgoraClientManager.cs
  • Assets/FunctionalTest/NewScreenShareMChannel/AgoraMultiChannel2.cs

NewScreenShareClientManager側サンプルソース解析

  • StartScreenButtonを押す
  • ClientManagerTestシーンは、背景のぼかしオン・オフができる。配信前に画面確認ができない。
  • MainScreenNewシーンは、背景のぼかしオンができる。オフができない。配信前に画面確認ができる。
  • NewScreenShareClientチェックは、画面共有時に、動画配信とは別のwindowを開く感じ。その際、画面共有接続には、SCREEN_SHARE_IDが使われる。
  • ドラッグ移動するコード。makeVideoView()とmakeImageSurface()ののどちらかが.AddComponent<UIElementDragger>();されてること。

"PERMISSION_DENIED"エラーが出た場合

エラー詳細 

AgoraRTCException: AgoraRTCError PERMISSION_DENIED: NotAllowedError: Permission denied by system
  1. macの場合は、macの設定/セキュリティとプライバシー/プライバシー/画面収録/Chromeをonに

配信元画面の大きさを変更

videoSurface.GetComponent<RawImage>().rectTransform.sizeDelta = v2 * 2f;

配信先画面の大きさを変更

if (remote)
{
    Vector2 v2 = AgoraUIUtils.GetScaledDimension(640, 360, EnforcingViewLength);
    videoSurface.GetComponent<RawImage>().rectTransform.sizeDelta = v2 * 2f;
    remoteUserDisplays.Add(videoSurface.gameObject);
    UserVideoDict[uid] = videoSurface;
}

ハンドラ

channel1.ChannelOnJoinChannelSuccess = Channel1OnJoinChannelSuccessHandler; // 自分が接続開始
channel1.ChannelOnLeaveChannel = Channel1OnLeaveChannelHandler; // 自分が退出時
channel1.ChannelOnUserJoined = Channel1OnUserJoinedHandler; // 別ユーザが接続してきたとき
channel1.ChannelOnError = Channel1OnErrorHandler; // エラー時
channel1.ChannelOnUserOffLine = ChannelOnUserOfflineHandler; // 別ユーザーの接続が終了したとき
channel1.ChannelOnScreenShareStarted = screenShareStartedHandler_MC; // 自分が映像開始時
channel1.ChannelOnScreenShareStopped = screenShareStoppedHandler_MC; // 自分が映像停止時
channel1.ChannelOnScreenShareCanceled = screenShareCanceledHandler_MC; // 自分が映像キャンセル時
channel1.ChannelOnVideoSizeChanged = onVideoSizeChanged_MCHandler; // 映像サイズ変更時

映像の背景ぼやかす

AgoraMultiChannel2のenableVirtualBackground()を動作させる。

"getDisplayMedia"のセキュリティエラーが出る場合

エラー詳細 

AgoraRTCException: AgoraRTCError UNEXPECTED_ERROR: SecurityError: Failed to execute 'getDisplayMedia' on 'MediaDevices': Access to the feature "display-capture" is disallowed by permission policy.

chrome ver94以降だとこのエラーが出るっぽい。iframeを使用しないようにするか、iframeタグないに、以下属性をつけるかすればよいっぽい 

<iframe frameborder="0" allow="camera *; geolocation *; microphone *; autoplay *" class="openctiSoftPhone" dataauraclass="openctiSoftPhone"></iframe>

参考：https://trailblazer.salesforce.com/issues_view?id=a1p4V0000029erUQAQ&title=domexception-failed-to-execute-getdisplaymedia-on-mediadevices-access-to-the-feature-display-capture-is-disallowed-by-permission-policy

デモ画面共有コード

MainSceneNew.sceneを適当なsceneへコピーし、JoinChannel、LeaveChannel、StartScreenShare、StopScreenShareだけのこして、以下のcsをVideoCanvasに設置 

using System.Collections;
using System.Collections.Generic;
using UnityEngine;
using UnityEngine.UI;
using agora_gaming_rtc;
using agora_utilities;

public class DisplayShareScene : MonoBehaviour
{
    [SerializeField] private string APP_ID = "YOUR_APPID";

    [SerializeField] private string TOKEN_1 = "";

    [SerializeField] private string CHANNEL_NAME_1 = "YOUR_CHANNEL_NAME_1";
    public Text logText;
    private Logger logger;
    private IRtcEngine mRtcEngine = null;
    private AgoraChannel channel1 = null;

    public Button startScreenShareButton, stopScreenShareButton;
    public bool useNewScreenShare = false;
    public bool useScreenShareAudio = false;

    public Button[] joinChannelButtons, leaveChannelButtons;

    protected Dictionary<uint, VideoSurface> UserVideoDict = new Dictionary<uint, VideoSurface>();
    private List<GameObject> remoteUserDisplays = new List<GameObject>();

    public VirtualBackgroundSource myVirtualBackground;
    public int blurDegrees = 2;
    public string hexColor = "#00FF00";
    public string imgFile = "seinfeld.jpg";
    public string videoFile = "movie.mp4";

    // Use this for initialization
    void Start()
    {
        if (!CheckAppId())
        {
            return;
        }
        joinChannelButtons[0].onClick.AddListener(JoinChannel1);
        leaveChannelButtons[0].onClick.AddListener(LeaveChannel1);

        InitEngine();

        //channel setup.
        updateScreenShareNew();
    }

    public void updateScreenShareNew()
    {
        startScreenShareButton.onClick.AddListener(delegate { startScreenShare2(useScreenShareAudio); });
        stopScreenShareButton.onClick.AddListener(delegate { stopScreenShare2(); });
    }

    void Update()
    {
        PermissionHelper.RequestMicrophontPermission();
        PermissionHelper.RequestCameraPermission();
    }

    bool CheckAppId()
    {
        logger = new Logger(logText);
        logger.DebugAssert(APP_ID.Length > 10, "Please fill in your appId in VideoCanvas!!!!!");
        return (APP_ID.Length > 10);
    }

    //for starting/stopping a screen share through AgoraChannel class.
    public void startScreenShare2(bool audioEnabled)
    {
        channel1.StartScreenCaptureForWeb(audioEnabled);
    }

    public void stopScreenShare2()
    {
        channel1.StopScreenCapture();
    }


    public void enableVirtualBackground()
    {
        channel1.enableVirtualBackground(true, myVirtualBackground);
    }

    public void setVirtualBackgroundBlur()
    {
        mRtcEngine.SetVirtualBackgroundBlur_MC(blurDegrees);
    }

    public void setVirtualBackgroundColor()
    {
        mRtcEngine.SetVirtualBackgroundColor_MC(hexColor);
    }

    public void setVirtualBackgroundImage()
    {
        mRtcEngine.SetVirtualBackgroundImage_MC(imgFile);
    }

    public void setVirtualBackgroundVideo()
    {
        mRtcEngine.SetVirtualBackgroundVideo_MC(videoFile);
    }

    void InitEngine()
    {
        mRtcEngine = IRtcEngine.GetEngine(APP_ID);
        mRtcEngine.SetChannelProfile(CHANNEL_PROFILE.CHANNEL_PROFILE_LIVE_BROADCASTING);
        // If you want to user Multi Channel Video, please call "SetMultiChannleWant to true"
        mRtcEngine.SetMultiChannelWant(true);
        mRtcEngine.EnableAudio();
        mRtcEngine.EnableVideo();
        mRtcEngine.EnableVideoObserver();
        mRtcEngine.SetClientRole(CLIENT_ROLE_TYPE.CLIENT_ROLE_BROADCASTER);

        channel1 = mRtcEngine.CreateChannel(CHANNEL_NAME_1);
        channel1.SetClientRole(CLIENT_ROLE_TYPE.CLIENT_ROLE_BROADCASTER);

        channel1.ChannelOnJoinChannelSuccess = Channel1OnJoinChannelSuccessHandler; // 自分が接続開始
        channel1.ChannelOnLeaveChannel = Channel1OnLeaveChannelHandler; // 自分が退出時
        channel1.ChannelOnUserJoined = Channel1OnUserJoinedHandler; // 別ユーザが接続してきたとき
        channel1.ChannelOnError = Channel1OnErrorHandler; // エラー時
        channel1.ChannelOnUserOffLine = ChannelOnUserOfflineHandler; // 別ユーザーの接続が終了したとき
        channel1.ChannelOnScreenShareStarted = screenShareStartedHandler_MC; // 自分が映像開始時
        channel1.ChannelOnScreenShareStopped = screenShareStoppedHandler_MC; // 自分が映像停止時
        channel1.ChannelOnScreenShareCanceled = screenShareCanceledHandler_MC; // 自分が映像キャンセル時
        channel1.ChannelOnVideoSizeChanged = onVideoSizeChanged_MCHandler; // 映像サイズ変更時

    }

    public void JoinChannel1()
    {
        channel1.JoinChannel(TOKEN_1, "", 0, new ChannelMediaOptions(true, true));
        if (joinChannelButtons.Length > 0 && joinChannelButtons[0])
        {
            joinChannelButtons[0].interactable = false;
            leaveChannelButtons[0].interactable = true;
        }
    }

    public void LeaveChannel1()
    {
        channel1.LeaveChannel();
        if (joinChannelButtons.Length > 0 && joinChannelButtons[0])
        {
            joinChannelButtons[0].interactable = true;
            leaveChannelButtons[0].interactable = false;
        }
    }

    void JoinChannel()
    {
        mRtcEngine.JoinChannel(TOKEN_1, CHANNEL_NAME_1, "", 0, new ChannelMediaOptions(true, true, true, true));
    }


    void OnApplicationQuit()
    {
        Debug.Log("OnApplicationQuit");
        if (mRtcEngine != null)
        {
            channel1.LeaveChannel();
            channel1.ReleaseChannel();

            mRtcEngine.DisableVideoObserver();
            IRtcEngine.Destroy();
        }
    }

    float EnforcingViewLength = 180f;
    // 映像サイズ変更時
    void onVideoSizeChanged_MCHandler(string channelID, uint uid, int width, int height, int rotation)
    {

        logger.UpdateLog(string.Format("channelOnVideoSizeChanged channelID: {3}, uid: {0}, width: {1}, height: {2}", uid,
            width, height, channelID));
        if (UserVideoDict.ContainsKey(uid))
        {
            GameObject go = UserVideoDict[uid].gameObject;
            Vector2 v2 = new Vector2(width, height);
            RawImage image = go.GetComponent<RawImage>();
            v2 = AgoraUIUtils.GetScaledDimension(width, height, EnforcingViewLength);
            if (rotation == 90 || rotation == 270)
            {
                v2 = new Vector2(v2.y, v2.x);
            }
            image.rectTransform.sizeDelta = v2;
        }
    }

    void screenShareStartedHandler(string channelId, uint uid, int elapsed)
    {
        logger.UpdateLog(string.Format("onScreenShareStarted channelId: {0}, uid: {1}, elapsed: {2}", channelId, uid,
            elapsed));
    }

    void screenShareStoppedHandler(string channelId, uint uid, int elapsed)
    {
        logger.UpdateLog(string.Format("onScreenShareStopped channelId: {0}, uid: {1}, elapsed: {2}", channelId, uid,
            elapsed));
    }

    void screenShareCanceledHandler(string channelId, uint uid, int elapsed)
    {
        logger.UpdateLog(string.Format("onScreenShareCanceled channelId: {0}, uid: {1}, elapsed: {2}", channelId, uid,
            elapsed));
    }
    // 自分が映像開始時
    void screenShareStartedHandler_MC(string channelId, uint uid, int elapsed)
    {
        logger.UpdateLog(string.Format("onScreenShareStartedMC channelId: {0}, uid: {1}, elapsed: {2}", channelId, uid,
            elapsed));
    }
    // 自分が映像停止時
    void screenShareStoppedHandler_MC(string channelId, uint uid, int elapsed)
    {
        logger.UpdateLog(string.Format("onScreenShareStoppedMC channelId: {0}, uid: {1}, elapsed: {2}", channelId, uid,
            elapsed));
    }
    // 自分が映像キャンセル時
    void screenShareCanceledHandler_MC(string channelId, uint uid, int elapsed)
    {
        logger.UpdateLog(string.Format("onScreenShareCanceledMC channelId: {0}, uid: {1}, elapsed: {2}", channelId, uid,
            elapsed));
    }
    // 自分が接続開始
    void Channel1OnJoinChannelSuccessHandler(string channelId, uint uid, int elapsed)
    {
        logger.UpdateLog(string.Format("sdk version: ${0}", IRtcEngine.GetSdkVersion()));
        logger.UpdateLog(string.Format("onJoinChannelSuccess channelId: {0}, uid: {1}, elapsed: {2}", channelId, uid,
            elapsed));
        makeVideoView(channelId, 0);
    }

    void EngineOnJoinChannelSuccessHandler(string channelId, uint uid, int elapsed)
    {
        logger.UpdateLog(string.Format("sdk version: ${0}", IRtcEngine.GetSdkVersion()));
        logger.UpdateLog(string.Format("EngineOnJoinChannelSuccess channelId: {0}, uid: {1}, elapsed: {2}", CHANNEL_NAME_1, uid,
            elapsed));
        makeVideoView(channelId, 0);
    }
    // 自分が退出時
    void Channel1OnLeaveChannelHandler(string channelId, RtcStats rtcStats)
    {
        logger.UpdateLog(string.Format("Channel1OnLeaveChannelHandler channelId: {0}", channelId));

    }

    void EngineOnLeaveChannelHandler(RtcStats rtcStats)
    {
        logger.UpdateLog(string.Format("EngineOnLeaveChannelHandler channelId: {0}", CHANNEL_NAME_1));
    }

    void Channel1OnErrorHandler(string channelId, int err, string message)
    {
        logger.UpdateLog(string.Format("Channel1OnErrorHandler channelId: {0}, err: {1}, message: {2}", channelId, err,
            message));
    }


    void EngineOnErrorHandler(int err, string message)
    {
        logger.UpdateLog(string.Format("EngineOnErrorHandler channelId: {0}, err: {1}, message: {2}", CHANNEL_NAME_1, err,
            message));
    }

    // 別ユーザが接続してきたとき
    void Channel1OnUserJoinedHandler(string channelId, uint uid, int elapsed)
    {
        logger.UpdateLog(string.Format("Channel1OnUserJoinedHandler channelId: {0} uid: ${1} elapsed: ${2}", channelId,
            uid, elapsed));
        makeVideoView(channelId, uid, true);
    }

    void EngineOnUserJoinedHandler(uint uid, int elapsed)
    {
        logger.UpdateLog(string.Format("EngineOnUserJoinedHandler channelId: {0} uid: ${1} elapsed: ${2}", CHANNEL_NAME_1,
            uid, elapsed));
        makeVideoView(CHANNEL_NAME_1, uid);
    }

    // 別ユーザーの接続が終了したとき
    void ChannelOnUserOfflineHandler(string channelId, uint uid, USER_OFFLINE_REASON reason)
    {
        logger.UpdateLog(string.Format("OnUserOffLine uid: ${0}, reason: ${1}", uid, (int)reason));
        DestroyVideoView(channelId, uid);
    }

    void EngineOnUserOfflineHandler(uint uid, USER_OFFLINE_REASON reason)
    {
        logger.UpdateLog(string.Format("OnUserOffLine uid: ${0}, reason: ${1}", uid, (int)reason));
        DestroyVideoView(CHANNEL_NAME_1, uid);
    }

    public void RespawnLocal(string channelName)
    {
        GameObject go = GameObject.Find(channelName + "_0");
        if (go != null)
        {
            go.name = "Destroying";
            Destroy(go);
            makeVideoView(channelName, 0);
        }
    }

    public void RespawnRemote()
    {
        if (LastRemote != null)
        {
            string[] strs = LastRemote.name.Split('_');
            string channel = strs[0];
            uint uid = uint.Parse(strs[1]);
            LastRemote.name = "_Destroyer";
            Destroy(LastRemote);
            Debug.LogWarningFormat("Remaking video surface for  uid:{0} channel:{1}", uid, channel);
            remoteUserDisplays.Remove(LastRemote);
            makeVideoView(channel, uid, true);
        }
    }

    GameObject LastRemote = null;

    private void makeVideoView(string channelId, uint uid, bool remote = false)
    {
        string objName = channelId + "_" + uid.ToString();
        GameObject go = GameObject.Find(objName);
        if (!ReferenceEquals(go, null))
        {
            return; // reuse
        }

        // create a GameObject and assign to this new user
        VideoSurface videoSurface = makeImageSurface(objName);
        if (!ReferenceEquals(videoSurface, null))
        {
            // configure videoSurface
            videoSurface.SetForMultiChannelUser(channelId, uid);
            videoSurface.SetEnable(true);
            videoSurface.SetVideoSurfaceType(AgoraVideoSurfaceType.RawImage);
            // make the object draggable
            videoSurface.gameObject.AddComponent<UIElementDragger>();

            if (uid != 0)
            {
                LastRemote = videoSurface.gameObject;
            }

            if (remote)
            {
                Debug.Log("is remote " + uid.ToString() + remote.ToString());
                Vector2 v2 = AgoraUIUtils.GetScaledDimension(640, 360, EnforcingViewLength);
                videoSurface.GetComponent<RawImage>().rectTransform.sizeDelta = v2 * 3f;
                remoteUserDisplays.Add(videoSurface.gameObject);
                UserVideoDict[uid] = videoSurface;
            }
            else
            {
                Vector2 v2 = AgoraUIUtils.GetScaledDimension(640, 360, EnforcingViewLength);
                videoSurface.GetComponent<RawImage>().rectTransform.sizeDelta = v2 * 3f;
            }
        }
    }

    public VideoSurface makeImageSurface(string goName)
    {
        GameObject go = new GameObject();

        if (go == null)
        {
            return null;
        }

        go.name = goName;

        // to be renderered onto
        go.AddComponent<RawImage>();

        // make the object draggable
        go.AddComponent<UIElementDragger>();
        GameObject canvas = GameObject.Find("VideoCanvas");
        if (canvas != null)
        {
            go.transform.SetParent(canvas.transform);
        }
        // set up transform
        go.transform.Rotate(0f, 0.0f, 180.0f);
        float xPos = Random.Range(-Screen.width / 5f, Screen.width / 5f);
        float yPos = Random.Range(-Screen.height / 5f, Screen.height / 5f);
        go.transform.localPosition = new Vector3(xPos, yPos, 0f);

        // configure videoSurface
        VideoSurface videoSurface = go.AddComponent<VideoSurface>();
        return videoSurface;
    }

    private void DestroyVideoView(string channelId, uint uid)
    {
        string objName = channelId + "_" + uid.ToString();
        GameObject go = GameObject.Find(objName);
        if (!ReferenceEquals(go, null))
        {
            Object.Destroy(go);
        }
    }
}
動画が出ない場合

以下のコードを使ってる場合は、"VideoCanvas"名のGameObjectをヒエラルキーに設置してるか確認。 

GameObject.Find("VideoCanvas");

チャンネルのClientRoleコールバック

channel1.ChannelOnClientRoleChanged = HandleOnClientRoleChanged;
channel1.ChannelOnClientRoleChangeFailed = OnClientRoleChangeFailedHandler;
void HandleOnClientRoleChanged(string channelId, CLIENT_ROLE_TYPE oldRole, CLIENT_ROLE_TYPE newRole)
{
    Debug.Log("Engine OnClientRoleChanged: " + oldRole + " -> " + newRole);
}
void OnClientRoleChangeFailedHandler(string channelId, CLIENT_ROLE_CHANGE_FAILED_REASON reason, CLIENT_ROLE_TYPE currentRole)
{
    Debug.Log("Engine OnClientRoleChangeFaile: " + reason + " c-> " + currentRole);
}

左右反転対応

VideoSurface videoSurface = makeImageSurface(objName);
videoSurface.EnableFilpTextureApply(true, false);

画面共有の音声がオフにできない問題

コミュニティの質問

https://github.com/AgoraIO-Community/Agora_Unity_WebGL/discussions/205

「https://wiki.nonip.net/index.php?title=Unity/Agora/voicechat_community/デモ画面共有&oldid=16595」から取得