z-index not working as expected

[sagivf]

I'm trying to overlay the users video (the publisher) over the user he is communicating with (the subscriber).
The order of the participants joining the session seems to affect whether this is possible or not.
I'm not sure if the z-index has any effect or if it's just inconsistent.
The issue is for IOS and Android. web works fine.
I tried a load of different CSS setups, but nothing was consistent.

I can create a project and screenshots, but I want to first see if someone has encountered this and if there is someone who can look into it.

[sagivf]

I'm basically trying to achieve this -
https://tokbox.com/blog/wp-content/uploads/2017/11/OpenTok-Plugin-for-Cordova-2-287x287.png.

But like I said this only works in some cases, depending on who enters the room first.

[youssmak]

There is already an issue about this #86
I'm working with a fork as a temporary workaround https://github.com/talkspiritlab/cordova-plugin-opentok

[sagivf]

Thanks @youssmak, I tried using https://github.com/msach22/cordova-plugin-opentok and it didn't help.
https://github.com/talkspiritlab/cordova-plugin-opentok works well you say?

[sagivf]

@youssmak talkspiritlab fork works great! Thank you very much.

[sagivf]

@youssmak This works great on android.
On IOS it seems like the negative z-index isn't working. I get a dark screen. 😢 😢

Does this work for you guys?

[wolfenrain]

@sagivf Did you add the transparent CSS rules?

[sagivf]

@wolfenrain I did. If I set up a plain Ionic app, could you have a look?

[alexisjcr07]

Hello, I have the same error, but in my case I am trying to show another view on the video.

my code:
.ts

none(){
this.cc = 'none';
this.style = 'none';
}
block(){
this.cc = 'block';
this.style = 'block';
}

streamCreated: (event) => {

let subscriberProperties = {width:"100%",height:"91% !important"};

if(event.stream.videoType === 'screen') {
this.session.subscribe(event.stream, 'screen-preview', subscriberProperties);
this.none();

}else
{
this.session.subscribe(event.stream, 'subscriber', subOptions);
this.block();
OT.updateViews();

}
},

.html
.css
#subscriber {
width: 100% !important;
top: 0;
height: 91% !important;
overflow: hidden;
position: fixed;
margin: 0px;
background: #0c0c0c;
left: 0px;

}

.none {
width: 100% !important;
top: 0;
height: 91% !important;
overflow: hidden;
position: relative;
margin: 0px !important;
left: 0px;
display: none !important;

}

.block {
width: 100% !important;
top: 0;
height: 91% !important;
overflow: hidden;
position: relative;
margin: 0px !important;
left: 0px;
display: block !important;

}

#publisher {
width: 30% !important;
height: 18% !important;
overflow: hidden;
position: fixed;
margin: 0px !important;
right: 14px;
bottom: 80px;
left: auto;
z-index: 40;
}

#screen-preview {
width: 100% !important;
top: 0;
height: 91% !important;
overflow: hidden;
position: fixed;
margin: 0px !important;
background: transparent !important;
z-index: 999;
left: 0px;
}

[mmubasher]

@sagivf I was able to achieve that layout by just this code

<div id="publisher" style="width: 30vh; height: 25vh; position: absolute; top: 0; left: 0; z-index: 1000; pointer-events:none;"></div>
<div id="subscriber" style="width: 100%; height: 100vh; position: absolute; top: 0; left: 0; z-index: 500; pointer-events:none;"></div>

[msach22]

@youssmak @sagivf Thanks for the updates and the work you've done here. Are you folks willing to create a PR to the main fork so the community members can also benefit from the change?

[Shtibel]

I also have the same problem
trying to add html button with absolute position on top of the video and I can't see the element

[Shtibel]

Please when can you add z-index support for ANDROID and IOS ?
We need it ASAP!

Thank you.

[vikaskonsam]

@mmubasher
Are you able to overlay HTML based control elements (button to start/stop call) over the subscribers view?
My layout is similar to this https://tokbox.com/blog/wp-content/uploads/2017/11/OpenTok-Plugin-for-Cordova-2-287x287.png with some HTML elements over the subscriber's view.

[Shtibel]

Hן @vikaskonsam ,
No we are not able to see the overlay html...
We need to accomplish this:

But all the HTML overlay elements are not visible in Android and IOS
That is the problem...

[vikaskonsam]

@Shtibel
I understand your problem as I am stuck in the same issue.
I was actually looking for a response from @mmubasher as he seems to have some solutions. I was hoping it covers our problem. I should have tagged him in my comment. My bad.

For android we are just making use of the https://static.opentok.com/v2/js/opentok.min.js and this seems to work.
You may face a problem with getting camera and microphone permissions and for that we are making use of https://github.com/dpa99c/cordova-diagnostic-plugin.

[Shtibel]

HI @vikaskonsam ,
Don't do that !
Don't user the opentok.min.js on android !

That is what we did at first but when I talked to tokBox support about other problems they say that in hybrid cordova application (like Ionic) we should use only the plugin cordova-plugin-opentok because the app can brake if you use the opentok.min.js and the performance will be pure and the device will heet.

So we need this fix for android as well and ASAP!
So please @mmubasher when can you help with this ?

[mmubasher]

@Shtibel Unfortunately, I do not have a solution for overlaying buttons on top of native views that hold the streamed videos. Problem is, these native views are rendered on the screen and HTML with any z-index gets rendered behind them. I have created a sample project here.
Currently, I am looking into rendering controls on top of the native view since HTML workarounds do not work.
The solution I presented was to render publisher's video on top of subscriber stream.
cc: @vikaskonsam

[Shtibel]

Hi @msach22
As you can see on previous comments we need the z-index problem fixed as soon as possible.
When can you work on it ?

[vikaskonsam]

@Shtibel -
I have had a partial success in android by doing the following:

1. Set #publisher css with z-index of -1
2. Set #subscriber css with z-index of -2
3. Set background:transparent for ion-app
4. After publishing remove any content inside the publisher container

this.session.publish(this.publisher);
if (OT.updateViews) {
    OT.updateViews();
    document.getElementById("publisher").innerHTML = "";
}

5. Do the same on streamCreated also

this.session.on({
  streamCreated: event => {
  this.subscribed = true;
  this.session.subscribe(event.stream, "subscriber", {
  insertMode: "append",
  width: "100%",
  height: "100%",
  style: {
            buttonDisplayMode: "on",
            audioLevelDisplayMode: "on"
  }
});

 if (OT.updateViews) {
    OT.updateViews();
    document.getElementById("subscriber").innerHTML = "";
  }
})

Have had success with this on Android 7, but not Android 8

You may also want to look at this file and see if you can do some changes there.
cordova-plugin-opentok/src/android/OpenTokAndroidPlugin.java

Try looking at the lines where addView and setTranslationZ functions are used.

[msach22]

Hey folks, sorry for not being able to get to this. I understand this is quite important - I have been investigating further, but I don't have a 100% solution at this time. I'll update the issue once I have more details. Let me see if @wolfenrain has any ideas?

[wolfenrain]

Yes I think this has to do with how the current native views behave. The plugin by default uses SurfaceViews. Unfortunately for us the setZand setTranslationZ only put the views either in front or in back. I would assume that there might be more cases in which it doesn't function as expected.

I made an implementation #86 a while ago that uses TextureViews those come with its pros and cons. But as far as I can remember, I never had any of the issues that are mentioned here using the TextureViews.

[Shtibel]

Hi all,
Maybe the ionic-native-google-maps solution can help you fix this z-index problem and let html elements hover over the native elements.
Please have a look on how they solve this problem
This is the plugin: https://github.com/ionic-team/ionic-native-google-maps

This is a full description on how it works: LINK
And as you can see in slide 9:

They managed to put the native google maps element behind the webview (IOS and Android) !!!
Hope this can help you fix this in the cordova-plugin-opentok because it is really important to have this ability.

Hope I helped you.

[msach22]

@Shtibel Thanks for this - I will check it out and let you know!

[Shtibel]

Hi @msach22 ,
Thank you.

Waiting for your check results :-)

[bijanmmarkes]

Hello @msach22, Any updates on this, It is critical for our company to overlay HTML over the publisher and subscriber video. If any help is needed I would be happy to contribute to your branch.

Kind Regards

[bimix]

Thanks @youssmak, I tried using https://github.com/msach22/cordova-plugin-opentok and it didn't help.
https://github.com/talkspiritlab/cordova-plugin-opentok works well you say?

How did you implement it to work? I removed the old plugin and installed this pull request but it doesn't build, I don't know what should I change in my code.

[KasunGamage]

@youssmak This works great on android.
On IOS it seems like the negative z-index isn't working. I get a dark screen. 😢 😢

Does this work for you guys?

any update on this? Is this the repository are u using https://github.com/talkspiritlab/cordova-plugin-opentok? any how is this repo fix the z-index issue in ios?.

[phiferd]

I was able to get controls to overlay the native code (only tested on iOS so far) using https://github.com/talkspiritlab/cordova-plugin-opentok (and they are clickable)

@msach22 Whey aren't these changes merged back into this repo? Is there some other issue with them? I don't want to be on an unsupported/unofficial fork if I can avoid it.

@bimix
If you're seeing a black screen while using a negative z-index, you probably need to add some additional css as described here: https://github.com/opentok/cordova-plugin-opentok/tree/master/docs

I ended up using the following CSS because I only want the app to be transparent when using the video feature:

body.video-session-in-progress .page.page-current,
body.video-session-in-progress,
body.video-session-in-progress .OT_root,
body.video-session-in-progress #app {
  background: transparent !important;
}

.OT_root video,
body.video-session-in-progress .page.page-previous {
  display: none;
}

when a video session is started, I added the "video-session-in-progress" class to the body, which hides or makes transparent a number of elements. When the user exits the video session, I remove the class and the app goes back to normal.

[ErxrilOwl]

It is 2020, any update on this?

[mozhn]

It is 2020, any update on this?

Unfortunately.

[cardo961]

Hi there? Been fighting up with this issue for some time now. Is there any update on this?

[msach22]

HI all,

We are not currently maintaining this plugin so I don't have any update on this.

[mozhn]

HI all,

We are not currently maintaining this plugin so I don't have any update on this.

Hi, what are you using?

[simplenetworks]

HI all,

We are not currently maintaining this plugin so I don't have any update on this.

What should Ionic developers use?

[habiba1997]

@sagivf I was able to achieve that layout by just this code

<div id="publisher" style="width: 30vh; height: 25vh; position: absolute; top: 0; left: 0; z-index: 1000; pointer-events:none;"></div>
<div id="subscriber" style="width: 100%; height: 100vh; position: absolute; top: 0; left: 0; z-index: 500; pointer-events:none;"></div>

Thank you it worked :D

[damianrejman]

It might help damianrejman/cordova-plugin-opentok@db1692b

[ramseyfeng]

Any update on this issue, it's 2021 September TT