Spy on browser through unity ? [Solved]

I've got an idea. I'd like to peek at what's happening in my browser. I'd like to somehow get the timer thing from youtube, best explained with picture:
And i'd like to track that, so i know where user currently is with the video. It doesn't really matter why i need it, i just need it. But this is probably almost impossible to do, i can't really even think of a way in my head. But just in case someone knows how to do this, i'd like to hear it. Thanks in advance.

 

Seeing as Unity is more for animation and game design, I don't think it's your best option for what you want to do. Maybe someone has written a Chrome extension for what you want to do? You'd have to look around and get some other opinions but I think that using Unity to do this would be quite a hassle.

 

Yes i forgot to mention that. I know unity is a horrible engine to do this in, but i don't have time to waste on learning new tools. I'd like a way to do this in unity.

 

1) Why do you want to do this? What will it be used for?
2) Does it need to be the users normal browser or a browser within the application?
3) If your answer to 2 was a browser within the application then try checking whether you could achieve this with Awesomium. Its a SDK for displaying web content.

 

It really does not matter why i need it. Sorry but it really doesn't. I explained what i need, but what i'm gonna do with that output is my problem. I want to track a browser, like chrome, not within the application. I found "Selenium webdriver", i think it could work but since unity doesn't support higher .NET, i cannot use it :/ "System.Drawing" is not in the global assembly chase, therefore Selenium cannot be installed... If i cannot track the timer thing, i'd atleast like to command my browser to pause/play the video in the tab. I'd basicaly like to control pausing/playing the video that's in the chrome tab THROUGH unity.

EDIT: Still trying to get Selenium to work. But it keeps saying "System.Drawing does not exist", even though the System.Drawing.dll exists if you search for it in the library. What the heck is going on ?


EDIT: Never mind. Seems like unity doesn't want to use it because of OpenGL and DirectX. But i need it in order to install selenium into my project. God damn it.

 

I had a quick look at the page for this "Selenium" that you are trying to use and I assume it would be much easier and better to assume that Visual Studio is much better suited to your needs than Unity. If System.Drawing is not playing nice could it have something to do with Unity using an old version of .Net/C# etc. If so, I would HIGHLY suggest you to switch to Visual Studio for this adventure.

However, as you are reluctant to explain what it is you would like to do, there is no way for me to truly grasp any specific routes to take to achieve it (whatever "it" is you want to do).

(And YES it would help to explain what you want to do).

 

I just never worked with anything beside Unity and it's library, monobehavior. It would be an overkill to try and learn regular visual studio now to do all of this. I need to learn so many things in order to achieve the system i created in unity. And yes, i even tried putting System.Drawing in my assets folder just now, and visual studio said it's using .NET 4.0, while unity is 2.0. So yes, unity will not work with selenium... I never get any third party APIs to work with unity because it's not supporting higher .NET framework versions -.- .

Here's the whole idea of mine, if you really wanna know it. It's so long but simple at the same time. I want to create a music player that will collect data from youtube playlists, get all of the URLs of each video from a playlist, get all of the titles of every video from playlist, store it all under a list of a custom class. Now i shuffle the list, and i want to open the URLs on my chrome browser, which i did using "Application.OpenURL". But it creates a problem now. I cannot automatically play the next song when it finishes, this is why i wanted to track the timer in the first place, remember ?
I came to an idea for automatically playing next songs, and it kinda worked. I made a countdown timer. I stored durations of each video into the list aswell. So unity counted the duration of video, and once it counted the right lenght, it would play the next one. I made that, but there's another problem. Now, if i want to pause the video, unity will still think the video is running, because it doesn't actually track where the video currently is at, or if it's paused or not. The timer will count down regardless of anything, because it doesn't know what's actually happening in my browser, nor it can command it to do anything without an API like selenium.

If you're wondering how i collected all the info. Well i used python for that, i used youtube API with python to collect the video and playlist data from youtube. But that's as far as i want to go with python. I don't want to create everything in python, that's too much work. I don't know anything about the language, i barley squeezed the videos info and wrote it on notepad and had unity read it from notepad.


I probably wrote all this for no reason because there isn't another solution than to switch to something else than unity...


EDIT: Oh, and if you're wondering why am i even creating all this when youtube has it's own playlist system already. Well, i don't like it. Their shuffle doesn't work properly, i want the order to be different every time but it just doesn't work in their system. And i thought it would be a fun little project to do, and turned out to be a nightmare because unity can't support any external APIs almost.

 

Perhaps you can make a brower specific plugin that communicates to your unity application, using websockets?

 

Um. I'm not familiar with websockets, what is that ? Quick googling didn't explain it.

 

Networking, with it you can communicate with other computers or other processes on your own computer. Its what Unity itself uses to send data to your profiler or frame debugger for example.

But, not to burst your bubble here, i dont think there is a solution with the constraints you have put on yourself. Solely using Unity this is fairly hard to do.

Maybe you can go deeper into the OS and detect 'Youtube' like activity. Like audio coming out of the speakers or network activity towards youtube.com. Your app could also communicate with the youtube API to receive the duration of a video

 

Yeah i'm gonna abort the mission then. Thanks anyways. I'll wait to see if someone has a better idea in mind.

 

If you are tracking time already and have a Pause messing it up then just set the pauseTimerStart = Time.time and when unpaused then get the pauseTimerStop = Time.time, determine the duration of the pause and subtract from the videoElapsedTime (time passed since the video started playing) to determine how long the video has been playing.

 

The thing is, unity can't know when i paused the video, as i said before... (As i'm not pausing the video using unity, but just by clicking pause in the actual browser)

 

If the videos are all using sound then you can do a frame check every second (for performance sake) or so and if you get the exact same value in two frames then the video is paused. Then start the pauseTimer and keep checking until the video parse gives you a different value. This kind of check can also be done using video frames and getPixel values and comparing and even allowing a delta value that needs to be exceeded to cause pauseTimer stopping and starting.

 

So there's a way of tracking sound of external applications ?

 

Hmm..sorry..missed that. Try looking in the System.Info class for speaker sound activity. You can get info like driverCaps and such so they may have a way to detect the db coming from a speaker at the current time.

Look at this for a possible solution
http://stackoverflow.com/questions/...hat-video-was-opened-in-player-with-net-c-c-c

 

Yeah but i might be playing other sounds at the same time on my PC though. Is there a way to track sound from a specific application rather than everything ?

 

Honestly if you're spending all this time trying to come up with hacky ways to do this in Unity then you'd probably be better off just writing a simple web page/app that uses the Youtube API (https://developers.google.com/youtube/iframe_api_reference)

 

Would probably be easier to make a player that downloads the videos in realtime from youtube and plays them inside unity

 

I think this was the right decision. It isn't worth the effort and you should use something better structured for what you are trying to do. Tools should work for you, you shouldn't be doing work for the tools, and programming tool-sets these days are ridiculously easy to install and use.

There are quite literally a hundred easier ways to do this.

Are there easier Unity ways? Well, I could recommend leveraging WWW, a simplified Youtube API, custom browser approach, internal browser approach, abstraction web layer, web socket, web service, a custom WebGL build or a local abstraction page. But I don't have the time or interest in elaborating on those when I don't know enough of what you are doing.

Going specifically on your ask, Unity monitoring an HTML5 and Flash wrapped text box on any browser, any tab and any page specifically for youtube - sure. Unity supports extensions. Create an extension as a DLL to add whatever functionality you feel is missing for every platform and browser you want to support. Even better, create an extension that just does packet sniffing and block them from accessing https for youtube. Native extensions get Unity to do something isn't meant to do and to "extend" its functionality. You want a round peg in a square hole, and the Unity team will let you cut it into a square peg

 

Yes i realized that. That's why i made this post, to see if people have a "non-hacky" way of doing this.

I see.

Please don't think i'm ignorant. I read everything and did my research on everything you mentioned. But yes, this will not be worth it for just a simple custom youtube playlist player. Yes it would be really nice to make it but all of these ways are just an overkill. I'll be better off if i were to use regular visual studio without unity, target .NET 4.5 and use all the APIs i need to make it way easier for myself. But it will take time to get used to regular C# without monobehavior library. I still think it will take less time than doing this in unity, but i'm probably gonna abort the mission anyways, because neither of the methods are worth it for such a small idea, and i soon won't have time to waste on that.

 

Somehow I doubt that.

 

Whatever... Don't expect me to try proving it to you. I don't know why you even reply with useless messages like that.

 

Is this what you were trying to do https://diaryofdennis.com/2015/01/0...e-device-as-remote-control-for-youtube-on-pc/