Non-game Ideas

Let's discuss ideas for non gaming projects here

For non game, I can see 2 things that could have a big impact.

-Social VR: But as I said, I don't see how we can do better than the big players already doing that...
-Tools for artists: Imagine if the whole creation pipeline is done in VR/AR. If tools for modeling, rigging, texturing... or even a game engine were remade for VR, it could give a huge productivity boost for artist (at least for beginners). It would be great to do one of this tools. And i'm pretty sure we could use eye tracking as input. The good thing is that we don't need artist for that. But it'll will be hard for us to do that if we don't have input controllers... Do you have a razer hydra or something like that? I think there's a huge opportunity for that!

What about a visual HUD type interface for an OS? something that basically works as a standard HID interface and driver to provide mouse and keyboard support. You could put a "ghosted" keyboard icon near the top center/bottom center/bottom right corner of the display that when you look at it an blink (maybe blink twice?) it brings up a semi transparent keyboard that you can type with by looking at the keys and either focusing on a key or by blinking.
And when the keyboard is not in use it acts like a standard mouse. Blink to click. And then maybe make blink one eye or hold one eye closed to click and drag.

I could see something like that being very handy.

Good idea! It could be really useful to have a standard interface for keyboard and mouse. It could be even more useful if it support multiple input hardwares (fove eyetracking, oculus touch, leap motion...).

But I don't think "blink to click" is a good idea, you don't know when you're blinking and it can end up being really uncomfortable. I think that looking at a the display then pushing a button of a controller is a better idea. But of course we don't really know that for the moment, and we'll have to experiment that with the Fove...

Sooooo..... Long blink then?

Is anybody interested in doing something like a VR family tree viewer?
I was thinking it would be cool to be able to load a standard Gedcom file (which you can export for most programs and Ancestry website) and be able to see the links to people in your tree. By looking at their pictures or avatars you the program could bring up info about them or the periods in history where they lived?

l_c_r_ant wrote:Sooooo..... Long blink then?

Blinking gets very tiring, very fast. I have experimented with using a stomp-box style external button for "select". Or large arcade machine buttons placed under the wrists. I would only using blinking for the selecting action in the worst case scenario (i.e. if the user has no mobility beyond eye movement).
A VR compatible desktop environment when coupled with eye tracking would be a real boon to anyone who has limited mobility or lacking fine motor control.

To select when looking at things I like the apps that monitor how long you have the cursor over a button and then auto select after a short time, however this is not practical if you want to use it to type keys on a VR keyboard.

In terms of a VR desktop, there are two already out there that are very good, Virtual Desktop and Live a Rift View. I expect both of these to support the Fove and add eye tracking to select, so I don't think reinventing this wheel is worthwhile.

What about making a relaxation app and use the eye tracking to drive the mood/music/visuals of the app. Eg looking up will increase music volume/tempo, down the opposite. Looking left and right could change visuals or music style. We could then combine hand tracking (via leap motion) to navigate the space. I am thinking some quite spaced out.

Admin wrote:
In terms of a VR desktop, there are two already out there that are very good, Virtual Desktop and Live a Rift View.

Yeah, that's true. But there's no VR desktop for linux yet

I agree, It's not really useful to make a whole OS interface. But I think what is missing in all vr application/game, is at least a virtual keyboard. I'm pretty sure it could be really useful for developer if they could easily integrate a keyboard to type username,password, url and search request... It would be a shame if people with eyetracking or oculus touch had to remove their HMD to type a simple search request... We could make one in unity and people could use it in their projects.

I like virtual the OS idea, but as mentioned there are already some projects working on that. As Maz mentioned, blinking is to me a worst-case scenario choice for how to select things. Just because we're letting people use their eyes, it doesn't mean we should force them to stop moving their hands, right?

What about combining hands and eyes? Has everybody seen the Leap Motion demo where the user interacts with the app via a virtual wrist mounted menu system. This would be cool to do where the user selects by looking at the options for a short period of time, obviously this would still require a leap motion for the fore arm position.

What about creating a VR viewer for 3d point cloud data or Ground Penetrating Radar scans, both of these are used by archeologists to map dig sites, etc. this could be quite a cool project if we could get our hands on some real data. I have a relative who is an archeologist so that should not be a problem.

Could be awesome to experiment with eye+hand tracking. I'm not sure a lot of people will have a leap+fove to test it

I have a leap on my dk2 so can test that bit, I will probably buy another for use on the Fove when it arrives, they are quite cheap now. I keep seeing them on eBay for around £30 (US$50).

Well this is why I was suggesting it as a simple HID driver interface mainly for mouse control.

1. It could make integration into many different current programs very simple and only maybe require a few tweaks to make it work very well.

2. This would also be something to really open the applicable markets in to fields well beyond gaming or a simple VR OS.

As far as the blink-click idea goes I certainly wouldn't make it exclusive.  Just like you can plug in multiple mice and keyboards at the same time and they all work, so would this.  There would be nothing stopping providing a one/two/five button device that could be hand/palm/wrist mounted to provide the button clicking interface in addition to the blink-click.  You'd just have a mouse with buttons and no positional tracking system.

I mean from the little bit of programming work I did once upon a time, I seem to remember that the mouse cursor always had an X-Y location property associated with it.  So if you're rendering the desktop at a given resolution at a specific location (fixed or virtual screen type) with the eye tracking I wouldn't think it'd be that hard to figure out the corrisponding X-Y location on the screen the user was looking at and from there you would give the user two choices, absolute and tracking control.  In absolute, the driver would update the mouse pointer's X-Y with that of the location being looked at.  In Tracking mode you'd use the difference between where the pointer was and where the user was looking to move the pointer towards that spot at a distance driven speed, i.e. the farther the destination, the faster the cursor moves.

I mean I could see this being VERY useful in CAD software like AutoCAD as a basic mouse driver.  When you draw something, 99% of the time you are looking right where you need the action to take place whether it's drawing a line point to point or selecting a part.

I know the Oculus doesn't show the whole monitor when looking through the lenses, so icons up in the corners could be a problem if you just use it as a HID mouse replacement.

How about using eye saccades like input? I've read on google cardboard subreddit that they try to implement nodding as input - you bend your head towards your shoulder and inventory opens up etc.

How about SDK that provides regular artificial saccades as input, for example looking from left to right initiates walking. I understand this may be tricky.

So here is an alternative - a button on the left and right edge of the screen - if you fixate first on left than on right, and there is less than... 50ms? difference between fixations, it works as event. And you start walking.

For educational or exploring experiences that should be just enough. And it opens up a completely hands free self sufficient solution.