touchsreen navigation ideas

hebdemnobad · 13-Nov-2014 20:29:13

colleagues:

in implementing the modded 3d player i'm working on, i received very constructive advice from a friend with 30 years experience in 2d interface design, which is making it immediately evident to the user about how to navigate through an os3d scene on a tablet. i started up my modded player and he had no idea of how to interact with the scene (he doesn't play games and doesn't do any work with realtime 3d but he has had a long and successful career in interface design)

on at tablet or phone, you (or I at least) use two thumbs, using one thumb for rotating the camera and another for moving, switching rotate and move functions from one thumb to the other and back again. you hold the tablet/phone with two hands with your thumbs moving around in the lower center of the screen

even for a user like me who is used to using web3d viewers of various kinds, using a keyboard is not practicable, since it is small, you need to invoke the full keyboard emulation of a pc keyboard, and the keyboard takes up valuable screen space. add to that the fact that many tablets aren't very big, and the arrows are smaller than a full grown person's index finger, let alone thumb

so i'm thinking of a user navigation interface that allows the user to use 2 thumbs with arrows and a trackball, something like this:

up arrow to move forward

<<<<<left arrow to turn left ********** right arrow to turn right>>>>>>>>>>>>>>
* *
*trackball *
* to look *
*up and *
* down *
**********
down arrow to move backward

my website is down for a couple of days so I couldn't post an image....
i welcome any advice/pointers/code/links to implementing the trackball element of the interface to rotate the camera shell around the camera x axis...i think i can figure it out myself but if anyone has some code lying around, feel free to post. i will then post my implementation of it.

i also welcome input on the advantages/disadvantages of dedicating an os3d viewport to the display the navigation controls as ogre .mesh objects, or dedicating an os3d viewport to hold a flash/webnavigator widget with a 2d interface painted on it, or using existing 2d scol api with bitmaps and a rendering of a 3d trackball as a 2d bitmap.
***
an alternative to simplify things further would be to use the trackball to rotate around the camera shell x axis, and the camera shell y axis, so the trackball would allow the user to look up and down, and rotate the camera shell left and right along the camera shell y axis...in this alternative there would just be 3 controls. the trackball for looking up and down and turning left and right, and one up arrow to move forward along the camera shell z axis and a down arrow to move backward along the camera shell z axis.

i'm thinking aloud here, but if anyone has any suggestions, feel free to share. as i wrote above, i'll post the complete navigation code to this thread.

for fps navigation, i think the first alternative fits in with the existing plugit, since mouse position does camera rotation only, and other keys/interface elements move the camera

arkeon · 13-Nov-2014 23:03:46

for touch screen you should use the walkthrough plugIT not the FPS one.
this way you can rotate and move with one finger.

OS3D ha tuio implemented for multi touch, but it should need a driver on windows to convert touch infos to tuio (vrpn protocol)

I could make a navigation system based on FPS where the screen is splited in 3 zones left for camera rotation , middle for action and right for translation. But I'll need a windows tablet to test and implement native windows touch inputs.
Will wait until next year for that.

hebdemnobad · 13-Nov-2014 23:44:11

arkeon wrote:

for touch screen you should use the walkthrough plugIT not the FPS one.
this way you can rotate and move with one finger.

Yes... I'm thinking:
up arrow down arrow trackball, in order determined by user, all in a horizontal line.

Up arrow moves forward, down arrow moves backwards along camera local z axis, track ball rotates camera along local x axis (looking up and down from horizontal center of trackball) and rotates camera along local y axis (turning left and right from vertical center of trackball). That way you can look up or down without moving forward or backward and you can move forward and backwards without looking up or down. And allow user to configure the controls (where the arrows and trackball are positioned left to right.)

Perhaps that will do ok with two thumbs...

hebdemnobad · 14-Nov-2014 01:17:43

I will first implement this ui as a plugit, so users can customize the interface and allow interaction with other plugits.

arkeon · 14-Nov-2014 09:05:01

for now only one finger at a time will work.
since only the mouse will be available here.

I'll look to implement window multitouch in the lib2dos dll

hebdemnobad · 14-Nov-2014 14:17:37

arkeon wrote:

for now only one finger at a time will work.
since only the mouse will be available here.
I'll look to implement window multitouch in the lib2dos dll

Is this a feature you will be putting in the upcoming VM realease , or will it wait for 6.3.2?

Last edited by hebdemnobad (14-Nov-2014 14:18:21)

arkeon · 14-Nov-2014 14:20:58

I give it a try right now

hebdemnobad · 14-Nov-2014 14:38:04

arkeon wrote:

I give it a try right now

Wow Thx!

arkeon · 14-Nov-2014 15:10:21

I did it with a plugIt, but have nothing to test it for now

arkeon · 14-Nov-2014 15:19:21

I'm installing tuio on android and windows tuio driver to simulate touch screen on windows.
will see if it works

hebdemnobad · 14-Nov-2014 15:38:00

arkeon wrote:

I'm installing tuio on android and windows tuio driver to simulate touch screen on windows.
will see if it works

wonderful!

arkeon · 14-Nov-2014 18:40:10

WIP

hebdemnobad · 14-Nov-2014 18:58:33

that's pretty fast implementation, even if wip...

Last edited by hebdemnobad (14-Nov-2014 18:59:18)

arkeon · 14-Nov-2014 19:31:08

I'm integrating the new callbacks for plugits so it will be shared and I will be able to manage it in navigation plugits.

_CBwinTouchPointAdd
_CBwinTouchPointRemove
_CBwinTouchPointUpdate

arkeon · 14-Nov-2014 22:58:38

All done ^^

the input multi touch plugit allow to show the finger positions on 3D buffer.
you can also use it to get tap / hold / swap events.

the FPS navigation plugit now manage multi touch.
for now the left part of the screen manage translations and the right part manage camera view

combined with multi touch plugit you can use tap for jump, hold with a sequence for crouch or run for example.
sure it will be better in full screen.

plugits can now use :
setPluginInstanceCbTouchPointAdd inst mkfun6 @cbAddCursor obstr;
setPluginInstanceCbTouchPointRemove inst mkfun4 @cbRemoveCursor obstr;
setPluginInstanceCbTouchPointUpdate inst mkfun8 @cbUpdateCursor obstr;

tell me if it work as expected. (I tested with my android phone for touch inputs, so the move ratio on fps plugit can be too fast or too slow)

you can download the new beta here :
first : https://www.arkeon.be/svn-scol/trunk/se … plugin.exe
then : https://www.arkeon.be/svn-scol/trunk/se … _setup.exe

hebdemnobad · 14-Nov-2014 23:39:57

Wow I'll try this tonight!

hebdemnobad · 15-Nov-2014 14:38:13

it works congratulations and thx!!
I have some questions about it. will ask later in weekend.

arkeon · 15-Nov-2014 15:59:12

great

hebdemnobad · 15-Nov-2014 17:54:48

wow and you are able to enable and disable touch input...that's great to offer the user the choice of using something more obvious (2d ui/widgets on 3d scene (if widgets remain enabled when cbAddCursor is not called..I will have to test that out)...after being made aware of this by this by people who are not familiar with 3d viewers) or more accurate (when multitouch is enabled)

one issue i noticed and it may not be an issue.....if i keep my finger on the screen on the left for example to translate the camera, and move it to the right half of the screen, that finger will continue translating until i remove my finger. and vice versa for camera rotation..that's the way it's supposed to work, right? (i don't have any experience with fps touchscreen games so i wouldn't know if this is what you want or not...)

Last edited by hebdemnobad (15-Nov-2014 18:02:11)

arkeon · 15-Nov-2014 21:03:40

yes I made it like a "joystick" so you can augment the move speed going more on the direction.
and you can reduce by moving to the hit point again.

this is almost what they do on tablet FPS games

Scolring - Forum

#1 13-Nov-2014 20:29:13

touchsreen navigation ideas

#2 13-Nov-2014 23:03:46

Re: touchsreen navigation ideas

#3 13-Nov-2014 23:44:11

Re: touchsreen navigation ideas

#4 14-Nov-2014 01:17:43

Re: touchsreen navigation ideas

#5 14-Nov-2014 09:05:01

Re: touchsreen navigation ideas

#6 14-Nov-2014 14:17:37

Re: touchsreen navigation ideas

#7 14-Nov-2014 14:20:58

Re: touchsreen navigation ideas

#8 14-Nov-2014 14:38:04

Re: touchsreen navigation ideas

#9 14-Nov-2014 15:10:21

Re: touchsreen navigation ideas

#10 14-Nov-2014 15:19:21

Re: touchsreen navigation ideas

#11 14-Nov-2014 15:38:00

Re: touchsreen navigation ideas

#12 14-Nov-2014 18:40:10

Re: touchsreen navigation ideas

#13 14-Nov-2014 18:58:33

Re: touchsreen navigation ideas

#14 14-Nov-2014 19:31:08

Re: touchsreen navigation ideas

#15 14-Nov-2014 22:58:38

Re: touchsreen navigation ideas

#16 14-Nov-2014 23:39:57

Re: touchsreen navigation ideas

#17 15-Nov-2014 14:38:13

Re: touchsreen navigation ideas

#18 15-Nov-2014 15:59:12

Re: touchsreen navigation ideas

#19 15-Nov-2014 17:54:48

Re: touchsreen navigation ideas

#20 15-Nov-2014 21:03:40

Re: touchsreen navigation ideas

Board footer