Link

This selection technique is a mix between a bubble cursor and a Ray-casting method. The idea is to chose the closest object to the ray going from the controller. (Closest being either euclidian distance or angular distance)

• Reach: infinite (the ray can be as long as needed)
• Cardinality: Single
• Progressive refinement: Continuous (The bubble changes size continously)

SQUAD

Link

In this Selection techniuqe, you begin by using a sphere to create a encapsulate loosely your object. The sphere position is determined by a ray casting method. The radius of the sphere being higher the furthest it is from camera.

But because you also might have other objects in the sphere, You need to refine your choice by selecting where is your object among the 4 quarters.

You do this until you only have your object.

• Reach: infinite (the ray can be as long as needed)
• Cardinality: Single
• Progressive refinement: Discrete (Tou divide the number of items by 4 each time)

iSith

Link

In this Selection technique, you have one ray on each hand and you select the object closest to the projected intersection of this rays.

• Reach: Technically infinite (the rays can be as long as needed, but the intersection is hard to finetune if far away from the handles)
• Cardinality: Single
• Progressive refinement: None

1. What is the difference between the CAVE System and Head-Mounted Displays? What are their advantages/disavantadges?
2. Where do you think the CAVE fits into the Reality-Virtuality Continuum of Milgram.
3. Does the position in the Reality-Virtuality Continuum changes when more people are experiencing the CAVE system at the same time?

Comparison with HMD

The CAVE System is really different from a HMD: it consists in multiple screens placed around the user to represent the virtual world. It displays a picture based on the position of the user so that it matches the right perspective.

Let’s look at the major points that differ from the HMD:

FoV and Panorama

Because the CAVE System is all around the user, it have a full Field of View and allows the user to look in any direction as fast as he wants, contrary to the HMD that need to recompute the scene at each move and that have a limited FoV.

Body representation

The HMD uses a virtual body whereas the user can see himself in the CAVE system. This can be either a avantage or a disavantage depending on the use case:

• The body is realisitic and perfectly accurate
• The occlusion cannot be changed, meaning the user will never be able to see something close or pick up an object for instance

Progressive Refinment

The HMD basically needs to recompute a new scene at each frame, meaning that it will not be able to create a finer render. But if the user does not move it’s head to much on a CAVE System, the image can be refined more and more.

This would be really intresting today, as with a bit of time, we can produce photorealistic results and we can use AI to upscale the other results in the mean time, as done in the blender viewport. This could be a real advantage for 3D artists that want to work in VR.

Collaboration

Because the CAVE System is less intrusive, a small number of user can collaborate on the same device, whereas they would need to have one HMD each to collaborate.

Other points

The paper also pinpoints other differences, but I feel they are not up to date anymore, for instance the time needed to track the head position, the bad resolution of screens, etc.

Conclusion

I think the CAVE System is an intresting alternative to HMD, especially for artists who needs to have something that can produce photorealistics renders and need to have a non intrusiove device for long sessions.

But it stills lacks the possibility to change the occlusion of the hands, which can be a real turndown for all the interactivity…

Milgram Continuum

The CAVE System is more of an Augmented Virtuality, close to only virtual, as the only real element you interact with is your body.

With more people

When you do collaboration, it tends more towards reality for two reasons:

• You have another real human being in the virtual world.
• The System alone cannot track multiple users at the same time, meaning other users will not have the same presence
  • With shuttered glasses, the System can quickly alternate between users. But it still can break illusion to notice the screen blinking.

This makes for a really intresting tool for collaboration as often the main issue with collaborating in VR is that you lose the reality and natural facial expressions of the other users.

Colision with static objects

In VR, the virtual world does not have the same collisisons as the real world. This can lead to issues, especially in small spaces, where a position empty in VR but have an object in the real world.

To solve this issue, I propose two solutions :

Virtual doubles

One option is to create virtual doubles of every object in the real world.

Each object in the real world have a virtual double and the user tends to avoid them.

There is several issues, one being that it requires a lot of assets to be created, the other being that it is hard to have an open space, for instance a meadow would be hard to create.

See through

When you move to close to an object, the object appears in your headset to avoid collision.

This is a good option, but it can break the immersion when you see the real object. Furthermore, if the player is doing a fast movement, he will not have enough time to react.

Loss of balance

For various purposes, you might want to have a different camera orientation on the virtual world than on the real world. This can lead the inner ear to belost and create motion sickness or loss of balance. Let’s see how one could fix this:

A rolling ball

One could use a ball that replicates the motion from the virtual camera in the real world, like New Atals does.

The Eight360 Nova

But this is really expensive and not everybody can have access to this.

Remaining seated

Another option is simply to ask the player to sit down before having a big rotation. This could be done by having a virtual double of the player real chair, or by asking him yo sit on the floor.

The concept

I implemented what I explained in the Elevator Pitch and added a few things that I will explain in more details.

Implementation

Selection

For the Selection, I took a ray from the player to the controller and multiplied it to have the virtual hand position. To select an object, you press the index trigger.

I had to make two choices:

• Where on the player would be the origin ?

I tried several approaches and found out that the neck was the most intuitive, as when you move you arm parallel to the ground, your virtual arm is parallel to the ground.

• How many times will I multiply the arm length ?

At first I felt like 3 times would be realistic, but it turns out that the SuperMarket assets are really big and therefore, I needed to make it at least 15 times to have something usable. I modified this value a few times with the rest of the implementation and had a coefficient of 20 in the end.

Improvement

I had an issue with this box:

On the first tests of this method, i realised this box was really hard to select beacause it is really thin and your virtual hand moves to fast. Therefore, I added a new fonctionnality: When you press the palm of your hand, you move your virtual hand more slowly, allowing for much more precision.

This is how I implemented it:

HandPosLeft = (HandPosLeft - PrecisePositionLeft) / coef + PrecisePositionLeft;

Where HandPosLeft is the position of the virtual hand as described earlier, PrecisePositionLeft is the anchored position and coef is the coefficient of the extension of the virtual arm.

But when I released the trigger, the virtual arm will teleport to the position it is supposed to be in. This was breaking the immersion, so instead, I added a smoothing function:

HandPosLeft = previousPos + smoothCoef * (HandPosLeft - previousPos);

This allows to reach the correct position quickly but be more fluid with a smoothCoef of 0.2.

Movement

I first need to detect when the virtual hand is underground so I can place the player in the movement mode. This is done with this piece of code:

if (playerOnGround && HandPosRight.y < 0)
{
  OnGroundRight = true;
  playerOnGround = false;
  rightPos = new Vector3(HandPosRight.x, 0.0f, HandPosRight.z);
}

Note that I will only do this if the player is on the ground, because otherwise, I need the player to move around the anchored hand.

Then, this is how I move the player:

if (OnGroundRight)
{
  CameraRig.transform.position -= (HandPosRight - rightPos);
}

To detect if the player is on the ground, I compare is current y position to the y position he had when the app began.

if (! playerOnGround && CameraRig.transform.position.y < baseY)
{
    CameraRig.transform.position = new Vector3(CameraRig.transform.position.x, baseY, CameraRig.transform.position.z);
    OnGroundRight = false;
    playerOnGround = true;
}

Improvement

This transition was really long:

So i added a simple feature to extend the arms length by a factor 2 when you press a button.

Both Hands

I wanted to have both hands able to do the same tasks, but juste adding the same code to the left hand added some complications: when the right hand is anchored and the left hand goes underground, the code would have both hands anchored and would create strange behaviours because it would try to move the player in two different directions at the same time.

To fix this issue, I created three different states:

• Right Hand anchored
• Left Hand anchored
• No Hand anchored

These three states are exclusive, therefore when the left hand goes underground, the right hand is released and there is no strange behaviours anymore.

Evaluation

I had two friends come over to test my implementation. User 0 is me, User 1 is a friend that never really played a video game and User 2 is a friend that already played some games in VR.

I asked them to rank from 0 to 10:

• How easy was it to learn and use the commands ?
• Did you felt present on the virtual environment ?
• How much fun did you have ?

Comments

I realised several things while they were playing:

• They did not use at all the precision feature, they always prefered to usse both hands: one to move themselves loosely to the object and the other one to make precision movements and select it.
• They always had one arm on the ground, they almost never touched the ground and when they had to, they were slower and less precise.
• Each time they had to move, they would extend the arms just for how funny the movement is, even if it would take longer and lead to less precision.
• The movement create a lot of motion sickness, which makes it almost unusable for more than 5 minutes…

Results

Here is the GitHub link.

Here is a video of the result:

I decided to immplement the method where have extendable arm. There is two differents parts to implement: one being the movement, the other being the selection. Let’s see how I intent on implementing, both of them.

Selection

The idea behid the selection is that you have a virtual double of your arm that is longer than your original arm, but but points in the same direction.

To implement this, I will look at the vector from the head to the controller and multiply it by some amount to have the position of the virtual hand. Then to select objects, you will be able to just grab them with a button on the controler.

Movement

To move, you just push your arm toward the ground and it will anchor in place, letting you orbit around the position of the virtual hand.

To implement this, I will check if the virtual hand is underground. If so, I will snap it back to the ground and move the player by the same amount I had to move the hand. Once the hand anchored, every frame, I will move the player and the virtual hand by the same amount in order to have the virtual hand placed at the anchor point. When the player goes down again and move past a certain threshold, I will free the anchor to let the virtual hand move as normal.

Both at the same time

I think the selection technique will reveal its true potential once the player is confident enough to select and lmove at the same time, one with each handmaking it both precise and fast.

Core idea

You move and scale a supermarket around you.

This is inspired by the hologram map in avatar, except you can scale it as big as you want and make it go all arround you.

Manipulation

You move in the scene by pinching with your hands as you would do on a tactile screen with your fingers. Then you move the supermarket arround you up to the point where the desired object is right in front of you, and here you just grab it

Goals

• Being more efficient
• Having a better spatial understanding

Evaluation

• Time
• Supermarket memory

Drawbacks

• Only single player

Doctor Octopus

Core idea

You have expandable arms to grab AND move

This is inspired by Otto Octavius in spiderman.

Manipulation

You have a second pair of arms which is an extension of your actual arms, but scaled up. Because they are bigger, you can push toward the ground and it will make you go up in the air. This way, you can move in the supermarket to get close to the product with one arm, and select it with the other one.

Goals

• Move less
• Movement in funnier

Evaluation

• Time
• Mastery of the environment (presence + memory of the Supermarket)

Drawbacks

• Hard to master
• Can lead to motion sickness

Magic powers

Core idea

You launch objects in the air then slice through mid air to select them, making a mess in the process

This is inspired by the video games Control and Fruit Ninja. Control for how you launch objects in the air and Fruit Ninja for how you select them once in the air.

Manipulation

You make a upward movement with your arms to lift object off the ground, then tyhy will magically float in the air, where you can select them by slicing through them with your hand or with a katana that you will have in your hand (to be able to reach further)

Goals

• Feel powerful
• Have fun

Evaluation

• Accuracy (Objects close together will be blown apart and easier to select in the air)

Drawbacks

• Lots of actions and coordination needed to pickup one item

Switching to PC

I am usually working on a MacBook Air, so I followed the instruction of the TD up to the point where I had to download the Oculus application, which do not exist for MacOS…

So I had to redo all of the configuration on my fix PC which I did not use for a while, installing Unity, visual studio, etc. Fortunatelly, I just had to follow the same steps as for my Mac installation and it was quick enough as I already knew how to do it.

I also reinstalled Hugo on my PC to make to be easier to make the blog from here. I actually had an issue to connect to the git server beacause I was logged to another old git account on my PC. If you ever find yourself in the same situation, you need to go to know that windows is handling the credentials, therefore you need to go to control panel > user accounts > credential manager > Windows credentials > Generic credentials and change the git user from here…

Using the Quest

Installing everything on the quest was straightforward, I just followed the instructions from the TD. Just note that the Oculus integration is now obsolete, but I decided to stick with it anyway, which proved to work. I then connected the Quest to my PC with the Oculus app, and I managed to make a basic scene rendered in the headset, but as soon as I quit play mode, the headset made a blackscreen and it was impossible to make it work again…

Therefore, I decided to buy a Quest 2, that I wanted to buy anyway, and I connected it via AirLink, which worked really well.

Roll a ball asset

I actually had to create a package from my mac and wetransfer it to my PC so I could use it. I then just had to import it.

Making it VR

To make it work in VR, I had to change the behaviour of the scene. The main idea is to attach a script to the hands anchors that will handle the button press. If the handle collides with the roll a ball obect and the user press the button, we attach their positions together.

In order to make this work, I had to introduce new colliders and to make two different layers that do not interact with one another so that the handle does not impact the game, just the rotation and the gravity.

It was hard to take a screenshot while being in VR, sorry for the bad quality

I followed the Unity tutorial named Roll a Ball. It was a simple Unity project to understand the basics of making a game.

Most of the setup is basic, most of the work is already done for you, you just need to open a template. The only issue is that I was not able to create a URP (Univesal Render Pipeline) project because my version of Unity is too old, so I created a more basic 3D poject. It worked exactly the same, except for the look of the scene editor:

URP Scene

My Scene

First scripts

The first script is really basic, you just need to take the player input and add a force to the ball. All of the physics is done via the rigidBody, you do not have a lot of control, but it is really simple to use.

The next script waas just for the camera to follow the ball, really simple too.

Collectibles

The collectbles are more interesting. What you do is you create a prefab that stores the material, the script, etc, and then you just instantiate multiple of them.

To create make the player able to collect them, we add a collider and a tag that will make the collision with the player run the OnTriggerEnter(Collider other) method. We deactivate the pick up via the player code on collision.

Furthermore, to optimize a bit the progrtam, we add a rigidbody to the prefab so that unity considers it as a Kinematic object instead of a static one, which speed up the computations when moving (which is relevant here because pickups spin at each frame).

Collectible Prefab

Build

To share thus game, you need to build it, otherwise the player needs to have unity installed on his device to be able to run it. Building is really simple in unity, you just chse a few settings, your platform and hit build. Here, we decided to make the game run in a window instead of full screen as it does not have a menu to close it yet.

In MacOS, every gamefile is grouped into a single .app file which can be runned by itself

My game running as a standalone

I already had Unity installed from another course. Because I did not want to use too much space, I decided to stay on the same version and just add the necessary modules for this course:

Once the modules installed, I created a new 3D project. The first thing i did was changing the background color while on play mode because I made too much unsaved changes on my previous project (every modifiations made while in project mode are discarded). This is done via the Preferences > Colors > Playmode tint:

You can immediately say when you are in play mode:

For the blog, I followed the Quick start tutorial of HUGO.

I changed the theme for the theme for the Hermit-V2. Therefore, I had to make a little setup in the hugo.toml configuration file.

Deployment

I deployed the site to GitHub by following the Host on GitHub Pages tutorial.

This is my first website !