CSE 566 Virtual Reality

Spring 2023

Prof. Arie Kaufman

Assignment 1: Basic VR

Credit: 19% of final grade

Due date: Monday, February 20, 2023, 11:59 p.m. (Stony Brook Time)

 

Now that you have completed Assignment 0 and have gained some experience with working in Unity, let’s use those skills and what you have learnt in class to design an interactive virtual environment. And what better place to use them than our own Stony Brook University? You’ll be designing one part of it, which, when combined with your classmates, creates an expansive virtual “campus”.

 

For a true VR experience, we strongly recommend that you meet the following minimum hardware requirements:

●       A Google Cardboard (or similar)

●       A mobile device with either Android version 6.0 or higher or iOS version 13.0 or higher (note that for later assignments you might need a higher version of the OS).

 

 

Overview

For this assignment, you will be designing a simplified 3D virtual scene of a subset of Stony Brook’s campus. Your virtual scene should have a building, roads/sidewalks/paths, some vegetation (for example, trees), a bird, and a hot air balloon. The details about each object will be explained later in this assignment. Though we would love you to be creative with this first assignment, you may download models from the Unity Asset Store, or any other source, provided you have permission to download them and cite each source properly in your documentation.

 

The scene will start with the user standing outside the campus building you will construct, facing it. To be able to explore the virtual world, we will also implement some basic interaction using ‘gaze’. Don’t worry, we will learn together how to make that happen! By gazing at the hot air balloon basket, the user should “teleport” (jump) into the basket. If the user gazes at the burner on the hot air balloon (above their head), it should rise up into the air. From there, it lazily drifts towards the direction the user is facing. If the user wants to land, they can gaze below at the weight bags placed inside the basket, to start the descent of the balloon. To exit the balloon when landed, the user can stare at the bottom of the basket for five seconds. For all camera views, the user should be able to freely see the virtual scene from the new viewpoint, even while the balloon is moving. Additionally, the user should also be able to experience how the virtual scene looks during the day and at night.

 

In addition to the above scene, you’ll be submitting another copy of the scene which will be merged with your classmates. This copy will be the same as above, but with certain functionality/objects removed that might prevent the scene from merging well with others. For example, if each scene had their own moon in night mode, we’d have multiple moons in the night sky once merged!

 

After the assignments are all submitted, we will merge the scenes together to create a larger campus that you can explore in a later assignment!

 

If you are excited and can’t wait longer to start getting your hands on designing your (first) VR project, then read ahead for details, otherwise watch this video and this video to spark some excitement.

 

Virtual Campus Scene


As mentioned above, your assignment’s virtual scene should have:
 

  1. A terrain – Corresponds to the ground. We will be working off of the Stony Brook Campus Map, for which we have divided certain grid elements on the map into 3x3 tiles (see Fig. 1). Each student will reconstruct the terrain for one tile. You will be provided with coordinates (in meters) of your terrain, including its origin and dimensions in the Unity World Coordinates system (1 unit in Unity corresponds to 1 meter), as well as origin and dimensions in terms of the Campus Map (where the top left of the map is the origin - 0, 0). See table below. It is important to try to place and scale your terrain as accurately as possible, or else it will not merge well with your classmates’.


We will not be deforming the height of the terrain for this assignment – only using it to paint textures on. Click here for more information about painting terrain.

 

Please use the following textures for your terrain:

-        Grass: https://freepbr.com/materials/grass-1-pbr-material/

-        Road: https://freepbr.com/materials/pebbled-asphalt1/

-        Sidewalk: https://freepbr.com/materials/concrete-3-pbr-material/

-        Brick: https://freepbr.com/materials/rounded-brick-1/

-        Dirt Path: https://freepbr.com/materials/dirt-with-rocks/

 

For each, download the “-bl” version, even if there is a “Unity” version provided.        

 

Note: You will need to set the color/albedo/diffuse of the Terrain Layer, the normal map (be sure to set its texture type in Inspector to Normal map!), and the mask map for each Terrain layer. Refer to Mask and Detail Maps in the Unity Manual for more information on Mask Maps. You will need to create your own Mask Map for each texture using the Metallic, Occlusion, and Smoothness (Inverted Roughness) textures. You do not need to create a Detail Map. A free, useful tool for this is Channel Mixer/Packer.

 

Note: You may need to disable/reduce the effects of fog in order to properly see your terrain from a distance.

 

If there are other buildings on your tile other than the one you were assigned to create, you can treat them as though they don’t exist, and simply put grass in that area.


Fig 1. Campus Map Areas

 

  1. An object serving as your Character – depending on what framework / headset / controllers you’re using, this will vary. However, this Character object should be designed as a prefab, a template that can be imported into multiple scenes. This should include whatever camera object you’re using (note: it should be first person view!), as well as scripts involving interactivity.

 

  1. Hot air balloon – this can be made with a downloaded model or using simple 3D geometry, such as a sphere for the balloon, a box for the basket, cylinders for ropes/mounting poles, two or more weight bags inside the basket, and a cylinder for the burner. The balloon should also have two spot lights at either end of the basket which shine down on the ground, and a point light underneath the burner; these should turn on during night mode. These lights should have associated models, such as a light bulb or actual spotlight model.

 

Once constructed, the balloon should be also turned into a prefab asset.

 

Fig 2. Example of a Simple Hot Air Balloon; this would be missing the spotlights and burner (presumably weight bags are in the baskets).

 

  1. One campus building – You will construct a basic replica of one campus building, based on the subsection of campus you are provided, as follows:

 

Tile(s)

Building

Top Left Corner in World Coordinates

Bottom Right Corner

A+B

Kenneth P. LaValle Athletic Stadium

(980.29, 0, 967.52)

(1,136.75, 0, 1,286.83)

C

Student Health Center

(980.29, 0, 1,286.83)

(1,136.75, 0, 1,452.87)

D

Sports Complex

(1,136.75, 0, 967.52)

(1,296.41, 0, 1,127.17)

E

Walter J. Hawrys Campus Recreation Center

(1,136.75, 0, 1,127.17)

(1,296.41, 0, 1,286.83)

F

Stony Brook Union

(1,136.75, 0, 1,286.83)

(1,296.41, 0, 1,452.87)

G

Physics

(1,296.41, 967.52)

(1,456.07, 0, 1,127.17)

H

Chemistry

(1,296.41, 0, 1,127.17)

(1,456.07, 0, 1,286.83)

I

Staller Center for the Arts

(1,296.41, 0, 1,286.83)

(1,456.07, 0, 1,452.87)

J

Earth and Space Sciences

(1,456.07, 0, 967.52)

(1,625.31, 0, 1,127.17)

K

Frey Hall

(1,456.07, 0, 1,127.17)

(1,625.31, 0, 1,286.83)

L

Psychology

(1,456.07, 0, 1,286.83)

(1,625.31, 0, 1,452.87)

M

Light Engineering

(1,625.31, 0, 967.52)

(1,784.96, 0, 1,127.17)

N

Student Activity Center

(1,625.31, 0, 1,127.17)

(1,784.96, 0, 1,286.83)

O

New Computer Science

(1,625.31, 0, 1,286.83)

(1,784.96, 0, 1,452.87)

P

Heavy Engineering

(1,784.96, 0, 967.52)

(1,944.62, 0, 1,127.17)

Q

Engineering

(1,784.96, 0, 1,127.17)

(1,944.62, 0, 1,286.83)

R

Computer Science

(1,784.96, 0, 1,286.83)

(1,944.62, 0, 1,452.87)

 

The origin for the campus map coordinate system is the top left of square A1 on the Stony Brook campus map (See Fig 3.). The Unity world coordinates have been determined using measurements from the campus map (converting from miles to kilometers to meters).

           

Fig 3. Coordinate System on Campus Map, In Terms of Unity Coordinate System

 

The subsection and the buildings are shown in Fig 1. Note that the terrain you’re reconstructing may not fully cover the area of the building that you’re reconstructing! In that case, you’ll need to create two or more terrains (tiles) – one unified terrain for your tile area, and other terrains/planes to support your building (which can simply be textured with grass). Please use the Assignment 1 spreadsheet to sign up for the building you would like to construct. We encourage you to select unique tiles of terrain to your peers.

 

The modeling of the building can be fairly basic, but it should approximate the shape of the building. However, there are a few critical features you must correctly replicate.

-        The (approximate) length and width of the building. This can be retrieved using Google Maps on Stony Brook’s campus, right clicking the map, and using “Measure Distance” (PROVIDED IN FEET, NOT METERS). However, you can download the following file for more precise measuring with the free software Google Earth Pro, which overlays the campus map that we provided onto the Google Earth map.

o   There are also some addons for Unity that you might find helpful for measuring in that space, such as this addon.

-        The approximate location of the building in World Coordinates. This can also be achieved via Google Maps or Google Earth Pro (recommended).

-        The (approximate) height of the building – it doesn’t have to be precise, but it should be in the ballpark area. There are a number of methods you could use to find the height of a building (Googling, using your thumb, etc.). Please give in your report reasoning as to why you chose this height.

 

For the interior of the building:

-        Don’t worry about modeling the interior, apart from having a floor, walls, and a ceiling. The interior walls, ceiling, and floor do not need to match the true layout of the building, but it needs to be realistic (e.g., you can’t see through walls to the outside).

-        However, the buildings should have a light source placed inside them and the outer face of the buildings should have at least 2 large transparent windows or many small ones to allow light to shine out during the night mode.

-        Be sure that the lights in your building will work with the provided Day/Night component, regardless whether other objects are added/removed to your scene.

 

The building should have colliders attached so that the balloon cannot travel through it.

 

Extra credit will be given for those who go above and beyond in their modeling – please write in your report how you did so.

 

 

 

Fig 4. An example of a 3D building
 

  1. Paths – You have been provided with links to several textures to use for paths (see the section on Terrain). Within your subsection of campus, you should replicate the following types of paths:
    1. Roads (no traffic markings)
    2. Sidewalks
    3. Brick Paths
    4. Dirt Paths

 

If there are stairs/terrain differences along paths, ignore them, and treat your entire subsection as a level terrain. Similarly, do not create noticeable “raised” areas for sidewalks or something else – everything is level. We recommend painting path textures at the appropriate locations, but you can use alternative approaches.

 

Be careful with paths at the edges of your campus subsection! Measure as accurately and precisely as possible to ensure your paths/roads/sidewalks will line up with those made by others. It may be beneficial to contact your “map neighbors” and agree on some common coordinates for paths that cross between your tiles.

 

  1. Bird - The bird can be a downloaded model or designed using 3D polygons (example shown in Fig. 5). The bird should be in the sky and have a constant continuous motion. You are free to design a path in a script for the bird’s motion, however, the path should demonstrate all quaternion rotations. Once the bird completes its path, it should restart its motion automatically from the starting point. You can choose any speed for its trajectory. The bird’s wings do not need to be animated.

 

Fig 5. Example of a simple bird. Source: https://sketchfab.com/3d-models/flying-bird-unity-21a70684bbe6441194f87fa46f5d653c

 

 

  1. Vegetation - There should be at least 5 trees in your virtual scene. You may add other kinds of vegetation as well, for example grass patches, flowers, bushes, to make your tour scenic. The trees can be a downloaded model or a simple 3D tree designed using a 3D cylinder and cone or sphere (see Fig. 6). If downloaded, try to use realistic vegetation. Larger trees should have colliders attached so that the balloon can’t travel through them.
                                                                           

 

Fig 6. Simple 3D Tree





Day and Night

Your scene should include the “Day Night Mode” prefab, included in this linked package, which you can import into your scene. The prefab should be placed in the scene root. You should NOT need to modify this prefab or its script at all - just use it to toggle day/night mode.

 

Your scene should start during the day. For day-time, you will need a directional light source placed at infinity to act as the Sun and the Sun should start from the ‘east’ and travel to the ‘west’. You are also expected to code for shadows and the shadows’ direction should change as the sun moves. The day/night cycle should be scripted by time, for example the day/night mode can be 100/100 seconds long.

 

For the night mode, you should have a moon that illuminates the scene. Unlike the sun, your moon can be stationary. All the lights of the buildings, and the spotlights and point light of the balloon should be turned on during the night mode. The intensity of the moonlight should be enough that the buildings and balloon are visible. You may design your virtual scene as beautifully as you want for the night mode, so that a user may enjoy the night-time view of the balloon ride.

 

Make sure that your scene lighting (e.g., sun/moon, NOT building/balloon lights) is a root-level component in the Scene Hierarchy, so that it can easily be removed by us when merging the scenes together.


Camera and User-view

The user should be able to explore the virtual scene in two different modes: the default scene mode and the tour-ride mode.

 

Scene mode (default): The camera (i.e., the user’s eye) should be initially located and oriented such that part of the building and at least 2 trees are within its frustum (visible). The camera orientation may change as the user navigates through the scene. Thus, this camera mode should cater for head rotation. Input from your mobile device’s gyroscope will be used to adjust the parameters for the camera orientation.

Tour-ride mode: Once the user is in the balloon, they can either look at the weight bags at the bottom of the basket to exit, or at the burner to cause the balloon to begin to rise. The balloon should be stationary unless the user is inside it. If the user looks at the basket, the burner turns on – this should be demonstrated using a fire effect located on top of the burner. The balloon also slowly rises into the air until it reaches a predetermined height. While in the air, the balloon lazily drifts in the direction that the user is looking. To start the descent, the user gazes at the weight bags placed inside the basket, to start the slow descent of the balloon.

 

If the balloon collides with another object when moving – for example, a building or a tree – it should not go through it, but instead be blocked by the object. This means that “landing” on top of a tree or building is possible. The user should be able to get out of the basket on top of the building (if the roof is flat), but not on top of the tree – to fully land, they should move the balloon horizontally with their gaze until they reach the terrain.

 

To exit the balloon when landed, the user can stare at the bottom of the basket for five seconds.

 

 

User-mode switching

The main interaction in this assignment will be to switch between the modes. In VR terms, this instant change in the user’s position / location is called teleportation, which we will learn about later in this course. For the scope of this assignment and given our hardware limitation, the simplest implementation of teleportation or user-mode switching is by gaze (staring at an object).

 

To start the tour-ride mode, the user should gaze at the balloon’s basket, say for at least 5 seconds. This can be implemented using Unity’s Ray casting functionality (for reference, see Gaze in Unity). To help the user interact, a loading icon should appear when staring at an object to show the user that she/he is ready to start the mode. If the user looks away, the loader should disappear and the timer reset.

In order to exit the tour-mode and return back to the default scene mode, the user should gaze at the bottom of the basket for at least 5 seconds. Similar to starting the tour-mode, a loading icon should appear when staring down, to show the user that the program is ready to enter the default mode. If the user looks away, the loader should disappear, and the timer reset.

 

Project File Guidelines

When submitting your project, you should provide two copies of the scene. The first has a tile with vegetation and paths, a building and any necessary terrains/planes to support it, full interactivity, the balloon placed within the scene, full night mode, etc., and should be named “<your_tile_letter>_full.unity” (e.g., “C_full.unity”)

 

The second only has your assigned tile with the building, paths and vegetation. It is okay if the building in this scene is not completely “supported” by terrain within your tile, since it will be supported by your neighbors or terrain we provide. The scene should be named “<your_tile_letter>_merge.unity”.

 

Additionally, you should provide your Character, your Building, and your Hot Air Balloon as prefabs. The prefabs should be able to be imported into a completely new scene, and still be able to function, without any references to your main project! (Although you can assume both the Character and Hot Air Balloon will be correctly imported/connected together).

 

Some other guidelines

Before starting this assignment, if you are not familiar with Unity and have not tried out Assignment 0, we strongly suggest that you go through Assignment 0 and complete it first. Additionally, check out the Unity Manual and see the Unity reference page on Input for a comprehensive overview of its functionality.

Before starting the assignment, think carefully about how you structure your scene’s object hierarchy. Begin by designing each object separately and test that your camera works fine. Then, put all the modes together in your virtual world and test out your teleportation and navigation.

 

Submission

It is strongly recommended that you test your application before submitting your work. Please save your Unity project in a folder named “CSE566 Assignments” in Google Drive and enable the share of this folder rather than sharing the single project file. In Brightspace submission, please submit the link to your Google Drive folder.

 

For this assignment, include the following for full credit: your Unity project folder, a report, and a video. Your Unity project folder should contain your Unity saved scenes and all your Scripts and Assets that will be required to rebuild your project. Please do not include your project executable in the folder.

 

You will also need to upload a video recording of your application. To do that, turn on screen capture on your phone and run the application. Interact with your application and make sure you show all aspects of the assignment: head rotation, gaze, teleportation, day/night mode, and all required VR objects for this scene.

 

For your report, you should include at least the following:

-        A title: “CSE 566 Virtual Reality, Spring 2023, Assignment 1: Basic VR”

-        Your name and Stony Brook ID

-        Unity version

-        Hardware used

-        Directory hierarchy

-        Any extra functionalities/ features that you implemented for this assignment

-        Details on implementation: references to the downloaded 3D models or how you designed your own model; how the motion paths in your scene were implemented; how the lights and shadows in your scene were implemented; how the scene transition was implemented for switching between the modes, to mention but a few.

 

Extra Credit

Extra credit of up to additional 10% of the assignment value in the final grade will be given to extra functionalities and/or features that you implemented for this assignment, including particularly detailed building models. Please describe the extra functionalities and/or features in your report.