The Virtual UCF Arboretum: Using Beauty and Knowledge to Engage Learning

The Virtual UCF Arboretum Project

Overview

The Virtual UCF Arboretum Project is an immersive, multimodal, interactive, virtual reality (VR) and augmented reality (AR) application design, development, and informal learning evaluation effort. Important to the educational outreach mission of cultural institutions, arboretums, botanical gardens, and natural history museums, this work pioneered methods and processes required to understand how to create virtual replicas of exhibits, dioramas, and outdoor spaces to achieve actual learning and engagement outcomes. The non-expert may experience the virtual environment much like they experience the real environment. But unlike the real environment, these virtual applications are interactive, thus making learning experiential, embodied, and engaging at a click of a button. Any virtual 3D object of interest may be selected for learning at the moment of personal interest and curiosity.

As a three dimensional, real-time interactive data visualization of the real-world based on ecological research data, this artifact becomes an accurate and highly detailed virtual model of the plants, insects, and birds true to that location and time, and when those virtual objects are combined correctly on a virtual terrain, a highly immersive, realistic, and accurate virtual landscape of the environment is produced. It is like walking through the glass of and into an exhibit or a diorama found in natural history museums. By launching these immersive applications on global digital distribution channels (e.g., Apple iTunes, Google Android Play Stores, and itch.io), they become widely available, achieving broad impact and equity by default of the design, allowing anyone, anywhere, and at anytime the ability to produce a virtual exhibit or a virtual diorama in any school, living room, or backyard, responsive to individual curiosity in the moment, and linked to a web of knowledge and educational story to support inquiry and deep learning.

As a simulation of a real field trip executed at true scale, with photorealistic detail and 3D complexity, it is a useful knowledge artifact for scientific communication about the natural world. The Virtual UCF Arboretum Application allows users to take a leisurely virtual hike for as long as they wish and in any direction desired, even off-trail in this wild Central Florida environment. Much like a real hike, they experience nature as a realistic visual and audio landscape based on biological research field data. Not your typical fantasy game-level environmental art and not immersive 2D-360 video VR that keeps you on the trail, but a new type of immersive knowledge artifact and photorealistic visualization of the plants and animals found in the real environment.

Created to work as an educational simulation of a real field trip, it is unique in that it visualizes geographical information system (GIS) data of native plants and flowers over a large open world of 100 hectares (247 acres) of a Central Florida terrain in the Unreal Game Engine. It is a combination of three main system components, a virtual nature model constructed at scale and displayed as a data visualization, every object is annotated with interactive facts and concepts using a virtual plant field guide that also contains a virtual plant atlas representing the real world collection of native plants.

The application was completed in October 2018, expanded to work with VR HTC Vive headsets in November 2018, and connected to a treadmill and projected wall display in 2019. Ongoing work is transforming it to AR apps for learning, anywhere, anytime, and by anyone, a virtual arboretum, botanical garden, or natural history museum exhibit in every pocket.

How to use: The Unreal Engine model can be used on any VR ready PC, with graphic cards comparable to the NVIDIA GeForce GTX 1070, or more powerful. Once you download and install it, users can select the Escape key to open the Help Menu. Use the arrow keys to move around the open world, and at any plant where the cross hairs turn bright green you can click with the mouse to open the plant atlas and read facts and concepts about that plant.

The Virtual UCF Arboretum Application (Unreal Engine Desktop Virtual Field Trip)

Figure 1. The Virtual UCF Arboretum
Figure 1. The Virtual UCF Arboretum Application view of the Virtual Mesic Flatwoods, Central Florida

The Virtual UCF Arboretum Application was released as version 1.0 in October 2018, and is available as an Unreal Game Engine application open to exploration and learning at a click of a button. The environment game level for the application was constructed as a large open world, using GIS data of plant inventories, plant population densities, field gathered information, and drone images. The data sets required for sparse information fusion are combined with domain expertise to achieve accuracy and realism. All photorealistic 3D plant models are botanically correct, and each one is dispersed using actual plant population data to simulate realistic dispersion and densities. The ambient bioacoustics are accurate representations of insects, amphibians, and birds found in the real environment. The virtual environment is unique because it integrates a virtual plant field guide in-game as a HUD connected to an external website as a digital media plant atlas to include interactive access to facts and concepts. Such educational information allows individual users to pursue deeper inquiry at the moment of personal encounter and curiosity as they are exploring the virtual nature environment.

Software: The Virtual UCF Arboretum Application (Version 1.0) 

The Unreal Game Engine application may be downloaded and used to explore the virtual, multimodal, and interactive environment on any Windows PC with a graphics card.

Figure 2. The Virtual UCF Arboretum
Figure 2. The Virtual UCF Arboretum Application view of the Scrub with an example of the user interaction between the plant object, the Sabal Palm, and the Virtual Field Guide, showing part of the Educational Story

Users may select a plant to open the virtual field guide for learning in-game. Image (Figure 2) shows the integration of the application as a virtual nature game level in Unreal, and the interactive virtual plant field guide (e.g. HUD displaying an external and independent website). The technical design component is similar to a heads-up-display (HUD) user interface (UI) common in many games or other 3D UI VR applications. Many AR applications use the UI annotation as an overlay on top of real-world objects, as a type of virtual tool-tip. Such design factors represent interaction functionality that are context-sensitive and semantic. Such semantic interactions enhance the learning experienced by increasing inquiry and engagement through active learning of the objects encountered in the environment.

The Virtual UCF Arboretum Plant Field Guide

Figure 3. Shows the main page of The Virtual UCF Arboretum as an interactive plant field guide website
Figure 3. Shows the main page of The Virtual UCF Arboretum as an interactive plant field guide website

This standalone website was released as version 1.0 in October 2018. It was developed to support the production efforts of gathering all source data, facts, and story for the application between 2016 and 2018, and to provide the educational content in the virtual plant field guide in game. It is also an easily accessible educational website to support real-world field trips on any web-enabled device at the real UCF Arboretum, school, after-school programs, or in homes nationwide.

Published on UCF website: https://arboretum.ucf.edu/virtual

The Virtual UCF Arboretum Virtual Plant Field Guide published on PBS LearningMedia, 2018

PBS reviewed all content to identify and align with the national STEM educational standards. It is a trusted, digital distribution channel used by teachers nationwide for K-12 educational material and works in any web browser on any device. As such it has a significant and broad impact on education anytime, in schools, in afterschool programs, and at home.

Published on PBS LearningMedia: https://www.pbslearningmedia.org/resource/virtual-ucf-arboretum/virtual-ucf-arboretum/?fbclid=IwAR3Sur3GVnALQadQY1SLAOCrIIvxdMGBceGPcEOWG5nKjz8BzBI_Txa1oW4

Figure 4. Shows the main concept pages with video story embeded
Figure 4. Shows the main concept pages with video story embedded
Figure 5. Shows the plant page with facts, educational story and embedded 3Dplant models
Figure 5. Shows the plant page with facts, educational story, and embedded 3D plant models

The Virtual UCF Arboretum Plant Atlas

Figure 6. The Virtual UCF Arboretum Plant Atlas
Figure 6. The Virtual UCF Arboretum Plant Atlas

This standalone collection of 3D plant models was released as version 1.0 in October 2018. All 36 plants native to Central Florida are accessible and usable in Google Cardboard or any AR enabled phone, and have been integrated into the project’s virtual plant field guide website and the Virtual UCF Arboretum Application. A collaborative, interdisciplinary process supported digital media students in the gathering of research materials in the field, for review and feedback to result in highly detailed 3D plant models (Figure 5).

Published on SketchFab.com: https://sketchfab.com/VirtualUCFArboretum

Figure 7. Top image of UCF undergraduates gathering data and photos in the field. Top right image of field research measurements and photos. Bottom image, one of the resulting 3D plant models published in the Plant Atlas, and used in the Plant Field Guide. Optimized for AR versions have recently been completed.
Figure 7. One of the resulting 3D plant models published in the Plant Atlas, and used in the Plant Field Guide. Optimized for AR versions have recently been completed and will be published on the Epic Games Marketplace under The Harrington Lab at UCF

The Virtual UCF Arboretum Project in VR (HTC-Vive headset)

Figure 8. The Virtual UCF Arboretum in HTC-Vive VR headsets demonstration at the Orlando Science Center Otronicon Events.
Figure 8. The Virtual UCF Arboretum in HTC-Vive VR headsets demonstration at the Orlando Science Center Otronicon Events.

The Virtual UCF Arboretum Application was modified to work with VR HTC-Vive headsets and hand controllers available as version 1.0 in November 2018. It was presented to the public at the Orlando Science Center Otronicon Events in 2018, 2019, and 2020.  Undergraduate students assisted in the demonstration and public talks.

Figure 9. The Virtual UCF Arboretum is shown integrated with a treadmill and hand controllers. An undergraduate was grant funded in this interdisciplinary research.
Figure 9. The Virtual UCF Arboretum is shown integrated with a treadmill and hand controllers. An undergraduate was grant funded in this interdisciplinary research.

UCF Arboretum as Data Visualization

Figure 10. Left image is of the UCF Arboretum Natural Communities ESRI GIS map representing all natural communities by a color coded location. Each natural community represents a list of plant as a species inventory and plant population density data per area. The right image is of the Unreal terrain level representative of a 1,000 x 1,000 meter sample area of the real arboretum terrain, trails, and waterbodies found in the Natural Communities map area marked with the red square.
Figure 10. Left image is of the UCF Arboretum Natural Communities ESRI GIS map representing all natural communities by a color coded location. Each natural community represents a list of plant as a species inventory and plant population density data per area. The right image is of the Unreal terrain level representative of a 1,000 x 1,000 meter sample area of the real arboretum terrain, trails, and waterbodies found in the Natural Communities map area marked with the red square.

Natural Communities

Figure 11. Basin Swamp
Figure 12. Baygall
Figure 13. Depression Marsh
Figure 14. Dome Swamp
Figure 15. Scrub
Figure 16. Scrubby Flatwoods
Figure 17. Waterbody
Figure 18. Wet Flatwoods
Figure 19. Xeric Hammock