MARS PATHFINDER ROBOTICS VISUALIZATION APPLIED TO SUB-MARINE ARCHAEOLOGY I. Problem and Motivation II. Wave Overview III. Technical Details IV. 3D Visualization V. Stereo Vision VI. Gathering Data and Telemetry VII. Measurement Tools and Vehicle Control VIII. Test Results I. PROBLEM AND MOTIVATION Human undersea exploration can often be hazardous and costly. With modern technology, robots can perform many tasks in extreme environments where manned submersibles would venture to their peril. Mechanisms such as ROV's are cheaper to operate than their human counterparts and are expendable in emergencies. However, humans find it difficult to assimilate spatial-temporal information directly from remote sensors and cameras. The challenge in designing ROV interfaces is to provide operators an intuitive feel for the environment being explored through visualization and analysis of real-time and archived data. II. WAVE OVERVIEW The Intelligent Mechanisms Group (IMG) at NASA Ames developed a number of robotic visualization and control technologies for the 1995 Mars Pathfinder Mission. The WAVE system (Weatherproof Aquatic Virtual Environment) applies these technologies to marine robotics. Aspects of the system were tested August 10-28th in the Arctic Ocean as part of The Jeremy Project, a combined research effort by the US Coast Guard, National Oceanic and Atmospheric Administration, Santa Clara University, and NASA . The WAVE system uses modular software architecture to achieve an easy to extend visualization and control environment for robotic exploration. Using WAVE, the scientist interacts with a virtual reality environment that they use to visualize the ROV and the ROV's environment, to control manipulators and thrusters and to analyze scientific data such as the size of an artifact. Reviewing archived data with WAVE is as simple as playing the recorded telemetry back through the interface. The hope is that using WAVE, ROV-based marine exploration will be faster, cheaper, and better. III. TECHNICAL DETAILS The WAVE system is a compilation of technologies developed by the Intelligent Mechanisms Group at NASA Ames. The system can be divided into four distinct modules: 3D visualization; automated 3D modeling; data and telemetry gathering, and measurement tools/vehicle control . Due the high modularity of the components, the task of replacing components with new technologies, as they become available, is relatively simple. IV. 3D VISUALIZATION WAVE's 3D rendering and human interaction is handled by a component called VIZ . The VIZ module is an Open Inventor-based 3D rendering server with a threaded sockets interface for message passing. The VIZ sockets interface accepts a simple command list . Examples of the commands are "add object", "change the position of object", and "change color of object". The objects are real-time rendered in an interactive 3D virtual world. Users may control the perspective, manipulate 3D software dragging tools, or view the environment via optional Stereo Graphics Crystal Eyes 3D vision glasses support (see figure 4.1). User input to the VIZ module occurs via dragging tools in the virtual world. Dragging tools in the virtual world provide the ability for user input to the system such as moving manipulator arms by dragging joints or placing 3D measuring tools in order to determine the length of an object. V. AUTOMATED 3D MODELING To provide the user a better understanding of the ROV's environment, it is beneficial to collect a 3D model of the terrain and other objects. The Ames Stereo Pipeline is used to build photo-realistic 3D models of the ROV's surrounding. The Ames Stereo Pipeline, also developed by the IMG, measures the shift of an object between the left and right image (the parallax), similar to the workings the human eyes, to determine the distance to the object. Applying this technique across all pixels in an image pair, the Pipeline reconstructs the precise topography of the scene. The original image is applied as texture to give the model a photo- realistic appearance (see figure 5.1). The advantages of a 3D environmental model are the ability to view included objects from different perspectives, and to perform measurements of length, width, or volume on all or parts of objects present. VI. GATHERING DATA AND TELEMETRY The WAVE system's modular design allows science data and telemetry information in the 3D VIZ interface to be easily updated. Information can be read, recorded, and updated asynchronously via a process running in a separate thread, or even on a separate computer. When new data becomes available, it is logged and a message sent from the process reading the data (via a sockets interface) to the VIZ module in order to update the appropriate information in the VIZ module's virtual world. As an example, the module that updates vehicle orientation is handled by a separate process which polls the compass at a specified interval, logs the orientation of the ROV and finally sends a command using the sockets interfaced to the rendering server. The command contains the name of the object to update and the new orientation. VII. MEASUREMENT TOOLS AND VEHICLE CONTROL ROV control can be accomplished using a control console or the 3D tools in the VIZ-based 3D world. The dragging tools in the 3D user interface are part of the virtual reality model of the vehicle and allow the ROV operator to direct the ROV by pulling a dragging tool in the virtual world. This information is relayed to the ROV's thrusters through a message passed via sockets from the VIZ module to the thruster control module. When the user changes the position of a dragging tool, the VIZ interface performs a "call back" to any threads which have notified it of an interest in the position of that dragging tool. The VIZ module sends a command via the socket interface to interested processes, informing the interested processes that the event occurred and of any information associated with the event. The interested process handles the event and updates information in other modules as necessary. To measure an object, such as an archaeological artifact, using the 3D VIZ-based interface, the user would click on the two ends of the artifact. The VIZ module would alert interested software modules, sending messages containing the location of the two points clicked. The alerted module would make any calculations necessary and report the dimensions of the artifact. VIII. TEST RESULTS The WAVE system was tested in the Arctic Ocean during August 1998 as part of The Jeremy Project using a Phantom XTL ROV lent to NASA by the ROV's maker Deep Ocean Engineering. The goal was to demonstrate the technology ready for ROV missions. Because of constraints, it was not possible to test the full extent of the system in the field. The ROV was not modified for computer thruster control, and we did not have the opportunity to test the integration of systems to determine the ROV position. During the summer field test, aspects of the WAVE system tested included the ability to build models of complex objects using the Ames Stereo Pipeline and the ability to measure those objects using the 3D interface. The Ames Stereo Pipeline succeeded at building 3D models of complex objects from a distance of approximately 3 feet, with "diver estimated" visibility of 10 feet. The measurement tools were able to calculate the size of objects in models built by the pipeline with error of 10%. Under conditions of greater water column visibility, WAVE system measurements are expected to have error margins less than 10%. The WAVE system was evaluated by the two Jeremy Project archaeologists, Jeremy Bates of Santa Clara University and Dr. Michele Hope of the Alaska Office of the Minerals Management Service, as "significantly better" than systems currently in use for field collections of data on marine artifact size and location. LOOKING TO THE FUTURE The plans for WAVE are to further develop and test the abilities of the system extending it into new applications. Desired additions to the system include alternate methods of automated 3D modeling, integration of vision based positioning systems, and intelligent navigation. Further tests include determining the underwater characteristics of the Ames Stereo Pipeline in controlled conditions and gathering data regarding the accuracy of the vision based measurement system underwater. ACKNOWLEDGEMENTS VIZ was written by Laurent Nguyen, Kurt Schwehr and Deanne Tucker. Eric Zbinden wrote the Ames Stereo Pipeline. Thanks to the Intelligent Mechanisms Group, to Jeremy Bates and Aaron Weast for help on The Jeremy Project, to the members of the US Coast Guard for the hard work and Thanks to Dr. Philip McGillivary and Hans Thomas for guidance. BIBLIOGRAPHY MARSMAP: Analyzing Pathfinder Data using Virtual Reality, American Geophysical Union, Fall Meeting, 12/ 97, San Francisco, CA, US A. C. Stoker et al. Mapping Mars Using Virtual Reality, ISPRS Conference on Extraterrestrial Mapping Workshop on "Mapping of Mars", 04/ 98, London, UK E. Zbinden et al. Analyzing Pathfinder Data using Virtual Reality and super- resolved imaging submitted to the Journal of Geophysical Research C. Stoker et al. Applying Virtual Reality Technology to Marine Archaeology, Society for Historical Archaeology Conference 01/1999, Salt Lake City, Utah. J. Bates et al. MARSMAP: An Interactive Virtual Reality Model of the Pathfinder Landing Site, Lunar and Planetary Society Conference 04/ 1998, Houston, TX. C. Stoker et al. Information regarding the Jeremy Project, named for itıs principal investigator Jeremy Bates, is available at http://screem.engr.scu.edu or http://quest.arc.nasa.gov/actic98 Information about VIZ and other Intelligent Mechanisms Group projects is available at http://img.arc.nasa.gov/~schwehr/viz and http://img.arc.nasa.gov respectively. The OpenInventor 3D toolkit is developed by Silicon Graphics http://www.sgi.com A short list of VIZ commands is included below. More information is available at http://img.arc.nasa.gov/~schwehr/viz. VIZ Commands include: add, changeAlpha, changeColor, changeFont, changeOrient, changeParent, changePos, changeScale, changeText, changeType, event, hide, remove and attachSensor. For information regarding the Ames Stereo Pipeline: http://img.arc.nasa.gov/~zbinden/photoRealVR/topvr. html