Virtual Humanities: Immersive Environments and 3D Simulation

Immersive environments are one of the newest areas of DH experimentation. 3D simulation, Virtual Reality and Augmented Reality were until recently only accessible with high end hardware and produced end products that were so large they were hard to deliver over the internet. As the technology improves and internet bandwidth grows, these projects are now coming into their own. Today we will first explore some of the newer ways to generate 3D content, explore some projects that show the possibilities of these developments, and finally try our hand at making simple interactive 3D experiences.

Beyond GIS

David Bodenhamer suggests that 3D technologies can move us beyond GIS as a representation of space to a deeper understanding of place. He writes

For humanists, the goal is not proof but meaning.   The challenge, then, is to use geospatial technologies to probe, explore, challenge, and complicate—in sum, to allow us to see, experience, and understand human behavior in all its complexity and to view its deep contingency.  As in traditional humanities scholarship, the aim is less to produce an authoritative or ultimate answer than to prompt new questions, develop new perspectives, and advance new arguments or interpretations.

David Bodenhamer, “Beyond GIS: Geospatial Technologies and the Future of History,” 2013

In groups: explore one of the following immersive 3D projects:

• Chinatown: Time Travel from the NY Times
• BAGAN: Google Arts & Culture
• Centring Spenser: Kilcolman Castle

Designate a reporter and be prepared to share the project and briefly answer all three questions. Reference the readings as you discuss.

• What new questions does it prompt?
• What new perspectives does it develop?
• What new arguments or interpretations does it advance?

---

How to Make a Model

Manual 3D modeling techniques like those we’ve attempted in SketchUp are very effective and have had a long history of producing impressive digital humanities projects.  Lisa Snyder’s long-running project to recreate the World’s Columbian Exposition of 1863 in Chicago is a prime example of what these techniques can accomplish in skilled hands.

Increasingly, however, computers are doing more of the heavy lifting. There are several methods of generating 3D models that rely on algorithms to create geometric meshes that are being adopted for DH projects along with techniques borrowed from the film and video game industries like motion capture. The award winning Virtual Angkor being a prime example of the leading edge of these techniques.

Two major current methods for generating 3D content are procedural modeling and photogrammetry, which we’ll discuss today.

---

Procedural Modeling

This term refers to the generation of complex geometry from basic shapes through the application of code-based rules.  The leading platform for this type of work in DH is CityEngine, owned by ESRI, the makers of ArcGIS.  This technique allows a user to produce, modify and update large, textured models of entire cities quickly and iteratively.  The output can be explored online or integrated with gaming software or 3D animation packages to produce video games, simulations and movies.

This software was developed for modern city planners and urban architects, but has increasingly been put to use on historic landscapes and built environments, as in the impressive work of Marie Saldaña who developed a Roman temple rule set.

For the past several years, a project here at Carleton led by Serena Zabin and Austin Mason has been using procedural modeling to recreate colonial Boston circa 1770 as part of a video game centered on the Boston Massacre called Witness to the Revolution. You can see a flyover of part of the reconstruction in the video below and read more about the origins and process from Carleton students’ posts on the development blog.

Procedural modeling is very powerful, but we’ve taken the game in a different direction with professional 3D artist support to hand model the city in a sytlized art style, as can be seen on the project website.

https://youtu.be/cQdyd5qe-Sc

---

Photogrammetry

Photogrammetry is another algorithmic modeling technique that consists of taking multiple overlapping photographs and deriving measurements from them to create 3D models of objects or scenes.  The basic principle is quite similar to the way many cameras these days allow you to create a panorama by stitching together overlapping photographs into a 2D mosaic.  Photogrammetry takes the concept one step further by using the position of the camera as it moves through 3D space to estimate X, Y and Z coordinates for each pixel of the original image; for that it is also known as structure from motion or SfM.

Photogrammetry can be used to make highly accurate and realistically photo textured models of buildings, archaeological sites, landscapes (if the images are taken from the air) and objects.  Close range photogrammetry of historical objects offers the possibility of both digitally preserving artifacts before they may be lost or damaged, and of allowing a whole suite of digital measurements, manipulations and other analyses to be performed that allow insights into the material that might not be visible to the naked eye.  The technique is gaining in popularity and usage, since it produces very impressive results comparable to high end laser scanning technologies for a mere fraction of the cost.

Modeling the Northfield Depot

In past years, we have used the historic Northfield Depot, a train station built in 1888 and currently undergoing restoration through the efforts of the local Save the Northfield Depot campaign, as a test case to pratice photogrammetry.

Using MetaShape (formerly PhotoScan), the leading photogrammetry software for archaeological reconstruction, we have produced solid, shareable models like the one below with nothing but photographs.

Northfield Depot by meDHieval on Sketchfab

A demo mode of MetaSahpe is available for free that will let you try everything except exporting and saving your model. If you want to explore more, they offer a 30-day free trial of the full Standard or full Professional editions, but the software is very expensive after that.

To make a model like the one embedded above you could follow the instructions below:

• Download the sample photograph dataset and unzip it on your computer
• Follow the instructions in the Photogrammetry with PhotoScan Tutorial document.
• For more, Agisoft has excellent tutorials on their website that stay up to date with advances in the software. Here is one for how to build a complete (360-degree) model.

---

MUSEUM PROJECT: Photogrammetry with Autodesk ReCap Pro

This year, we are going to create 3D models of objects in the Perlman Teaching Museum’s collection to begin to build out a digital 3D collection for the museum. You had an introduction to the collection last week, and this week you are going to sign up for a time to go back to the museum and make a model of an object with your group. We are going to stick with the education licenses for the Autodesk suite for this, to see how the cloud photogrammetry process in ReCap Pro compares to hand modeling in Fusion 360.

We’ll go through the basics of taking photos, processing, and cleaning a model in class.

Download Software

Before we get too far, make sure you are logged into Windows on your lab computer.

Go to the Autodesk Education site and Download ReCap Pro software

You’ll also need to download and install the Autodesk Desktop Connector software for Windows

Taking Photos

The most important step in photogrammetry is collecting data: i.e. taking good photographs that will have sufficient information for the algorithms to stitch images together and extract 3D geometry from them.

With a mobile phone camera, you’ll want to make sure you follow these guidelines:

• Have your scene well and consistently lit — DO NOT CHANGE LIGHTING between shots.
• Turn off the flash — because, again, DO NOT CHANGE LIGHTING between shots
• Use the standard photo settings rather than a “portrait” mode or similar. You want deep focus
• Have the object fill as much of the frame as you can
• Start at one point and circle the object, overlapping photos by ~50%
• Repeat at multiple heights to get full coverage
• Take several close up shots to get shadowed areas and detailed textures — but DO NOT CHANGE camera settings, zoom, etc.
• If possible, get shots of the top and bottom of the object for full 360 degree coverage

Processing photos

ReCap Photo makes processing easy by handling most of it for you in the cloud. The main steps are:

1. Load photos
2. Upload to cloud
3. Wait for processing
4. Download model
5. Inspect and clean
6. Export to desired format(s)

Get the images from the shared google drive folder. Follow the tutorial here to practice processing a model