menu

Observational Virtual Reality link

Imagine that your good friend has recently purchased a Virtual Reality (VR) headset and invited you over to their place to experience it. "Come on over," they said; "it'll be fun" they said. You arrive, only to find your friend hogging the VR headset. Boring. You can't exactly see what they're doing. All you see is them wearing the goofy goggles waving their arms around like a buffoon.

What if you could experience what your friend was experiencing? What if you could see your friend in the virtual world, as before, but also see what they are seeing as if you were able to be in that world with them? Wouldn't that be a much more compelling experience? Here at the Imaging Research Center you can now do that.

We have assembled an Observational VR lab space. Observational VR, otherwise known as Mixed Reality, allows people to watch subjects in both the real and virtual worlds simultaneously. To accomplish such a feat, we have set up our VR computer in a green screen room and added a video camera. The video camera then records a subject using VR in front of the green screen.

In the virtual worlds that we create we add a third person virtual camera where it’s movement, position, and rotation are tied to tracked VR controller. That tracked VR controller is attached to our video camera. Then using a software program such as OBS Studio we combine the video feeds from the virtual camera and video camera into one where the green screen is keyed out and replaced with the virtual world. Now the subject can be seen in context, interacting directly with the VR environment, rather than looking like someone making funny movements for no reason.

HTC Vive controller affixed to a camera.

Of course, our primary purpose in creating this space is to further research into interactive virtual environments. Do you have ideas about using this new space? Questions about observational VR? Do you want to see it in action? Come join us at the IRC’s Observational VR event:

Tuesday May 2nd
4pm - 6pm
ITE 108

Mark Jarzynski

April 18, 2017

Scan Your Stuff link

Scan Your Stuff
ITE 109
April 7, 12:00pm - 4:00pm
April 8, 10:00am - 2:00pm

Maybe you have an object that you'd like to preserve for posterity. Or perhaps there's something you'd just like to have a digital version of so that you can carry it around with you, and interact with it, like this:

Beanie Dog by ircumbc on Sketchfab

If so, you're in luck. For just over a year the IRC has been developing a photogrammetry rig for research. It captures objects and turns them into virtual 3D models. This April the IRC would like to extend an invitation to the wider campus community to join us and capture your own items with our facility. We will be holding open hours from 12:00pm until 4:00pm on April 7, and from 10:00am until 2:00pm on April 8 in ITE 109. Students, Faculty, Staff, and university affiliates are all welcome to bring items to be scanned. Models will be built during the following week before being available for download. We are hoping to build a library of scans that challenge our capabilities and discover the limitations of our current processing. If you permit us to use the scan of your item in our library then we will make it available for other researchers to use, and it will become publicly available.

But just what is photogrammetry? Photogrammetry is a method of automatically reconstructing real objects into 3D models using flat images. We use 94 cameras to capture images of people and still objects, then use an algorithm to stitch the images together and reconstruct the geometry of the subject. This means we can capture true 3D models directly from reality without manually reconstructing the shape or textures.

Photogrammetry relies on the surface detail of the subject to allow points on the surface to be located in multiple images. Therefore it works best on objects that have matte or grained surfaces, like most natural materials. Skin, wood, most fabrics, and patterned surfaces have the best results. Shiny objects such as polished metal or shiny plastic are more challenging, and transparent items such as glass are not currently possible. We are happy to try anything, but must stress that this research is still evolving and not everything will give an accurate, detailed model. For this session we are targeting objects with a size roughly between that of a suitcase and a basketball.

The scans we generate are standard OBJ files along with a JPG texture. We will provide both a raw point cloud and a reconstructed mesh. Files will be available in the university box drive for the near future. Depending on the object scanned we routinely get between 500,000 and 2,000,000 polygons per capture. The models we generate may have holes where we could not reconstruct the surface or other artifacts. While we hope to eventually eliminate these issues algorithmically, for the moment they require manual touch-up. We are happy to provide the source files the model was generated from to help you make necessary changes, but cannot help with the actual manual processing.

You may be wondering what you would use your scan for. Photogrammetry is already being used in a wide variety of applications today. Artists can use photogrammetry to gather references without interpretation, giving a clean slate to build their vision on top of. Historians can preserve detailed models with real-world coordinates rapidly and without even touching the object they wish to preserve. Game developers and video producers can use photogrammetry to generate assets for their projects without having to spend hours reinventing objects and scenery they have already found out in the world. We are excited to open this world up for others to explore, and cannot wait to see what you will do with it.

Mark Murnane

March 28, 2017

Light City link

Here at the IRC, we like it when stuff lights up, and we like things that help Baltimore. So here’s a story about something that does both. Some years ago I attended a great cocktail party put on by my friends Brooke Hall and Justin Allen, founders of What Weekly (the hip but serious e-zine that for years kept Baltimoreans aware of what was happening—both figuratively and literally) and What Works Studio. They took a few of us aside and told us about a new idea they had. They wanted to hold a huge light festival here in Baltimore. They’d seen versions of this in other cities outside the US and had a take on it could be a great fit for Baltimore. They got some feedback, which, at least from me, was: great idea but we don’t need another Grand Prix race; we need something that seems at least remotely connected to who we are as a city. A number of us thought it should also strive to have some social value beyond bringing bodies to the Inner Harbor. As it turned out, they had already been thinking about the lights mostly as a way to draw people into programming where they could learn about and consider the city in a deeper way.

Brooke and Justin went to work, and maybe a year later I heard them on David Warnock’s weekly, Baltimore’s Future on WYPR. Warnock’s foundation had given them an award for the idea and the entrepreneurial chutzpah they were showing to make it real. The idea sounded appealing to a lot of people and they’d tweaked their design to include days of panels and speakers on important issues facing the city. Soon after this, they landed the first real money to make it happen. I blinked and it was 2016: the first annual Light City festival was going to happen in Baltimore. The Baltimore Office of Promotion and the Arts (BOPA) had gotten involved, and it looked like the logistical and planning support that would be needed would be forthcoming.

Visitors to Light City 2016 enjoying the festival.

Cover Image: Diamond by UMBC graduate, artist Mina Cheon and architect Gabe Kroiz

The rest is history. The 2016 Light City was a smashing success measured by the great outdoor exhibits, the “Light City U” (as in university) panels and speakers, and the fact that 400,000 PEOPLE(!) came to Baltimore’s waterfront to see the work of many of our best artists and hear insightful dialog on key issues. It went very, very well. Unfortunately, a tangle erupted between BOPA and the creative duo, Brooke and Justin. Whatever happened or however this gets resolved, two truths must emerge: 1) We like Light City and want to support it, and 2) if our city wants creative entrepreneurs to generate these kinds of gifts we have to do what’s necessary to make sure relationships don’t go south this way. We need to incent and reward creativity and initiative.

Some participants truly take the festival to heart.

All this said, Light City is going to happen again and that’s a great thing. An enormous amount of effort is going into it and I know the artists are excited. Light City U has become Labs@Light City and there is a stunning lineup of speakers and panelists including some of my favorites like Baltimore City Health Commissioner, Leana Wen, musician Wendell Patrick, writer D. Watkins, social entrepreneur Joe Jones, artist/performer/poet/activist Michelle Nelson, UMBC’s luminary President, Freeman Hrabowski...the list goes on. Go. Here them. See the art that lights up. Bring the whole family. Eat. Drink. You’ll be glad you did. See you there! March 31st – April 8th.

Find out all you need to know at: lightcity.org

All photos are by former IRC Director, photographer and media artist, Dan Bailey. See the rest of his album at: https://www.flickr.com/photos/ircumbc/albums/72157667197654631

Lee Boot

March 2, 2017

Digital Humanities and the IRC link

What are the so-called "Digital Humanities"? It's difficult to find a single short answer—in fact, one can't even find consensus on whether the phrase is singular or plural. Are the Digital Humanities a set of tools, practices, theories and approaches? Is it a new field of scholarly endeavor that should rank alongside older disciplines like History, English, or Philosophy? Does a practitioner have to have the skills of a programmer? And how should we define the Humanities to begin with?[1]

Interactive virtual sculpture: Madeleine I, Henri Matisse, 1901

Cover Image: Point cloud from an infrared scan of Madeleine II, Henri Matisse, 1903

At the IRC we are loathe to wade into the longstanding debates over definitions of Digital Humanities, especially as some of the early conversations seemed to center on defining who was “in” or “out” of the field. We view ourselves as practitioners, rather than theoreticians. As such, we believe in the “big tent” approach, seeing the Digital Humanities as a way of using digital media to both ask big questions and visualize knowledge. Digital tools—like textual analysis, mapping and GIS, and virtual reality—are used to both support and define scholarship.[2] Importantly, at the IRC elements of research and scholarship are present in both the “Digital” and the “Humanities” aspects of projects. We don’t want to just work on projects using predefined tools; we want to make new and better tools.

Left above. Detail from interactive kiosk: BEARINGS of Baltimore, Circa 1815, 2016

Right above. Still from video: The Unbuilt Hurva, 1993

Sometimes humanists are afraid of working with digital tools because they worry about not having a technical background. But they shouldn’t be, because digital work is inherently collaborative and multi-disciplinary. Working together allows each scholar to build on their own strengths. The IRC has a long history of collaboration with humanities scholars, through projects like The Unbuilt Hurva, Matisse: Painter as Sculptor, Mapping Baybrook, and Visualizing Early Baltimore, and the Art of Transformation.

Screenshot: Art of Transformation, IRC project with Baltimore community partners and humanities scholars, 2016

We want the IRC to function as a hub of innovation, a place where scholars from all disciplines—humanities, social sciences, life sciences, visual arts, computer science and engineering—come together. Indeed, the IRC is already such a place, as the staff includes scholars from Visual Arts, History, and Computer Science. At the IRC we use our skills to make knowledge matter. The IRC is a place for scholarly experimentation, in form and content. It’s a place to build and use new tools, like MapTu. We want to push the limits, and we look forward to many future collaborations.

Screenshot: Interactive mapping site, Mapping Baybrook, 2012
  1. ^ According to the 1965 legislation that founded the National Endowment for the Humanities: "The term 'humanities' includes, but is not limited to, the study and interpretation of the following: language, both modern and classical; linguistics; literature; history; jurisprudence; philosophy; archaeology; comparative religion; ethics; the history, criticism and theory of the arts; those aspects of social sciences which have humanistic content and employ humanistic methods; and the study and application of the humanities to the human environment with particular attention to reflecting our diverse heritage, traditions, and history and to the relevance of the humanities to the current conditions of national life." https://www.neh.gov/about
  2. ^ See Spencer W. Roberts, “Reading Series: Part 1, Enacting Digital Humanities”

Anne Sarah Rubin

February 16, 2017

It's Alive! link

Resurrecting The Rack

The IRC has been working to resurrect a legendary creature that we have come to call "The Rack." The Rack is a relic from a bygone age when audio and video was recorded onto smallish rectangular boxes called "tapes." As has been the case with many things of a technological nature that could have benefited from some standardization, tapes came in a wide variety of shapes and sizes. To record, capture, and transfer media to and from all of these different kinds of tapes, an absurd array of media decks, converters, monitors and patch bays were all linked together in an eye-watering symphony of cables and connectors and complication. Hence, The Rack. And while it could be quite the angry beast to work with in its day, with the passage of time The Rack's old-timey charm has grown, and we refer to it now with more than a little nostalgic fondness and affection.

Removing a few of the floor tiles next to The Rack revealed some pretty epic cable snarls that had to be untangled.

The Rack does have an undeniable beauty about it. It is a museum of some of the world's finest industrial grade electronics manufacturing and craftsmanship. A peek inside a top-of-the-line tape machine from the mid-1990s reveals an astounding array of intricate moving parts squeezed into the tiniest places. Although engineers may have hoped that these machines would stand the test of time, like ice that melts with each new season, an ever changing media technology landscape has rendered most of The Rack's hulking components obsolete. Of course ice is recycled seamlessly by our planet's ecosystem; an obsolete Professional S Video Cassette Recorder is chucked rather un-seamlessly into a big hole with about a million other machines of its kind, and that's that.

Rack guts.

How do we retire this hardware in a responsible way? It's an ongoing challenge. Experimenting with cutting edge technology and conducting media research requires lots of equipment. Much of this equipment is costly to produce and extremely well built, but not built to stay relevant. Our solution to this dilemma in the case of The Rack is to fuse the old with the new.

While we haven't recorded anything to tape in a very long time, much of what the IRC has created in our 30-year history is still archived on tapes. So, in order to view and digitize this media, and extend the life of all this remarkable machinery, we've brought The Rack back, replete with all its clicky clacky buttons, swirling knobs and blinking lights. There's still lots of testing and labeling left to do, but with the addition of a computer and modern breakout box, in the near future we will capture and preserve media recorded on all kinds of different formats such as: VHS, BetacamSP, HDCam, and even the legendary and elusive U-Matic. Our hope is that The Rack will be an exception to the rule of parting with such awesome equipment once it's past its peak, to the detriment of the planet and our capacity to examine and safeguard the past.

Ryan Zuber

January 30, 2017