Elphel Development Blog

Panoramas From the Backpack

Tacking high-resolution panoramic images in the remote places, that can only be accessed by feet is an option now available with the Elphel-Eyesis 360 degree panorama camera. The camera’s size (1.3 x 0.3 meters) and relatively light weight (10kgs) allow to mount it on a backpack frame and carry by a person.

There were multiple requests for the backpack option by our customers since the development of Elphel-Eyesis camera, but other projects were of higher priority, until this spring, when we finally decided it was time to take Eyesis hiking. After all, we have worked hard on this project for many months, so we ought to have some fun with it too, and take panorama images of the places we knew and enjoyed for it’s scenery.

On April 19th, 2011, we took the Elphel mobile office to camp for 5 days in Southern Utah near the Goblin Valley State Park to try out Eyesis in beautiful places not yet available in continuous panoramic imagery, mainly because of their inaccessibility for car, ATV, or even a tricycle.
Hiking with Eyesis in the backpack (30 lbs/14 kg total weight with the battery pack) allows us to capture a continuous stream of geotagged (GPS) 360° panoramic images. With the current battery pack (just a regular UPS with lead battery) we can take up to one hour and forty minutes of footage at a rate of 5 frames per second.

Route #1 - Along the Creek	Our first test hike was around our camp, along the creek, and into a tiny, 20 meters long, canyon. The green dots mark the route and points 1 and 2 show the location of 2 panoramas presented.
	hands-free operation even allows rock climbing with Eyesis
After the initial set-up was completed on a small and lightweight EeePC we started the recording and disconnected the computer again, as it is better to have hands free while hiking, or even rock climbing. All images and corresponding metadata are stored on Eyesis’ internal SSDs automatically. Photograph on the right had been tilted and clipped to make it look like we are actually rock climbing with Eyesis. It is a joke, of course.

It’s quite difficult to keep an upright position all the time (especially in a steep terrain) to ensure properly leveled panoramas. Climbing up or down-hill will result in a image like this:

Image 2: Elphel Campsite (Panorama Viewer requires a browser supporting WebGL like Firefox4 or Chrome)

Current development of the IMU (Inertial Measurement Unit) logger, among other features described in Eyesis 4Pi article, will also help to overcome the tilt problems. Our original plan was to have the IMU support ready for this trip, – may be you can see Andrey inside the mobile office working on it, but the time was limited and we end up not using IMU on this trip. All panoramas you see here are oriented manually.

Hiking Among the Goblins (Or is it Mars?)

The next day we ventured away from the camp, taking Eyesis to the Goblin Valley State Park. The GPS was working fine, again and recorded the trail where we made panoramas. Once all the panoramas will be processed we will have them available as a virtual tour via the Elphel Panorama Viewer.

Route # 2 - Goblin Valley

Points 1, 2 and 3 show the locations of panoramas we present here.

Goblins and ...

a Martian

Image 1: Bright sunny day in Goblin Valley (Panorama Viewer requires a browser supporting WebGL like Firefox4 or Chrome)

Image 2: Dramatic Shadows (Panorama Viewer requires a browser supporting WebGL like Firefox4 or Chrome)

Image 3: Stone formations gave valley it's name (Panorama Viewer requires a browser supporting WebGL like Firefox4 or Chrome)

The Little Wild Horse Canyon

The Little Wild Horse Canyon is very narrow. At times, the canyon is the width of a person. Since the Eyesis camera can focus on objects no closer than four meters, we show the panoramas that were made outside the narrow slot canyon.

Route #3 - Little Wild Horse Canyon

The satellites were out of sight most of the time, so the GPS data was not as accurate in this narrow winding canyon. Again, the IMU (and other distance measurement sensors, that we plan to add in Elphel 4Pi camera) will mitigate these problems. However, the panoramas turned out spectacular.

What is that? - A rocket launcher!

It is a very popular hike, so Eyesis got a lot of attention from visitors – almost as much as the canyon itself!

Image 1: Approaching the narrow slot (Panorama Viewer requires a browser supporting WebGL like Firefox4 or Chrome)

Image 2: The Canyon kept winding (Panorama Viewer requires a browser supporting WebGL like Firefox4 or Chrome)

Update (by Andrey)

There is a nice discussion of these panoramas on a PanoToolsNG forum:

“That’s of course just my snobbish sight on it because I can’t see a reason why one would want to take the pain to carry such a thing just to bring horrible pictures from beautiful places to the rest of the world.”

What I saw was not too impressive; lack of focus, poor stitching, uneven horizon, huge nadir hole, all for over $24,000! or did I miss something?

Well, these panoramas are not perfect, of course. But I got an impression that it is an apples-to-oranges comparison (you may find more an apples-to-apples one in our previous blog post). The panoramas we tried here were made with a rig designed to be mounted on a car roof, the backpack version was a quick test (we actually planned this trip to try camera on ATV, but we had to change our plans for the reasons out of our control), so nothing was done to make a camera vertical – that would need either angled mount of the optical head or (as a quick fix) a counter-balance attached to the camera (something like a carrot on an stick extending in front of an operator). That tilt caused the black nadir hole being asymmetrical, you may want to look at our project under development – Eyesis-4pi – it is designed to have the same resolution in the zenith and a zero nadir void — but there still will be some vehicle or person visible, unless the camera will be in a free fall (i.e. in space) as we were already suggested 🙂 .

The camera we used simultaneously acquires all 9 images in a way that there are no voids even when the camera is moving fast, so we acquired thousands of panoramas during each of the trips, the acquisition rate was set to about 2 frames per second. Yes, you definitely can make artistic panoramas using a single inexpensive camera using adapter that keeps the entrance pupil of the lens in the same point, but that would not work for the fast moving camera – so our only tool to keep the parallax low was to design the camera to have components as close as possible – but the parallax would never be zero

Poor focus? Well, this may be true for the very close objects, the camera is designed for the relatively high distance to the objects. The focus is fixed, of course, and the lenses are rather high power to avoid motion blur when the camera is moving at highway speed. So with f=4.5mm and F=2.0 the near limit of the DoF is approximately 6 meters, in our experiments we had much closer shots too. The lenses are also limiting the resolution (especially near the edges of the images), and we went a long way to improve the quality by postprocessing. You may analyze the raw/processed images pairs used in one of these panoramas by our image comparison tool (works in FF better than in Chrome).

Currently this postprocessing only corrects aberrations (not the distortions), so the panoramas are stitched manually. We never invested much time into perfecting the panorama stitching, because we have to do it automatically in the end, and we plan to make a camera to be a measurement tool, able to precisely locate the position of each pixel. The software (it is all under GNU GPLv3) can currently measure the position of the calibration pattern (same one as we use for aberrations correction) is already capable of detecting the pattern nodes locations to the +/-0.05 pixels precision, but the overall pixel mapping processing is not finished yet. We’ll describe this calibration procedure more on our blog when finished.

Levelling the panoramas – yes, it is lousy too. We already have built the hardware for the IMU (inertial measurement unit)integration, that allows logging the high resolution sensor data (we use ADIS-16375) at the full senor rate (300Hz), we also made the hardware for 5Hz (or higher) GPS integration, including the precise timing pulses. Unfortunately, I was not able to finish the FPGA code for the data logger during that our trip, so the panoramas lack that important data and we’ll have to try again when ready (it will have odometer/pedometer input also).

We are a Free Software/Open Hardware company, and while we can not give away our cameras (and yes, they are much more expensive than a simple panoramic adapter you can use to make brilliant panoramas), our software is all under GNU GPLv3 license, we “release early” (all the software we use internally is maintained at Sourceforge), our camera documentation is also available under Free license. And we are inviting other developers, who have ideas and experience in the panoramic imaging to join this project.