Image-Based Synthetic Aperture Rendering MIT9904-14 PIs: Prof. Leonard
Image-Based Synthetic Aperture Rendering
MIT9904-14 PIs: Prof. Leonard McMillan (MIT LCS), Prof. Julie Dorsey (MIT LCS), and Dr. Hiroshi Murase (NTT)
The primary focus of our
research effort is to develop
technology to create virtual
experiences that will approach
the fidelity of the real world. In
the future, such technologies
will have a dramatic impact on
the way we work and play.
They will enable new forms of
commerce, bring together
individuals separated by large
distances, and provide us with
new forms of entertainment.
Ultimately we intend to
create a device for capturing
and processing dynamically
reparameterized light fields
in real-time. We call this
device a synthetic aperture
camera array. It is composed
of a two-dimensional array of
randomly accessible image
sensors that memory-mapped
in the address space of a host
processor. Such a system will
allow images to be synthesized
from a wide range of virtual camera positions in real-time. We plan to support
multiple simultaneous video streams to support stereoscopic display as well as
Light Field Parameterization
Low Cost Acquisition
Pod - A
We take a non-traditional approach to
computer graphics modeling and rendering, in
which a scene is represented by a collection of
images rather than the geometry and surface
properties used in typical computer graphics.
Essentially, we treat a collection of images as a
database of rays. New views can be constructed
from this database on a ray-by-ray basis by
selecting the closest ray to each desired ray.
A high-level block diagram of our proposed system is shown above. The cameras
host interface will be an industry standard personal computer bus. The camera array
will be constructed from modular sensor units mounted on a common motherboard.
Pod - A
Pod - B
Pod - B
Pod - B
Pod - B
Our dynamically reparameterized light field representation allows us to synthesize
images with photographic effects such as variable focus and depth-of-field. Depthof-field effects are created by varying the extent of the reconstruction filters used
on the camera surface.
A variable focal length can be simulated by varying the focal plane used in the
reconstruction process. In a synthetic aperture camera both the aperture and focallength settings can be varied from pixel to pixel. The allows effects that are
impossible with a traditional camera.
We have also prototyped two lowcost devices for acquiring light fields.
We have developed two acquisition
systems for acquiring light fields of
static scenes. The first uses a robotic
XY-platform to move a digital camera.
This system allows us to explore the
trade-offs between camera spacing and
resolution in order to estimate the performance of our camera array. This
system uses a precision image sensor,
precision optics, and a motion platform with a travel distance of approximately one
meter squares. It can acquire a 16 by 16 image light field in under 20 minutes, and
it cost approximately $10,000 US to construct. Our second system is based on an
off-the-shelf flat bed scanner, and an array of plastic lenses. We have modified the
scanner to operate off of battery power so that this system can be taken out into the
field to acquire images. Additional processing is required to correct for shortcoming
in the image sensor and low cost optics. None the less, the system can acquire an 8
by 12 image light field in under 3 minutes, and cost under $100.
The addressing of sensor modules will be interleaved in order to maximize
the communication bandwidth between the image sensors and the host
computer. Each sensor
pod contains a CMOS
image sensor, buffer Address
memory, and glue
logic. The multiframe buffer
memory is used for
(on board A/D)
two functions. It
is used to store
and it allows the
host to access
image rays asynchData
ronous to the image
will allow us to upgrade to higher resolution sensors as they become available.
We have also developed techniques for direct autostereoscopic viewing of our light fields. These methods
are similar to various lenticular techniques for viewing
stereo images. Our synthetic aperture generation approach provides much greater flexibility than tradition optical approaches. In particular
it can overcome many limitations such a focus control and skewed frustums. We have
demonstrated viewers with true parallax (both horizontal and vertical), and variable
controlled focus. Our displays have nearly all of the desired properties of holograms,
yet they are true color and viewable
under normal lights. Furthermore,
the technology is easily adaptable
to the display of dynamic 3-D
images. Currently we are only
limited by the resolution of flat
The image on the left,
when viewed through a
hexagonal lens array,
can be seen as a threedimensional image of a
flower. It can be simultaneously seen by
multiple viewers. It was
computed from a dynamically reparameterized light
field, which allows us to
precisely control the
focus at all viewing
angles. The inset
provides a magnified
view of the image.
Elements of Art LINE What defines a LINE? A line connects two points and can go on forever. It is made by a pointed tool such as a pen, crayon, marker, paint brush, etc. It can create shapes, texture, and...
Mill Rythe. This modern centre offers high standards of accommodation with extensive grounds and superb facilities. The centre offers spectacular views across ChichesterHarbour, an area of outstanding natural beauty. Schools allowed to introduce own approach to formative assessment
Many-body perturbation theory energies and gradients. Effective core potential energies, gradients, and second derivatives. Coupled cluster [CCSD and CCSD(T)] and configuration interaction energies. Segmented and generally contracted basis sets including the correlation-consistent basis sets.
Ready to download the document? Go ahead and hit continue!