Researchers from the Massachusetts Institute of Technology (MIT) released a study last summer describing an algorithm capable of recovering intelligible speech by analyzing vibrations in objects, which are then captured on video through soundproof glass. This study is now being adapted by the same researchers to show that the technique can be used to determine material properties of objects from video.
At the Conference on Computer Vision and Pattern Recognition in June, the researchers will be discussing nondestructive testing, a means of determining physical properties in a material without taking samples or subjecting the material to physical tests.
According to Abe Davis, one of the first authors for the study and an MIT graduate student in computer science and electrical engineering, one of the major contributions of the study is to connect computer vision techniques to established theories on physical vibrations.
"We make this connection pretty explicitly in the paper ... because it bridges these fields," he added.
One of the applications of nondestructive testing is that it will make it possible to spot structural defects in an airplane wing, for instance, by analyzing its vibrations during a flight caught on video.
Davis and colleagues tested the technique on two different objects: one made of rods of metal, wood and fiberglass and the other fabrics that have been draped over a line. Vibrations were produced with help from a loudspeaker. Vibrational frequencies in the rods were high so a high-speed camera was needed to capture a video of them. The fabrics, on the other hand, resulted in low vibration rates so an ordinary digital camera was sufficient for capturing a video of the material's vibrations.
Vibrations are normally undetectable to the human eye. Objects vibrate at certain frequencies, producing corresponding vibrational patterns. These unique patterns are what the algorithm in the visual microphone extract.
After gathering the videos, a machine was then used to find correlations between patterns and measurements of vibrations in the two materials. Results showed a good measure of elasticity in the bars and weight per unit and stiffness in the fabrics. The presence of deviations in typical vibrational patterns in an object will be indicative of structural problems. Researchers, at the moment, are working to identify possible deviations.
Other authors include Katie Bourman, Fredo Durand, Bill Freeman, Justin Chen and Michael Rubinstein. Bourman is Davis' fellow graduate student while Durand and Freeman are their advisors, respectively. Chen is an environmental and civil engineering graduate student, while Rubinstein completed his Ph.D. with Freeman but is now affiliated with Google Research.
