Google Tech Talk
August 2, 2010

ABSTRACT

Presented by David Sachs.

Gyroscopes, accelerometers, and compasses are increasingly prevalent in mainstream consumer electronics. Applications of these sensors include user interface, augmented reality, gaming, image stabilization, and navigation. This talk will demonstrate how all three sensor types work separately and in conjunction on a modified Android handset running a modified sensor API, then explain how algorithms are used to enable a multitude of applications.

Application developers who wish to make sense of rotational motion must master Euler angles, rotation matrices, and quaternions. Under the hood, sensor fusion algorithms must be used in order to create responsive, accurate, and low noise descriptions of motion. Reducing sensing errors involves compensating for temperature changes, magnetic disturbances, and sharp accelerations. Some of these algorithms must run at a very high rate and with very precise timing, which makes them difficult to implement within low-power real-time operating systems. Within Android specifically, this involves modifying the sensor manager, introducing new APIs, and partitioning motion processing tasks.

David Sachs began developing motion processing systems as a graduate student at the MIT Media Lab. His research there led him to InvenSense, where he continues this work with MEMS inertial sensors used in products such as the Nintendo Wii Motion Plus. David's designs incorporate gyroscopes, accelerometers, and compasses in various combinations and contexts including handset user interfaces, image stabilizers, navigation systems, game controllers, novel Braille displays, and musical instruments.

Comments

    • John Belmonte
      John Belmonte

      Regarding the gravity signal, he says "it's actually mostly gyroscope data, but with the accelerometer used to correct drift" (at 17:59). I can't find any information on how this is done.

      The W3C Motion Sensors Explainer hints at the same thing: "On most sensor hubs, gravity is isolated from the accelerometer using the gyroscope" (https://www.w3.org/TR/motion-sensors/#relative-orientation), but also lacks any detail.

      about 6 months ago
    • zichen liu
      zichen liu

      Even 8 years later, it seems to be the most comprehensive talk about sensors and sensor fusion I have ever watched. Thank you!

      about 7 months ago
    • László Heller
      László Heller

      Good presentation, explains the things behind!

      Although now in the latest APIs there are composite sensors like
      - ROTATION VECTOR (accelerometer + magnetometer + gyroscope)
      - GAME ROTATION VECTOR (accelerometer + gyroscope)
      - GEOMAGNETIC ROTATION VECTOR (accelerometer + magnetometer)
      which are using SensorFusion technique by default and their output can be used in methods like "GetRotationMatrixFromVector", "GetQuaternionFromVector".

      See also:
      https://source.android.com/devices/sensors/sensor-types#attitude_composite_sensors

      about 1 year ago
    • tom P
      tom P

      very good video, helps me so much writing my master thesis. additionally he has so a clear pronunciation

      about 1 year ago
    • Darwin Harianto
      Darwin Harianto In reply to tom P

      sorry, I know this is inappropriate, but can I see your thesis?

      about 11 months ago
    • Claudio Silvaggi
      Claudio Silvaggi

      Check out Ken Wheeler "Missing Secrets of Magnetism" 3rd edition.

      about 1 year ago
    • iradeourum
      iradeourum

      what is the "inegral" ??

      about 1 year ago
    • HiroVlogs
      HiroVlogs

      Anyone want to give definitions for all the terminologies used here?

      about 1 year ago
    • Christian Gingras
      Christian Gingras

      Interesting to compare the real world design vs the point of view of abstract University teacher who wrote the wikipedia pages about what should be the same subject.

      How the GPS work so well in any cell phone? The real data from the satellite is not precise enough to detect that you arrive at a street in 50 feet. But try to discover how the cell phone work from any web site or youtube video..

      Nobody knows. Wikipedia just mention the expensive equipment that are mandatory in commercial airplanes. They know the cell phone in their pocket appear to match the performance of these devices. But all they can say is : thanks to Einstein, science works. The black hole in their knowledge is discretely ducted under the rug.

      This video published in August 2010 explain the difficulty of sensing linear displacement from the 6 sensors. A good engineer can fill the gaps, extrapolate what the device do starting from the noisy input sensors to the smooth animation that Google map and other similar application draw of the cell phone.

      about 1 year ago
    • Sanjiv Soni
      Sanjiv Soni

      I saw this video again after three months and theres still so much to learn about this. This is so far the best resource on Motion Sensors and sensor fusion I have come across.

      about 1 year ago
    • SHOKH JAKHON KAYUMIY
      SHOKH JAKHON KAYUMIY

      what is the data format of 9-axis combination data?

      about 1 year ago
    • Tadesan
      Tadesan

      5:00 street view is run by the same technology pioneered by the video game Mist.

      about 1 year ago
    • Dan-Marius Dobrea
      Dan-Marius Dobrea

      Hi! If you want an app capable of working with all the sensors from CC2541DK, CC2650STK, ThunderBoard React and Sense (e.g. acclrm., magnetom., gyro. etc.) you can download it from: www.microsoft.com/store/apps/9p054xsjjr1n – the software is able of giving new uses (movement security systems - door or luggage, magnetic security system, accelerometer mouse and wireless presenters) to all these IoT SensorTags. For more info.: www.blesstags.eu. Thank you so much and have fun!

      about 1 year ago
    • Jian Guo
      Jian Guo

      Even 6 years later, it seems to be the most comprehensive talk about sensors and sensor fusion I have ever watched. Thank you!

      about 2 years ago
    • Evil Raccoon
      Evil Raccoon In reply to Jian Guo

      ok, 3 jun 2019 the same. I am happy to find this video.

      about 23 days ago
    • Arpan Das
      Arpan Das In reply to Jian Guo

      Even almost 9 years later...

      about 3 months ago
    • Habib
      Habib In reply to Jian Guo

      Indeed ...

      about 11 months ago
    • TİK Sunumu
      TİK Sunumu In reply to Jian Guo

      Even almost 8 years later, I totally agree :)

      about 1 year ago
    • Mateusz Mittek
      Mateusz Mittek

      Thank you!

      about 2 years ago
    • Ashish Vijayvergiya
      Ashish Vijayvergiya

      Can someone explain how the pan change worked at 38:00 assuming I have got the rotationMatrix from the sensor library, how to I pan the 2D image on my mobile screen?

      about 2 years ago
    • Aleksandar Kospenda
      Aleksandar Kospenda

      What we ripped out was the headphone jack. Apple 7 copy pasta.

      about 2 years ago
    • Mathieu Brouwers
      Mathieu Brouwers In reply to Aleksandar Kospenda

      I was looking for this ^^

      about 2 years ago
    • Aleksandar Kospenda
      Aleksandar Kospenda

      What we ripped out was the headphone jack. Apple 7 copy pasta.

      about 2 years ago
    • aa
      aa

      38:30 "Yes, it's pronounced "Oiler". I'm not sure why."
      MAYBE BECAUSE HE WAS SWISS?? DURRR... Goddamn american dumbasses... Holy shit, that was retarded.

      about 2 years ago
    • Tadesan
      Tadesan In reply to aa

      aa ok, it's a Swiss name. Then when it's translated to English it should be spelled in English.

      about 1 year ago
    • Phygar1
      Phygar1 In reply to aa

      An engineer at MIT doesn't speak German, I'm triggered.

      about 2 years ago
    • Bo By
      Bo By

      26:44 I doesnt get it ...why is use Kalman filter, and what was about pedometer. Someone can me explain it ,please?

      about 3 years ago
    • Atayyab Malik
      Atayyab Malik In reply to Bo By

      it's for the positioning of the character..for example if you want to
      move your character by moving your device or want to map your footsteps
      as input to your character's step... he was actually giving alternate to
      the Accelerometer (to avoid double integer errors) and proposed
      pedometer and Kalman filter to use for the above mention task.

      about 2 years ago
    • HAWXLEADER
      HAWXLEADER

      yet still the nexus 6p has tons of problems with its sensors. some gyro axis are not on zero when static, gravity value of the linear acceleration virtual sensor is off and there is no way to calibrate any of them...

      about 3 years ago
    • Piotr Piskorski
      Piotr Piskorski

      It was incredible. This video help me a lot to approach my thesis based on staft related with acc. Thanks. GreatJob

      about 3 years ago
    • saxman58b
      saxman58b

      The best explanation of how to interpret quaternions I've seen. Thank you!

      about 3 years ago
    • TheiLame
      TheiLame

      42:17 Hi im little bit dumb. How does he get 9 numbers from 3 axis points? How does the twist work to give 6 extra numbers? "Those axis twist around with the teapot..." i dont get it.

      about 3 years ago
    • Mark Chinski
      Mark Chinski In reply to TheiLame

      Search youtube 'linear algebra matrix transformations' for more info.

      about 1 month ago
    • TheiLame
      TheiLame In reply to TheiLame

      @saxman58b
      Thank you very very much for explaining! :)

      about 3 years ago
    • saxman58b
      saxman58b In reply to TheiLame

      +TheiLame I believe what he meant was that each of the three rows of the 3x3 matrix are the vectors describing the new x, y and z axis relative to the original xyz dimensions. For example, if you wanted to just rotate around the z axis by 90 degrees then the matrix to convert old dimensions to new would be:
      [0 -1 0 (vector describing the old x axis relative to the new coordinate system after rotation, i.e. x' = -y)
      1 0 0 (vector describing the old y axis relative to the new coordinate system after rotation, i.e. y' = x)
      0 0 1] (Z axis doesn't change so all vectors keep their same z coordinate)

      To test it, pick an arbitrary position in space, for example (1, 2, 3). If you multiply the matrix above by this vector you get a resultant vector of (-2, 1, 3), which is exactly what was intended, a 90 degree rotation around the z axis.

      This was a simple example, but it should work for any 3D rotation. Just make each row of the rotation matrix equal the vector describing the new direction of the three local x, y, z axis of the object. And unit vectors are required in the matrix, or else the space / object will expand or contract. Thus a 45 degree rotation version of the above would be:
      [sqrt(1/2) -sqrt(1/2) 0
      sqrt(1/2) sqrt(1/2) 0
      0 0 1]

      about 3 years ago
    • TheiLame
      TheiLame

      This is 5 years old video, is it still up to date in terms of what they use? :) anyone knows?

      about 3 years ago
    • TheiLame
      TheiLame

      This is 5 years old video, is it still up to date? :) anyone knows?

      about 3 years ago
    • saxman58b
      saxman58b In reply to TheiLame

      +TheiLame The math and physics all still makes sense. What has changed is that most phones now have all of this hardware built in, and on the iPhone at least (I only do IOS development) the software functions are now built into the standard libraries. So no need to hack the OS or add hardware to get the gravity vector, user acceleration, attitude (via Euler, quaternion and rotation angles) etc. It's almost all there and ready to use with little or no extra processing.

      about 3 years ago
    • TheLimonerthi
      TheLimonerthi

      22 people love Jesus

      about 3 years ago
    • Kerem Kambur
      Kerem Kambur In reply to TheLimonerthi

      They are 36 now.

      about 1 year ago
    • Bruce Sha
      Bruce Sha

      youtube(google) should integrate IQ and Knowledge into thumbs-up/down. because that way when you see the ratio you'll have a better chance of correctly evaluate yourself!

      about 3 years ago
    • iHued Smartphone Garage Door Opener
      iHued Smartphone Garage Door Opener

      Cool video. See our Smartphone Garage Door Opener with webcam ⛔ #iHued-Garage ⛔ P.S. I Liked your video hope you can do the same, thanks ;-)

      about 3 years ago
    • Trocki Tomasz
      Trocki Tomasz

      Euler was Swiss, this is why his name is pronouced "Oiler".

      about 3 years ago
    • John Kruebbe
      John Kruebbe

      This is how we could do position in VSN. Thank you!

      about 3 years ago