Alright Squares; Let's Talk Triangles
Hi and welcome to my first blog post! Today we'll be talking about the fascinating subject of TRIANGLES. Those of you aware of static offsets for angles C and B (in the context of autonomous vehicles) can skip this next paragraph.
This discussion is in the context of the behavioral cloning project in Udacity's Self Driving Car Engineer program. Udacity, like NVIDIA in their self driving vehicle, provides 3 camera inputs when recording driving behavior in their simulator. There is the center camera (represented by D in the diagram) and the left and right cameras (represented by C and B respectively). This allows neural networks to receive 3x as much data due to knowing what it's like to drive slightly to the left and right of the car when it's recorded. However, since the cameras are shifted slightly - so too does the steering angle of the car need to be shifted slightly (notice how C and B are not right angles like D?). Since the simulator only knows the center camera steering angle, Udacity, and (so far as I can tell) NVIDIA, suggests using a static camera offset for each side camera. This is to say that when the point A moves closer to point B, the absolute value of the steering angle from camera B will be equal to the absolute value of the steering angle from camera D minus the offset. (Whether this is a +/- is dependent upon the frame of reference). Likewise, if angle D shifts towards C, the absolute value of the steering angle from camera C will be equal to the absolute value of the steering angle from camera D minus the camera offset. What you see in the diagram is the state of angles B C and D when angle D is .5*pi and the steering angle from camera D is zero. Do I have you so far? The framework for this problem - and it is a problem - is over. Thank goodness; NOW TO THE FUN PART! I had originally used a static offset of .043 radians (or .1 in my model.py - it uses degrees/25), as that gave really strong results when driving straight - but suffered during turns. Mind you, I only used one or two drives around each course, so there was not a lot of data. Once I began scaling the farthest camera in relation to the center camera's angle, I got much stronger results with little data.
Well, for those of you that are only interested in the result and not the process/reasoning you can skip ahead to the next ALL CAPS paragraph start. When I started on this project, I had a fancy formula which scaled the angle of the point which was farthest from the goal (Point A) as a function of angle D which went something along the lines of... The steering angle of C is equal to the steering angle of D multiplied by the steering angle of D over the maximum angle of steering angle D all added to the offset plus the original angle. So...
Angle C = (Angle D * Angle D / Max(Angle D) ) + Offset + Angle D.
I started teaching myself trigonometry again to find out why my intuition in scaling the farthest angle caused my car to drive so well. But as I found out over many pages of manual calculation, if the length of line A-D is constant, then no matter what the change in the angle of the center camera is... the offset remains constant? Then why was I experiencing stronger results by scaling the offset of the farthest camera? The answer lies in a change of the length of line A-D. As the car is making a stronger turn from the center angle, it can be safely assumed that the distance between the center camera and its goal (point A) is shrinking in relation to when point A was a significant distance away from the car - when steering angle D was 0. This can easily be seen in how far away the most distant center point on the road is when the car is driving straight versus when it is in a 25 degree turn. After getting tired of manually calculating SOH CAH TOA/law of cosines and sines over and over again, I wrote a couple python scripts to chart out what the change in the angle would be given a center angle and a distance between each camera along with a distance to the goal at 0 degrees and the distance to the goal at the given angle. I chose parameters such that an arbitrary unit ~1 was equal to the distance between each camera (and around half the width of the car) if the distance between the center camera and the goal when driving straight was ~22 arbitrary units away given that the camera view was very close to the ground and it could not see that far. This worked well with my original experience of .043 radians being perfect while driving straight. I also brought back the angle times the angle over the maximum angle for determining which side angle to use for determining the length of line A-D in relation to the center angle offset; which worked really well for me and scaled in a reasonable way. SO WHAT DOES ALL THIS MEAN? Well, there's a bigger formula at play for our bisected triangle here. I don't know what it is; I only have a worn down and over-applicated version of that formula. BUT THERE IS A RELATIONSHIP! And you can predict it, and you can use it to get incredible results out of your offset cameras. I've only given you half the puzzle for the camera most distant from the goal of the car, but the results are undeniable. These are using the same two neural network architectures, sets of data, and preprocessing - except for the fine tuning of the offsets:
First, using a set of offsets which are dynamic (and performed much better than the original static offset of .043 radians) with a reduction of samples which have an absolute value of less than .01 radians for the center angle. This example exhibits the "drunk driving" phenomenon popularly reported:
Second, the same dataset, neural network, and preprocessing; but with an offset derived by the trigonometric relationship discussed before:
So squares; I hope I've made my case for dynamic triangles.