Sometimes one can spend hours of searching for information on an ancestor only to be corrected months or even years later. It's a good thing because you receive the correct information, but it's frustrating because you find you wasted quite a bit of time. Thankfully, I didn't waste much time in this case.
I didn't question whether or not this was his residence and I cannot recall asking to see the proof in writing. Months down the road I received and email from a friends that talked with an individual who had photos of my grandfathers house. Of course it was not the house I had taken pictures of. I went into investigation mode and noticed the house on the right in the picture. The house on the right of the picture above was not on google earth (street view), so I had to email an individual from Cape Charles that I knew. I asked him to take a picture of the house on the right of the picture I took. I was somewhat surprised at the result. Below is the picture given by one of the sons of my second great grandfather. Then the picture next to it is the picture of the house my friend took of the house next door to the picture above. It looks like we were wrong.