
According to the book "Weird NJ" (Sceurman, Mark and Mark Moran, Barnes and Noble Books, 2004 ISBN 0-7607-3979-X) he was a descendant of the Morris County Van Cleefs who were infamous in the area for their strange living and "procreational" efforts. In fact, one trait that Lee had was that he had one green eye and one blue eye. According to the book, "this telltale characteristic was corrected in the movies with colored contact lenses".
(imdb.com)