Is it Tele spank or a set up issue?

OilCityPickups

I had a similar issue with a Tele some while ago had another neck spare that had been fret dressed and swapped that out to eliminate a high fret. It was still there. This was with a six saddle bridge, and it drove me potty. I shimmed the neck for a larger break angle over the saddles and the problem went away (raising the saddles to give the same action). 

Frizer

There is a characteristic to some Teles similar to what you describe on the high E string. Can normally make it go away completely by using a top loading bridge, assuming you want to.