The phrase "rub the wrong way" means to irritate or annoy someone. The origin of this phrase is somewhat unclear, but there are a few plausible theories about how it came into common use. 

One theory suggests that the phrase comes from the idea of petting an animal, particularly a cat, against the direction of its fur. When you rub a cat’s fur in the wrong direction, it can cause discomfort or irritation to the animal, leading to an annoyed or agitated reaction. This imagery translates well to the idea of irritating or annoying a person by doing something they find unpleasant. 

Another theory is that the phrase originates from the practice of cleaning or polishing wooden surfaces. Rubbing wood against the grain, or in the "wrong way," can cause scratches, damage the wood, or create an uneven finish. This practice would be seen as incorrect and potentially damaging, hence the metaphor for causing irritation or annoyance. 

The phrase "rub the wrong way" appears in early American literature, indicating its use in colloquial speech. For instance, it is found in James Kirke Paulding's book "Westward Ho!" (1832): "They are so contrary that they will rub everything the wrong way."