AskHistorians

I can't speak to the origin, it's probably fundamentally unknowable, but I do think what's most likely is the conjecture about the combined influence of sleep paralysis hallucinations of proto-faces in the dark, seeing barn owls staring at you when you're cranked on tainted moonshine, and half-remembered descriptions of monsters from fairy tales. As for why it caught on, descriptions of alien encounters follow a cycle:

1. A person who may or may not actually believe what they're saying will present a story about close encounters.
2. This will be picked up by attention-seekers who lie about having the same experience, and by schizophrenic people who experience the very common phenomenon of thinking something they heard about personally happened to them.
3. With time, these stories will influence how aliens are portrayed in popular media.
4. The presence of that media in the zeitgeist will in turn influence following alleged close encounters.

Key details in the way things are described by the most recent person to get some sweet, sweet attention will immediately pop up in subsequent accounts and descriptions—among other people in that culture: notice that roswell-style alien greys are sighted almost daily in the Anglosphere, especially the United States, and almost never elsewhere. This feeds pop culture descriptions of aliens, and the release of the movies or TV shows invariably triggers accounts of alien visitors that are now very similar to that portrayal with one or two details changed like when children copy homework.

While we all know that many people who've had close encounters are either outright liars or mentally disturbed people who report the event out of a compulsion to do so or even sincere belief, it's hard to overstate the influence of people who are just plain stupid.

The main event at play is that stupid people will see something they don't understand and get hysterical about it. Let's be clear: I don't mean people being mildly startled by rare phenomena like the Aurora Borealis being really far south, I mean that we live among morons who call 911 because they saw the planets during the day. When they see things completely beyond their grasp, like literal optical illusions, they default to "well I am special and mommy's smart big boy, and I don't think that seems right, so it isn't". Add to that the fact that we have rocket launches and all sorts of flying toys for the military, and it's a recipe for disaster. If they get confronted with something like "could that UAP just be a party balloon, airplane, or helicopter and it's not at the distance or speed you think?" they respond with bloodcurdling shrieks that you're in the CIA. Belligerent oafs of this kind are in my completely asspulled opinion the primary driving force behind the establishment of the cornerstones of alien lore.