It seems like every day The Bachelorette’s Jef Holm and/or Emily Maynard have to deal with a new rumor. Do you ignore it or do you fight back?

Well, when it comes to your family you’re probably more likely to get mad and get fightin’ — and that’s what Jef did in response to In Touch’s new story, headlined "Bachelorette's Jef Holm's Family Poligamy Scandal: 'Jef's Grandfather Had 11 Wives.'" They even feature a scan of Carl Otto Nathaniel Holm and his alleged wives. The story does say that Jef's immediate family members not practice polygamy, but the report claims "many of his relatives — including aunts and uncles — are still followers of the FLDS Church, a sect of Mormonism that still practices polygamy."

Jef has said he was raised in the LDS faith but is not practicing at this time. He seems to think — understandably — that a story digging up stuff about his distant relations is crossing the line. He tweeted Wednesday night, “Now they're going after my family? My family DOES NOT, has HAS NEVER had any affiliation with polygamy. Ever. Mormons = 1👦 & 1👩. 👫 = ?.” Apparently the boxes represent male and female faces, as tweeted from a phone.

Sigh. These are the perils of signing up for The Bachelor/ette. You can get a huge, supportive fan base, a new love, and invitations to cool events, but you also put yourself — and those around you — in the crossfire for potentially hurtful rumors and speculation. Everything in life is a trade-off! Hopefully Team JEM can stay strong and go the distance, like The Bachelorette Season 7’s Ashley Hebert and JP Rosenbaum. Team Cupcake had to deal with the rumor mill too, but now they are all smiles and wedding plans.

Sources: In Touch, E!, @JefHolm