It goes without saying that Milo Ventimiglia plays the perfect husband and father on This Is Us without being a 1990s sitcom cliche — he’s devoted, he’s loving, and he’s super freaking sexy!

Turns out, this role is a blessing and a curse. While it’s brought him lots of attention, the actor says it sets a high bar for his real life romances.

“Totally hurts,” he told ET’s Leanne Aguilera on the Television Critics Association press tour red carpet on Thursday. “Totally hurts because people are like, ‘Man, I wish my husband were more like Jack.’ ‘Man, I wish my boyfriend would do the things that Jack would do.’ And I’m like, Jack’s not real. The dude standing in front of you is real and he’s probably sitting down watching the show with you, so therefore, give your dude some love.”

As far as how he’d grade his looks: “I’m neither here nor there. I personally like the mustache, but I don’t have the mustache right now,” adding that he’ll “never” take credit for continuing the mustache trend on TV. “I wouldn’t say it hurts me. It just kind of is what it is… Playing this couple that has been through this argument that we all saw at the end [of last season] and knowing the road of repair that they’re going to have to get back to being the couple that we love them to be, yeah. Mandy and I, we just want those happy moments because they’re fun to play. When it’s heavy, you’re like, ‘I really want to have a light moment.’”