The show premiered on Fox on Wednesday with Simon Cowell, Paula Abdul, Nicole Scherzinger and L.A. Reid judging the contestants.
The Parents Television Council (PTC) has lodged a broadcasting indecency complaint with the Federal Communications Commission after contestant Geo Godley was shown dropping his trousers and exposing himself during a song and dance routine on the two-hour show.
Godley, a 43-year-old blogger from Seattle, was wearing a G-string at the time and producers of the show covered sensitive areas with the X Factor logo when it aired.
The routine was met by anger from the judges. Abdul walked off stage, while Reid described it as "offensive, disgusting, distasteful and upsetting".
President of the Parents Television Council Tim Winter said Godley's performance was "indecent".
The group is angry that the clip was broadcast shortly after 9pm, during the hours when indecent content is prohibited on broadcast television.
The incident comes as the US Supreme Court is due to consider the broadcast decency law.
The PTC said Godley's performance offered a "compelling demonstration" of why the law needs to be upheld.
"If Godley performed his act in public, he would have been arrested," said Mr Winter.
"But if he performs it in front of a Fox camera, his act is beamed via the public airwaves into every home in the nation."
"The prolonged, previously videotaped footage of a contestant dancing nude on the 'X Factor' stage represents a conscious decision by the producers – with the approval of the network's broadcast standards department – to intentionally air this content in front of millions of families during hours when they knew full well that children would be watching."
He continued: "Families were led to believe the 'X Factor' would be family-friendly programming and instead were assaulted by graphic nudity.
"While we understand the entertainment hook that comes from the occasional train wreck, something so inherently and patently offensive as this obscene performance is more than a sucker punch; it is a violation of Fox station's broadcast licenses.
"We have filed a complaint with the FCC and are urging our members and other concerned citizens to follow suit."