NEW YORK (AP) — Republican U.S. Rep. Michael Grimm, a former Marine and FBI agent who’s been the target of a campaign finance investigation, was instead charged on Monday with evading taxes by concealing more than $1 million in sales and wages while running a small Manhattan restaurant — a case he called a political witch hunt meant to drive him out of office.

Grimm surrendered to FBI agents early Monday following a two-year investigation that initially focused on alleged attempts to bypass contribution limits. He pleaded not guilty through his lawyer in federal court in Brooklyn to mail, wire and tax fraud charges and was released on $400,000 bond secured by his Staten Island home.

The lawmaker, using a World War II memorial across the street from the courthouse as a backdrop, later accused prosecutors of leaking “all kinds of innuendos and accusations to support a political witch hunt” intended to “assassinate my character.” He vowed to return to work in Congress while fighting the charges.

“I know I’m a moral man, a man of integrity,” he added. “And I also know I have a lot more service and leadership to provide this country, and that’s exactly what I intend to do.”

Grimm, though, stepped down from the House Financial Services Committee. He told Republican House Speaker John Boehner he should be removed from the panel but said he plans to return once his legal issues are resolved. House Democratic leader Nancy Pelosi had sent a letter to Boehner saying Grimm should be removed, and a Boehner spokesman said the speaker believes the decision to step down was appropriate.

U.S. Attorney Loretta Lynch said investigators uncovered the restaurant fraud as they conducted the campaign finance probe. She denied the prosecution was politically motivated and suggested more charges were possible.

“We follow the evidence and see where it goes. … The investigation is broader and ongoing,” she said.

Grimm “was anything but an upstanding citizen,” said George Venizelos, head of the FBI’s New York office. “He cheated, evaded and then lied.”

The 20-count indictment alleges the tax fraud began in 2007 after Grimm retired from the FBI and began investing in an Upper East Side eatery called Healthalicious. It accused him of trying to evade payroll, income and sales taxes by paying his workers, some in the country illegally, in cash.

Grimm “physically handed out cash payments to his employees on numerous occasions,” the indictment says, with money taken from the restaurant’s daily receipts.

Authorities say Grimm left the business in 2010, the year he won his first term in Congress. A House Ethics Committee announced in November that he was under investigation for possible campaign finance violations.

During the 2010 race, Grimm acknowledged receiving $250,000 to $300,000 in contributions from followers of an Israeli rabbi, Yoshiyahu Yosef Pinto. Some members of Pinto’s congregation subsequently said they made tens of thousands of dollars in illegal contributions, including gifts passed through straw donors.

The Israeli businessman who had served as Grimm’s liaison to Pinto’s followers, Ofer Biton, pleaded guilty in August to an immigration fraud charge. Three days after that, the FBI filed a sealed criminal complaint accusing a Houston woman, Diana Durand, who had been romantically involved with Grimm, of using straw donors to make illegal campaign contributions.

Grimm, 44, made headlines in January after confronting a New York City cable news station reporter who tried to question him about the campaign finance inquiry.

After reporter Michael Scotto finished his report from a balcony in the Capitol, Grimm stormed back, leaned into him and said: “Let me be clear to you. If you ever do that to me again, I’ll throw you off this (expletive) balcony.”

Grimm’s legal troubles, while embarrassing to his fellow Republicans, were unlikely to determine the balance of power in the GOP-dominated House. Republicans have the majority in the House by a margin of 233 to 199 seats, with three vacancies.

Democrats, meanwhile, have been eyeing Grimm’s seat. The Democrats’ favored candidate, former New York City Councilman Domenic Recchia, has been hammering Grimm over his ethics and has impressed Washington Democrats.

News of the charges drew a mixed reaction from constituents on Staten Island.

“This is supposed to be someone who respects the law, and that’s certainly in question at this point,” said Marc Patrick, a retired teacher.

Eddie Garcia, who’s also retired, said he had voted for Grimm and credited the congressman with helping Staten Island recover from Superstorm Sandy.

Grimm is being prosecuted “because he is a rising star in the Republican Party,” he said. “They think they got him. I hope justice will prevail.”