After a nine-year federal investigation, the fraud and conspiracy case against former labor leader Anthony "Tony" Rutledge came down to this: a guilty plea to a single count of filing a false tax return in 1997.

"They got more than their one pound of flesh from me, for sure," Rutledge said yesterday.

Under a plea agreement with federal prosecutors, Rutledge will serve three years' probation and agreed to permanently sever his ties to Unity House Inc., the $42 million nonprofit labor organization founded by his father that was seized by the IRS in 2004 after questions were raised about its finances.

U.S. District Judge David Ezra ended the government takeover of Unity House yesterday, although he allowed the court-appointed company that has been running the nonprofit to take an extra four months to wrap up ongoing collection lawsuits and financial negotiations.

Ezra also noted that there is a "serious" ongoing IRS civil audit of Unity House's finances and its status as a tax-exempt organization.

Although Rutledge is required to cut his connection to Unity House, control of the organization will be returned to a Rutledge-friendly board of directors.

Ezra said in court that he didn't know if the plea agreement reached by the government with Rutledge met the ends of justice, but noted: "While there were serious (additional) charges laid against him, they have not been proven, and he is not guilty."

He said the court is required to give the government "a wide bracket of deference" when it negotiates plea agreements.

Rutledge's son, Aaron, a co-defendant in the case, pleaded guilty to a misdemeanor criminal charge of witness harassment and will serve one year of probation.

The two men were charged with multiple counts of fraud and conspiracy in a lengthy criminal indictment returned in December 2004. If convicted of all the charges, they faced a maximum of 53 to 93 years in prison and hundreds of thousands of dollars in fines.

Ezra rejected an earlier, though similar, version of Tony Rutledge's plea agreement in November because it would have allowed Rutledge to return to a position of financial responsibility at Unity House. The judge said he again rejected a version of the agreement Friday because he wanted to be sure the court-appointed receiver running Unity House was allowed enough time to complete ongoing activities which are important to the nonprofit's 20,000 beneficiaries.

After the plea deal fell through Friday, an angry Tony Rutledge told reporters that he believed Ezra had been "biased against me for many years" and was part of a "conspiracy" with two former prosecutors in the case, Marshal Silverberg and Edward "Ted" Groves.

Ezra denied those claims yesterday and said he wouldn't hold the comments against Rutledge.

"In all my years as a judge, that's maybe the first time a defendant has criticized the court before he was sentenced," Ezra said. "Usually it's after."

The prosecutors filed papers justifying the deal, but the papers were sealed because the government said making them public would unfairly expose legal strategies if the case had to go to trial and might cause "embarrassment" to witnesses in the case.

Rutledge's attorney Jeffrey Rawitz had high praise for the new team of prosecutors that took over the government case last summer, and harsh criticism for the two previous prosecutors.

Edmund Power and John Cox, two Washington-based attorneys from the Department of Justice who negotiated the plea bargain with the Rutledges, "are among the most honorable and decent prosecutors I've ever dealt with," said Rawitz.

But former prosecutors Silverberg, an assistant U.S. attorney in Hawai'i, and Groves, a former special attorney with the Department of Justice tax division, "represent the worst of the Department of Justice," said Rawitz.

Elliot Enoki, first assistant U.S. attorney in Hawai'i, said his office couldn't comment on Rawitz's statement. "Review of the conduct of federal attorneys is generally a function of another part of the Department of Justice," he said.

Groves, who left government service last year to go into private practice, in November called the government's plea bargain "a travesty of justice."

The ousted board of directors at Unity House that now moves back into power includes another of Tony Rutledge's sons, Anthony Rutledge Jr.

The plea deal empowers the reinstated Unity House directors to pay more than $1 million in legal fees owed by Tony and Aaron Rutledge. The board also is authorized to pay the men unspecified amounts in "severance pay" as well as compensate them for lost vacation, pension and health benefits.

Ezra must appoint two new members to the Unity House board of directors from a list of five names submitted by the Rutledges and the government. The names include former Honolulu Mayor Frank Fasi, two former city managing directors — D.G. "Andy" Anderson and Robert Fishman — former Unity House public relations consultant James Boersma and Charles La Bella, a former assistant U.S. Attorney in San Diego and Washington, D.C.

Reach Jim Dooley at jdooley@honoluluadvertiser.com.