Individual A spoke to FBI agents who contacted him about his agreement with Hastert, but otherwise he has never spoken publicly about his accusations.
His attorney, Kristi Browne, said it was Hastert who had reneged.
Let our news meet your inbox. The news and stories that matters, delivered weekday mornings.
"Mr. Hastert has decided that rather than live up to his promise to compensate his victim for his molestation and resulting injury, he will ask his victim to pay him," she said in a statement.
"He admits to agreeing to make payments, but then denies that it is an agreement that he has to keep. His response is predictable and we look forward to entering the discovery phase of the case."
Hastert, 75, was sentenced in April to 15 months in prison for making illegal cash withdrawals to pay Individual A and quash allegations he abused teenage boys while he was a high-school wrestling coach decades ago.
Although the disgraced power broker was not charged with any sex crimes because the statute of limitations had passed, the judge in the case branded him a "serial child molester" and ordered him to enroll in a sex-offender treatment program.
The alleged abuse took place well before Hastert became the Republican speaker of the House of Representatives in 1999. A few years after he retired, he was confronted by Individual A about misconduct on a wrestling trip in 1974.
Hastert agreed to pay Individual A $3.5 million and had forked over $1.7 million of that sum when federal investigators began looking into his unusual bank withdrawals.
Hastert claimed he was a victim of extortion but after FBI agents interviewed Individual A, they found his account of molestation credible.
After Hastert pleaded guilty to the financial crime, Individual A sued him, saying he was still owed $1.8 million. In his response, Hastert says there was no legal contract and even if there was, his accuser would have broken it.
Individual A's "retention of the $1.7 million is unjust," his attorneys wrote.
Not only does Hastert want his money back, he thinks Individual A should pay his attorney's fees.
At his sentencing, Hastert admitted he "mistreated" and "took advantage" of former students who came forward to say they were molested -- three of them anonymously.
"What I did was wrong and I regret it," he said at the time.
Tracy Connor is a senior writer for NBC News. She started this role in December, 2012. Connor is responsible for reporting and writing breaking news, features and enterprise stories for NBCNews.com. Connor joined NBC News from the New York Daily News, where she was a senior writer covering a broad range of news and supervising the health and immigration beats. Prior to that she was an assistant city editor who oversaw breaking news and the courts and entertainment beats.
Earlier, Connor was a staff writer at the New York Post, United Press International and Brooklyn Paper Publications.
Connor has won numerous awards from journalism organizations including the Deadline Club and the New York Press Club.