Outside the Dirksen Senate Office Building Tuesday, a collection of conservative groups (R-Penn.).
The newly re-elected moderate is in line to chair the Senate Judiciary Committee and he spent much of the day meeting with committee members.
But Specter was not invited to meet with fellow Pennsylvania Sen. Rick Santorum. Santorum is staunchly pro-life, and he is caught between his allegiance to Pennsylvania voters who supported Specter and the ambitions of a powerful conservative Sen. Majority Leader Bill Frist.
The conflict began just after the election when Specter, a longtime supporter of abortion rights . "When you talk about judges who would change the right of a woman to choose, overturn Roe v. Wade, I think that is unlikely," said Specter on Nov. 3.
The White House was infuriated by Specter’s remarks and the senator quickly issued a clarification saying there should not be a litmus test.
But over the last week, conservative groups, led by the religious Right, have been flooding the phone lines of Senate Republicans including Majority leader Bill Frist, who is preparing a campaign for the 2008 Republican presidential nomination.
This weekend, Frist refused to endorse Specter’s chairmanship.On Fox News Sunday, Frist said, "What I expect is for a chairman to understand that they are no longer responsible just to themselves or just to their constituents back at home; but as chairman of the committee, they're responsible to the feelings, to the wishes, the beliefs, the values, the procedures that are held by the majority of that committee."
And the committee includes not only Pennsylvania’s Sen. Santorum, but another hardcore conservative, Jeff Sessions of Alabama.
Adding to the intrigue, 18 years ago, when Sessions was nominated for a federal judgeship, Democrats blocked the nomination with the help of a young Arlen Specter.
Other conservatives still have not forgiven Specter for helping to derail the Reagan era Supreme Court Nomination of Robert Bork.
This past weekend, to try and tamp down the latest controversy, Specter made a round of television appearances.
Specter continues to try and reassure Republicans that he will carry out the president’s wishes. Conservatives acknowledge Specter probably does not have the backing of enough Republicans to be chairman, though they suggest the chilly response from Specter’s own Pennsylvania colleague should serve as a warning that this is a new era on Capitol Hill.
Watch more of "Shuster's Reports" on 'Hardball,' weeknights, 7 p.m. ET on MSNBC.