A number of House Republicans are in talks to block Rep. Jim Jordan's path to the speakership as the Ohio Republican tries to force a floor vote on Tuesday, according to multiple GOP sources. One senior Republican House member who is part of the opposition to Jordan told CNN that there he believes there are roughly 40 "no" votes, and that he has personally spoken to 20 members who are willing to go to the floor and block Jordan's path if the Ohio Republican forces a roll-call vote on Tuesday. "The approximately 20 I've talked to know we must be prepared," the member said. "We cannot let the small group dictate to the whole group. They want a minority of the majority to dictate and as a red-blooded American I refuse to be a victim." But another GOP source familiar with the matter said that Jordan has had positive conversations with members and believes by Tuesday evening he will be elected speaker of the House. The House is expected to hold a vote for the next speaker on Tuesday at noon, according to an email from House Minority Whip Katherine Clark obtained by CNN. The GOP source said that Jordan may decide to go to multiple ballots on the floor if necessary. Republicans are expected to meet behind closed doors Monday evening. Yet there is still sizable opposition to Jordan. The GOP member says there are some Republicans who are critics of Jordan and not willing to back him – and there are others angry at the hardliners who took out Kevin McCarthy and sunk Majority Leader Steve Scalise and don't want to reward those moves by electing Jordan, who is their preferred candidate. "I know of many hard nos. ...We can't reward this behavior," the GOP lawmaker said. "We can't let a small group be dictators." The Republican conference nominated Jordan as speaker last week after Scalise dropped his bid for the role. Scalise had initially been selected by the conference as its nominee – after he defeated Jordan 113-99 in the conference's first speaker vote – but more than a dozen Republicans said they would not vote for Scalise, forcing him to withdraw. Now Jordan is facing the same problem from Republicans angry at McCarthy's ouster and a small faction of the conference refusing to get behind Scalise after he won the first vote. After Jordan's nomination, he held a second, secret vote in the conference on whether Republicans would support him on the floor. Fifty-five Republicans voted no. To be elected speaker, a nominee must win the majority of the full House, which is currently 217 votes due to two vacancies. That means Jordan or any other Republican nominee can only afford to lose four GOP votes on the floor if every Democrat votes for House Minority Leader Hakeem Jeffries. Some of Jordan's allies have pushed for votes on the floor in order to try to call out the holdouts who aren't behind the Ohio Republican. But Rep. Dan Crenshaw of Texas railed against his House GOP colleagues who plan on rallying support for Jordan's speakership through a public pressure campaign, calling it "the dumbest thing you can do." "That is the dumbest way to support Jordan," Crenshaw told Jake Tapper on CNN's "State of the Union" Sunday. "As someone who wants Jim Jordan, the dumbest thing you can do is to continue pissing off those people and entrench them." — CNN