SNP Tommy Sheppard urged David Mundell to "stand down and make way for someone who will respect the wishes of the Scottish people and respect the national government of Scotland".
Mr Mundell insisted he would "continue to stand up and defend Scotland's place in the UK", as he said the Government's "door remains open for the Scottish Government to reconsider" its opposition to the European Union (Withdrawal) Bill.
He said: "Having worked closely with the devolved administrations on significant amendments, I'm of course disappointed that the Scottish Parliament has not yet granted legislative consent to the European Union (Withdrawal) Bill.
"The Welsh Assembly agrees these arrangements fully respected the devolution settlements. (David Lidington, Chancellor of the Duchy of Lancaster) is in correspondence again this week with Mike Russell and the door remains open for the Scottish Government to reconsider."
But SNP MPs lined up to berate the Government over its approach to the Brexit legislation.
SNP Joanna Cherry (Edinburgh South West) said: "Can the Secretary of State tell us whether he has explained to the Prime Minister that by a majority of three to one MSPs, four out of five parties in the Scottish Parliament withheld legislative consent and what has he advised her to do to recognise this overwhelming expression of the democratic will of the Scottish people?"
Mr Sheppard (Edinburgh East) said: "It takes a particular form of arrogance to try and force through a position which is supported by only one of the five political parties in Scotland and less than one quarter of the members of the Scottish parliament.
"Four out the five political parties in Scotland now understand that this is the first Secretary of State for Scotland in history who seeks to lessen the control of the Scottish people over their own affairs.
"Will he now stand down and make way for someone who will respect the wishes of the Scottish people and respect the national government of Scotland?"
Mr Mundell replied: "He let the cat out of the bag with his final words, Scotland has two governments, that's what Scotland voted for in 2014, for Scotland to be part of this UK and I will continue to stand up and defend Scotland's place in the UK."
SNP Patrick Grady (Glasgow North) added: "This is the Secretary of State who promised, who vowed to make Holyrood the most powerful devolved parliament in the history of the known universe, so why is he prepared to see this Westminster parliament override the rule of the Holyrood parliament that has withheld its consent."
Mr Mundell replied: "I'm not going to take any lectures on devolution from the SNP.
"Only today Nicola Sturgeon writes ahead of the SNP conference, that this weekend marks the start of a road of a new chapter in Scotland's route to independence, that doesn't sound very much like standing up for devolution to me.
"All along I've wanted to reach agreement, I still want to reach agreement in the exchanges this week with the Scottish Government, we've made it clear that we still want to reach agreement, but either the Scottish Government needs to reconsider their position or a new proposal needs to emerge."