François Legault’s remarks come despite his government plan to ban civil servants from wearing items of clothing such as hijabs and the Jewish skullcap.
The policy has been widely criticised as it targets minority groups.
A 2008 report said the crucifix which has been hanging in the legislature since 1936 should be removed.
But the Quebec government refused to implement its findings.
The debate over religious symbols in Quebec public life has been a perennial issue for the past decade.
In 2014, the Parti Quebecois proposed a so-called Charter of Values bill, that would ban all public servants from wearing “ostentatious” religious symbols or clothing.
Many decried the bill as Islamophobic and anti-Semitic, but proponents said the law was meant to promote secularism and separate Church and state.
Since winning a majority in last week’s election, Mr Legault’s Coalition Avenir Québec party has renewed the call to ban all religious symbols, and has even said he would consider firing teachers who refused to comply with the ban.
“We have to understand our past,” Mr Legault said.
“In our past we had Protestants and Catholics. They built the values we have in Quebec. We have to recognise that and not mix that with religious signs.”
His willingness to ban religious symbols – whilst refusing to take the crucifix down from the legislature – has drawn much criticism.
Canadian Prime Minister Justin Trudeau, who is from Quebec, said the state had no right to tell women who wear the hijab what they can and cannot wear.
Meanwhile, teachers and other civil servants have vowed to fight the proposed ban.
“Come look at what goes on in my classroom and what we’re learning and discussing, and see for yourself if it’s not neutral because I’m wearing something on my head,” Furheen Ahmed, a teacher who wears the hijab, told iPolitics.
“And I’d ask them: ‘What are you afraid of?’ Because to me this is coming from a place of ignorance and/or fear, and the ignorance, if you come sit in the classroom and see what happens, that might be lifted.”