Women and children often stay in abusive relationships because they have nowhere else to go, while many who flee violence become homeless.
Homelessness support services have told Victoria's royal commission into family violence there are limited housing options for women and children escaping abuse.
More than a third of homelessness is linked to family violence, the Council to Homeless Persons says.
Its research shows 56 per cent of homeless young people say they had to leave because of a violent family member.
But in many cases, families could avoid becoming homeless if the perpetrators were made to leave the home.
'If you knew the justice system could keep them away, you would be much more likely to stay at home,' Swinburne University lecturer Angela Spinney said on Tuesday.
'There are more children living in refuges in Australia than there are women, and that's a really shocking fact.'
Many families are forced to move between crisis accommodation every few weeks, if they can access one, she says.
Some refuges have rules about the number of kids per family they can house or prohibit boys over 12.
A domestic violence survivor last week told the inquiry a refuge once told her they only had room for two of her four children.
'I was offered the chance to pick which two of my children would stay with me,' she said.
Lack of long-term crisis accommodation is further compounded by the lack of housing for violent offenders.
'It's in our interest to house that one perpetrator rather than three or four people,' Dr Spinney said.
Allowing violent offenders to stay at home is an outdated approach based on the assumption that women must leave violent relationships, she said.
Meanwhile, Legal Aid Victoria is calling for changes to national laws that allow family members to represent themselves and cross-examine each other in the Family Court, to prevent perpetrators of violence from questioning their victims.