I will say, don't make it happen if you are really care of them. Australia is a fantastic place for English learning if this is the only reason you have. Believe me it is painful to live in a unfamiliar country without friends and those you love, even your wife has to look after a kid just by herself!
i am also considering if i should take my son who is only 2.5 years old to Adelaide. I have full time work and can drive. But i just don't know how to deal with all the issues if i look after him on my own in this isolating country.