Since infant and toddler brains are programmed to learn language, that's the ideal time to expose them to more than one. I've read that in a bilingual family it's easiest for the baby to learn if the mother always uses one language and the father always uses the other when speaking to the baby. It's easier for the baby's brain to separate the languages when they come from different people.
When my daughter was born we both used Total Communication (speaking and signing at the same time) because that was recommended by the school our deaf foster kids attended and she easily learned both. However speaking and signing are obviously different.
The older kids get the harder it is for them to learn a new language because their brains are changing.
Back when signing was forbidden in schools for deaf kids, or even more recently when those kids didn't start learning to sign until Kindergarten, the ones who were not from deaf families were at a great disadvantage because they were past the ideal learning age. I've heard of a few "feral" children, supposedly raised by animals without human contact for several years, who couldn't learn to talk, but don't know if those stories are accurate.
By the time humans reach puberty language learning has become quite difficult. It's too bad many school systems don't teach languages until high school.