No parent may remove their child from the UK without the permission of anyone else with Parental Responsibility such as another parent, or without an order from the courts.
This means that if your ex and parent of your child has decided that they wish to return to their country of birth, or start a new life somewhere abroad they must seek out your permission so long as you have Parental Responsibility.
However, if you decide that your child moving to start a life abroad is not in their best interests then you have the right to refuse their request.