If you are Australian born - you cannot "give up" Australian citizenship. You can make an announcement that you no longer consider yourself a citizen, and act as though you are not a citizen - however at if any point in the future you wish to take up your rights as a citizen again - they will still recognize it.
For countries t=who do not allow dual citizenship - it is basically just a construct in their own minds - they are saying they will not recognize you status as a citizen of another country - they cannot force another country to not continue to view or treat you as a citizen.
Giving up your passport wont change anything from Australia's perspective. If you later want a new one you would just go and apply for it an receive it the same as any other citizen.