I suppose this would be at least interesting to try..I usually only talk to people I know from real life online, but whatever..Not like you'll find out anything interesting
The only thing about TeamSpeak (dunno about Ventrilo..Never tried it that much), you have one central IP address which is your server. Once this is out, anyone can go on provided they have a password...For anyone who doesn't see the problems, just imagine a bunch of Chinese gold farmers coming on the server saying "You buy game gold!? China rule, USA no." and other random things..
At any rate, the password would have to be changed quite often, and possibly guarded by something on-site that requires 50+ posts or so to be able to see.