Well, the tech is for every website to be used, as a visitor to the site that may or may not benefit me. I think that was the reason for the same-origin policy and is, probably the source of concern of the OP.
Personally, I use NoScript and RequestPolicy to deal with it. After all, just because JavaScript exists does not mean I want any random website to execute arbitrary code on my machine (especially not with WebRTC).
Experience has shown that many users just grant such access when prompted, without thinking about it.
Prompts like that also do absolutely nothing to stop malicious use, hidden under a facade of legitimacy. For example, somebody could put together a demo purportedly showing "serverless pure JavaScript P2P file sharing in the browser" solely to trick people into using something harmful. (I'm not saying that's necessarily going on here, of course.)
http://www.w3.org/TR/webrtc/.
There are ways to turn it off on your browser, but why would you? :). The tech is yours to be used.