GPS tracking?

Ok, so one of the neatest features of my iPhone is the GPS with the little blue dot telling me where I am. :cool

I've heard all kinds of things here and there about Facebook accounts, and the new Gmail "Buzz" making it possible for people to keep track of their friends, not just in terms of what they are doing on line, but where they actually are.rolleyes

So here is my question. Does this mean that ANYONE who knows my phone number can somehow access information about where I am at any given time?:confused

My concern I think is obvious, that would mean that all those unscrupulous telemarketers who call me to tell me "there is a problem with your credit card" would now be able to determine when I am not home.:major