If you read the negative comments on APMEX, they mostly sound like BS. Shipping too slow, etc... All of the replies are that they will gladly do a return for anybody who is unsatisfied.
Buying bullion from a reputable dealer like APMEX, etc... is very safe. I know that with JM Bullion, anything more than 20 ounces of silver requires a signature on delivery. You will get what you ordered, and the price is what you lock in when you buy.
Just a comment on the 1 kilo bar that SquirtGun is considering. Yes, the premium over spot is low, and it is a good deal, but you really need to think about why you are buying silver. This thing is going to be harder to sell than 1 ounce or even 10 ounce coins, rounds, or bars. You won't be able to divide it up and sell part of it.
I would recommend starting out with 1 ounce coins (preferred) or rounds. Once you have a good quantity of 1 ounce coins, then maybe start buying 10 ounce bars as they take up less space and are easier to stack.
Again, it depends on your intent. If you think you will just liquidate all of your silver when it hits a certain price, then it may not matter. If you think you might want to cash in smaller quantities as you need cash, or use it to buy goats or bribe border guards, then smaller is better.