At what distance?
A 4" group at 100 yards is NOT going to remain so at 300.
It is impossible to give you pointers on shooting a rifle without one-on-one coaching. You have been given the advice to get good ammunition to eliminate that factor. You als NEED to dump anything that says Barska on it. They are cheap crap.
You MUST have a high quality sight if you intend to be able to shoot with any degree of accuracy. At a minimum that means a Burris Fullfield or Leupold VX2.
Besides that, you need to find someone who is a competent marksman that can teach you to shoot from various positions (sitting, standing, prone, etc) without the use of a bench.
People who use their rifles exclusively for hunting are, in general, some of the worst marksmen when real accuracy is demanded. Don't take it personally, but that's how it is.
There are several shooters here that can easily do a 3 to 4 inch group from the prone position using iron sights at 400 yards. Find them, swallow the ego, and learn from them.
Thanks for the advice. I shoot better with my AR and a red dot than my Remington and that Barska 8x32. I guess it's time to invest in some good glass. Any scopes you would recommend around $300-$400.