1. For investors (not speculators) it is very hard to own gold because you cannot attach a logical value to it. Unlike stocks or bonds, gold has no cash flow and has a negative cost of carry — it costs you money to hold it. It is only worth what people perceive it to be worth right now.
2. The gold ETF SPDR Gold Shares (GLD) is the sixth largest holder of physical gold in the world. If its holders decide to sell (or are forced to sell; think of hedge-fund liquidations), who will they sell it to?
3. In the past, gold had a monopoly on the inflation and fear trade. Not anymore. Now you have competition from Treasury Inflation Protected Securities (TIPS), currency ETFs, short US treasury ETFs, etc. (If you want to know more, I make this case in my book)
4. If, because of points two or three above, gold fails to perform as expected, the perception that gold is worth something may start disappearing.
5. Over the last 200 years, gold was really not a good investment. It may have a day in the sun, but it may not. And the cost of being wrong is fairly high.
The best way to deal with the risks of dollar devaluation and high inflation — with a much lower cost to being wrong — is, instead, to own stocks of companies that have pricing power of their product. When inflation hits, they will be able to raise prices and thus maintain their profitability. Also, companies that generate a large portion of their sales from outside the US will benefit from the declining dollar.
There is a wild card in the price of gold, though: China. If it decides to switch partially from owning US Treasuries to owning gold, the price of gold will skyrocket. (John Burbank made this case at the Value Investor Congress in Pasadena in May).