The first and biggest problem of the CastleVille rideable horses is that they cost Crowns (aka real money) to purchase and they’re not cheap either: the cheapest horse is 200 Crowns, while the most expensive Princess Horse will cost you 300. Even more, this appears to be a limited time offer, which makes things even harder.
Personally, I would suggest against hurrying to buy a rideable horse in CastleVille for so much money – it’s very unlikely that Zynga will completely remove this feature from the game and probably rideable horses will be made available in the future, either as rewards or available for regular coins, which sounds a lot better from the start. Of course, it’s a risk and you might end up having no horse to ride, but at least you’ll save some money.
Now regarding the use of the rideable mounts themselves, it’s pretty obvious: your avatar can mount the horse and then travel faster from one point to another in the kingdom. And look incredibly cool, of course!