Once I came inside I did some research. They are known as egg pimples or just calcium deposits. The eggs that you buy at the store have already been wiped and rubbed clean. Therefore, one would never usually see them, it also doesn't occur very often in general. Sometimes they can look more like boils than sandy bits. They can also get ingrained in the shell and when the bits get wiped off, it inadvertently makes a hole in the egg. Age of the chicken is a factor as well as it's health and strain. If egg is white - the grainy bits are white, if egg is brown - bits are brown, a definitive sign that it's not insect eggs.
For hatching eggs, note that an egg with egg pimples/rough shell has a 55% lower possibility of hatching. As shown and explained on this webpage: Maintaining Hatching Egg Quality
Some info about my chickens - they are free range with a bowl of oyster pellets available at all times. I also break up egg shells and spread it around their enclosure. I'm guessing they have more than enough calcium! They are all about a year old now, chipper and healthy. The one that laid the egg is one of my leghorns. A lot of my chickens took a break from laying (winter season) and are just now starting up again. That could be the reason for the deposits showing up now. I'm thinking that after another week or two I won't see this problem until next spring - I shall let you know if I'm wrong!
While egg pimples do occur occasionally, I periodically clean out my coop and inspect my chickens - just to rule out a bug problem. If you are unsure if you have calcium deposits or insect eggs - put the bits in a plastic bag and see if anything happens. That way you know for sure!