Generally when we look at seeds, grains, legumes, etc, we should keep in mind that the plants producing these don't want them to be eaten unless they can be swallowed whole in such a way that they pass undigested. Therefore the plants will always put some kind of poison in them, to deter animals from eating them in such a way that they are destroyed and can't germinate.
Plants also do this with their leaves, roots and all other parts of the plant that are not the fruit or pollen/nectar.
Domestication of plants has meant that we can choose from among those plants which have the lowest toxin loads in their wild forms, and reduce them even more through careful selective breeding.
However, I think we should still exercise caution when consuming these foods, and we should probably limit their intake.
Apple seeds won't harm you much if you eat the seeds from one apple, particularly if you eat them by accident, since you might not even be crushing the seeds with your teeth. If you're eating the seeds that have been taken from a few thousand apples, however, and not actually eating the apples in themselves, you could be in trouble.
