Not Toxic, But Can Still Cause Harm to Bodies
No, diamonds are composed mostly of carbon which is not poisonous. They are indigestible so while not being toxic, they could potentially cause harm to your digestive system if you were to actually consume one. True, too, of an animal.
Diamond is insoluble in any solvent present on Earth. When consumed, the solid form only will pass through the system, without any change. Nothing will mix up in the body. But, it could choke or block a system because of its hard structure, depending on its size and the object in which it is mounted.