What to do about visiting puppies

My neighbor recently got two small puppies that will probably be large dogs,from the looks of them.
They are always fooling around in my front yard. I have lots of saplings and small plants that could easily be chomped or stomped on.
I am really upset that now I can't even have a front yard to garden in safely. He let's these guys out in the morning and they are free to roam all over the place. They come into my yard and mess in my flower beds.
What should I do without making an enemy out of my neighbor?
My back yard is securely fenced for my dogs to play safely in.
I plant my daturas and bulbs in the front yard because they are poisonous. I also have a Carolina Jasmine that is poisonous growing where the pups can now get into it.
I fenced off my two tiny trees from Arbor Day Society because one bite and they will be goners. I don't want to have to barrier garden in the front yard also.
I am also afraid he may not have them vaccinated and we have wild animals that could carry rabies in the woods across the street.
He has no collars on them, no fenced in area at all. Obviously he has no experience with dogs or he would know they do not stay put if not contained.I could spend $3000 fencing in my front yard also, but why spend the money when he is just as capable of putting up his own fence.
It is not the puppies fault and I feel mean shooing them away when they look at me and wag their tails and keep coming toward me! Puppies are just too cute!