I checked the oleander plants to find not just one but several of the polka-dot wasp moth caterpillars, At this stage though they are commonly called the oleander caterpillar (Syntomeida epilais). As you can see from the photos below, they eat the entire leaf and can easily strip the leaves of an oleander bush in a few days.

If you see them, the IFAS site says their hairs are not stinging so handling them is not painful. I didn’t bother to find out if the hairs sting or not. I sprayed the entire bush to make sure I didn’t miss any of the younger stages hiding under a leaf somewhere on the bush. I have other plants in the garden specifically for feeding caterpillars, and I don’t want to harm any of the butterflies or bees in the garden, so I sprayed carefully making sure to avoid spraying anything other than the oleander bushes.