Generally speaking, the risk of nipple confusion is highest from birth to six weeks of age. It is not actually a case of confusion of the nipples, but rather more one of laziness. It takes less work to drink from a bottle's nipple than from a mother's nipple. If he is already fine with drinking from a bottle occasionally and returning to the breast without problems, you should not have a problem going back to work and breastfeeding.
The main thing I recommend is to use a newborn (slow flow) teat on your baby’s bottle. Milk is much easier to drink from a bottle than it is from a breast, and you don’t want your baby to develop a preference for the bottle. Also, it is very easy to over-feed a breast-fed baby with a bottle since the milk comes out so much more quickly. Using a newborn teat will make the milk come out slower so that your caregiver doesn’t accidentally give your baby more milk that he needs.
Pumping and working can be stressful, so try to find some things that help you relax: play some music; kick off your shoes; use a hands-free pumping bra so that you feel “productive” while pumping; close your eyes and think about your baby’s smiling face. And don’t forget to take pride in what you are doing: you are taking good care of your baby! Both you and your baby will benefit from your efforts to pump at work!