Has anyone had to deal with a carpenter bee issue before?

I was out by my shed yesterday and had some sawdust fall on me, looked around and saw 2 holes being actively dug out by the bees on one of the support beams for my shed's roof. I counted 9 entry holes, all the way from top to bottom. I'm guessing the whole piece of wood is hollow on the inside at this point. It's cedar and it certainly has not been stained in at least a decade. The other support beam had only 2 or 3 holes in it, so they've probably been at this for a long time. I'd kind of hate to kill them or relocate them because it's only like 30 feet from my garden and they're still good pollinators. The shed is pretty old and a bit rickety at this point, so maybe I'll just let it go this year.