I was out by my shed yesterday and had some sawdust fall on me, looked around and saw 2 holes being actively dug out by the bees on one of the support beams for my shed's roof. I counted 9 entry holes, all the way from top to bottom. I'm guessing the whole piece of wood is hollow on the inside at this point. It's cedar and it certainly has not been stained in at least a decade.
The other support beam had only 2 or 3 holes in it, so they've probably been at this for a long time. I'd kind of hate to kill them or relocate them because it's only like 30 feet from my garden and they're still good pollinators. The shed is pretty old and a bit rickety at this point, so maybe I'll just let it go this year.
Dealing with it in autumn seems like the best bet. I had seen 2 of them flying around the shed for at least a month before they started digging new holes, so it might just be 2 females. The shed itself has a fair amount of surface damage over a lot of exterior, a lot of wood gnawed off in a lot of places, a few holes going exterior to interior, vines growing inside of it, etc. The last 2 owners really neglected it, in fact the upper part of the shed is a cob-webbed mess of stuff that the original owner had left there when he sold it over a decade ago. It needs a lot of work regardless.