What "group"?
If you want to start yet another thread about post-TEOTWAWKI, then "being stalked/cased by a group with NV" would be a consideration, and in that end times situation, you wouldn't be wasting the power to run IR spotlights, natch.
OP posted "The scenario I'm thinking of is if power is out for a while (no street lighting), being able to see what is going on around the yard or street.", where the most realistic concern is not not Mad-max style organized gangs of murderous raiders., but rather a couple of guys in a pickup truck going around stealing generators at 4AM.
My point was directed at a very specific statement you made about people "seeing" your IR. The only way they are seeing it is if they have NV. If you're talking about seeing the cameras, well then either you missed them and they're already on your property, or they're far enough away they still have surprise on their side.
If you're not outside confronting someone by the time they spot your cameras, you're too late.