Manager offered to take me off the on-call rota 'because of the little one'. I didn't ask

It was meant kindly, I think. In my one-on-one today he said I could step off the on-call rotation for a while, 'with the little one and all'. Nobody else on the team with kids has been offered this. Two of them are fathers. On-call is hard with a preschooler, I won't pretend otherwise. But it's also where the visible work happens. The people who handle the bad nights are the ones remembered at review time. I said I'd think about it. Part of me wants to say yes. Part of me is annoyed that it was offered to me and not to them. Has anyone taken a step back like this and managed to step back in later?