Boyfriend of three years still refuses to give me his passcode to his phone. Is this a red flag?

So like I said, my boyfriend and I have been together for three years. I have asked him multiple times if I could have his phone passcode, and he says that I don’t need it. I trust that he would never cheat on me, but it is very weird to me that he hast to keep his phone password a secret. I gave him my phone passcode a long time ago, but for me it’s not a big deal. I know we’re two different people but I can’t help but be sketched out that he wouldn’t also share that with me. I’ve expressed my concern and he just shrugged it off like it’s nothing.I know that this is a minuscule problem but I’m just trying to protect myself.