Yes, you’ve figured it out correctly. @Kelownagurl did a great summary of app and web differences and she mentions photo captions there.
I have photos in my profile that might seem strange to someone without seeing the captions, but I’m hesitant to remove them as I think they’re useful, when you can read the caption. It’s too bad they can’t be viewed in both versions. Let’s hope it’s something that’s on the to-do list for 2022.