There is a part of the bible that is a collection of poems and songs. I have not gone through all of them, concerning the references to joy, but there are a couple of themes, so far.
Awareness of God's presence is a common theme. Realizing his nearness and how he is active on behalf of people bring joy. Seeing, hearing and walking with God are obviously different than relating with people. We connect with God by focusing our hearts.
Another means of knowing and understanding god comes from read and meditating on the Scripture. But knowing is not enough. People are designed to live in agreement with Scripture, Obedience brings joy. Joy because people are flowing with their design.