Mad Rummy Rules Was I Wrong To Do What I Did?

Was I Wrong to do what I did? - mad rummy rules

Last night, my husband and his brother sat at the kitchen table drinking a beer. About 12 at night, when my husband one evening and went to bed. His brother and I drank and played some rummy. Approximately 2 hours, my husband down the stairs all excited and said we made too much noise to sleep. All our boys were in the room next to us and none of them woke up. He was furious that his brother was still in our house, and I have not lied. He thinks that if he went to bed that his brother had disappeared. It is wrong of me to stay and play cards with his brother?