How can I express my regret without actually saying the words "I'm sorry"?

My boyfriend (19M) and I (18M) have a really close and fantastic relationship, and we just celebrated our four-month anniversary. However, there are times when he gets annoyed, irritated, or quiet if something goes wrong at work, home, or elsewhere. When I do something that bothers him, I often try to apologize, but he responds with comments like, "You should know by now I dislike apologies." This leaves me at a loss for words—I just don’t know how to respond beyond saying I'm sorry. It's a tough situation for me, and it tends to make me shut down. I've asked him what he'd prefer I say instead, but he also doesn’t seem to have an answer. Any advice would be appreciated!