I missed my boyfriend's graduation and he feels disappointed

I missed the biggest day in my boyfriend’s life and cannot stop crying and feeling absolutely horrible. My boyfriend had the graduation ceremony yesterday that marked the start of this huge career he’s been training for since he was little. It was really big it was held at a stadium and everyone had to wear these very fancy outfits they had to buy for like 900$ only to wear once to maybe twice ever. Only two people were allowed to go though so his mom and brother went while his dad and I were going to watch over zoom. He sent the zoom link three days ago with the time but not the date and I had it marked on my calendar for the 12th already (turns out that was the date for something else he told me about). Yesterday he was also teaching a class so I also figured he was doing that all day. Then he texted me while I was working that he was going to the bar later with his classmates and offered to pick me up and go with him later that night. I was confused because I thought he was talking about the students he was teaching. But I said sure and so left work as soon as I could and got all dressed up to go. Then he called me and said he was coming to pick me up and asked how was it watching his graduation link. My stomach dropped and it was then I found out it was today and that I missed it. I said no and tried to explain to him that I got the date wrong and felt absolutely horrible. I felt like I was going to cry but I didn’t want to make it sound like he had to console me because it’s me who messed up and he’s got a whole thing about that I genuinely do respect and understand. He then said he felt really disappointed because that day isn’t going to ever happen again and I had only one job which was to join the zoom link and that he now wasn’t sure if he wanted me to now come with him to the bar for (the turns out) after party. I said I totally understood and wished him a good night, apologized again, and told him I would treat him out this week. After that I just went to my mom and cried. I’ve been with him on this journey watching him grow into this incredible being and have watched him work restlessly hard in both times of joy and times of deep pain and worry. He’s the person I admire most in this world and he means so much to me. If I knew it was today I genuinely would have taken the day off. Then he calls me and says he’s here to pick me up which I was surprised by because I thought he wanted to go alone. So I quickly go and try fixing my makeup while I’m walking outside and he’s in his car. I knock on his door and give him a big hug and say congratulations. He seems a little withdrawn so I step back and check in to see what is happening. And he tells me he wants to talk first. I totally expected that eventually so I go sit in his passenger’s seat and listen and he just tells me that he’s still not sure if he wants to take me and needs a few minutes to think. At this point I’m confused because I thought it was a no yet he showed up. So I ask if I should get out of the car or something while he’s think and he says it’s up to me. A couple of quiet minutes go by and then he decides he no longer wants me there so I go back in my house and see him drive off. This point I’m a total wreck and sobbing and throwing up and start having a panic attack which I haven’t had since I was a teenager. My mom is with me then my brother comes and console me. But this entire time I feel like I don’t deserve consolation and need to accept the emotional weight of this consequence to me not being there today. They tell me that I shouldn’t beat myself this much over it and remind me that they also thought it was another day. I am frantically going over the texts on my phone to him trying to see where I missed but couldn’t find the date still. He called me at the bar and said he was too harsh and regretted not taking me and that he wasn’t having a good time. Then he calls me again asking if he can stay over. I say I wish but I think we both should get some good sleep. He ends up showing up anyways and we exchange a big hug then we decide I’ll just sleep over at his place. Things go well. But then then it dawned on me that this has been a pattern where I make a small mistake that leads to him feeling really hurt and then (I hope on accident) punishing me by treating me a little too harshly afterwards and then being super lovey and then going back to harsh if I mention I felt hurt by how harsh he was. It messes so much with my heart. This happened on my birthday and when my Grampa died too. And when I’ve told him the way he reacted hurt me too he always had said at the end of the day it’s me who made the mistake and needs to take responsibility for it. Which I agree with but also I feel like both feelings can be true at once. He calls me again today and says he changes his mind and he’s now unsure if he can trust me to support him in the future since I have the right intentions but missed doing this very important action.