Panic attacks on weed

Hi, just started smoking again after a long break, my use has always been pretty intermittent though. Had a panic attack again last night after hitting the bong, it's not the first time was very uncomfortable.
So I got some weed through my flatmate a couple of weeks ago and had a few smokes in my pipe ok but wanted to get more high so used my bong one night I didn't even have that much about 1 1/2 bowls and I went upstairs to watch a film but 15 mins or so later it's like holy shit I'm high! Had to pause the film and go downstairs to my room and freaked out for a while but was able to calm myself down eventually with some porn lol.
So decided to be more careful I'd count how many tokes I had, just a couple in my pipe to start with and gradually increase, so was able to do this fine after a few sessions with a manageable amount of anxiety (pot always makes me anxious but I tolerate it as it's a lot a fun). So tried the bong again tentatively, first 1 toke (they're always big tokes in the bong there's no real other way to use it) then next session just 2, was fine and so on.
Then last night had a few tokes before dinner had a delicious meal at KFC (K fri is so intensely delicious when I'm stoned) and afterwards had another bong hit started watching film and same thing happened after 15 mins this wave of anxiety washed over me, my throat felt constricted it's hard to describe but the body sensations I was having was freaking me out, I was in this state for a good couple of hours. Had to rely on porn again even though wasn't in the mood to distract me and calm me.
I'll never say a bad word about porn again it was my saviour in those 2 scary moments. I wonder if I should give it up I feel like much as I try to be resist the idea it seems that I just can't handle pot.
But here's the problem: it's so much fun. Like I've had a lot of fun with it aside from those 2 scary sessions mostly it's been great. I went to Europe in 2016 for first time and when I was in Paris scored some weed and had the most wonderful time in Paris high most of the time, same in Amsterdam it really enhanced my enjoyment of my holiday.
It feels like this weed is different I've smoked weed many times before in my bong in the past and having 2 panic attacks in close proximity is rare. I'm on Antabuse to stop me drinking and I was wondering if combining them might be having adverse reaction but can't find anything on google about that.
I dunno what to do, maybe I should just give it up but I don't want to as I keep saying it's a lot of fun, but I don't feel I need it to be happy. Maybe just stick to the pipe, I really wish I could just smoke it like most and be stoned and happy without the cursed anxiety jumping along for the ride.
I just find pot has such a strong effect on me.