If it’s terribly affecting your mental health, drop it, but personally I’d stay in them.
I took 4 AP classes and 5 paid college credit classes and was able to graduate a year early. I went to a local school and graduating a year early would’ve saved me around $40,000, even with my scholarships (I decided to only graduate one semester early).
I’d also look into what classes you are taking and how the schools you’re thinking of applying to take those credits. A lot of my AP classes ended up only applying towards general electives. Those electives are typically pretty easy to take in college, so if you’d rather spend more time on extracurriculars now, that’d be perfectly fine. Despite my straight As, I wasn’t able to get into some of my state schools because I didn’t have any extracurriculars, so it’s definitely a balance between the two.