These are all questions only you can answer. I choose 100 because that seemed like a manageable number and that's how many other top writers follow. When I trimmed my follower list, I cut the people whose writing didn't resonate with me.
How you go about your work on Medium is, essentially, up to you. You don't have to trim your list. I did because it was getting out of control.