I realized why twitching is happening. Ik handle is controlling rotation of the joint and when arm is going to different position joint rotates. When joint is moving up ik handle makes it rotate too to make joint chain face ik handle's end point. So there are two incoming connections that are making joint to rotate that probably "fighting" each other.