Dr.Polygon, I worked with what you had and could not find a problem. But I started deleting faces in the area where you are having the problem and used the Append Tool to do the faces again. I found that seemed to solve the problem. I just did one side. Here is what it looks like now. And I will attach the file I have worked with.
Hope it helps!!
Challenge me and I will try. Because If I should fail, failure also has a prize (Knowledge).