Expanding on what mickhah has suggested, before the extrusion, select the face centers along the edges. Delete these. Now, merge those vertices together. Now extrude the faces you want.