I have been able to duplicate this error 3 times now in EE v2.3.1.
Here is how I am duplicating the error:
1. Create your custom field set
2. Create the channel with your custom fieldset and status
3. Publish a new entry and start modifying all the publishing tabs and field layout. I deleted pretty much all the tabs and re-created them with new names. I made sure to include all the required fields and saved the layout. Refreshed, and everything is fine.
4. Go back to the channel fields administration page and add some new fields.
5. Go back to the publishing page to organize the newly created field and this is what I found.
The only fix I have found is to delete the channel and recreate it, which deletes all my layout settings. This is a huge pain. I have tons of issues rearranging fields and tabs with every version, however this bug is the worst yet, as it breaks the page.
Everything is fine until I add more fields after creating the channel. The site I am building now is rather large and quite complicated with close to 100 custom fields which makes it pretty hard to get every single field right without having to delete the channel and recreate the tedious layout every time. I realize I could grab the layout data from the db, which I likely will to avoid doing work over and over.