I reviewed your code. There was some misunderstanding I think about how to rebuild widgets :)�
I patch your version so the toolbar is only responsible of its own layout, and the ChangingButton too.
I can not commit in your repo since I do not have proper rights :)