I don't think there is such a tool or paper.
I built such a tool for a compiler IR but I don't think it will fit your needs as I show strange IR nodes with compiler related information (it doesn't look like the original smalltalk code at all). The visualization is done with Roassal.
I think if you get the Smalltalk AST with the semantic analysis done, you can easily detect which messageNode will be inlined to control flow operations, and therefore have a control flow graph. You can then display it with Roassal. However, Roassal is bad at handling graph cycles so I'd recommend that you show the control flow graph but back edges.
Good luck
Cl��ment