I have explored this a little bit, and the only way I kept the Glass in one piece while scattering pieces out of it was with the Volume Builder/Mesher.
Example: (The background is set up quickly, just to have something to refract in the scene.)
Since little pieces would “melt” with the volume effect, which is an excellent effect, but I guess not what you were after, I have set up two Voronoi Fracture objects with the exact same Point Source.
The MoGraph Selection allows me to make a piece invisible while, at the same time, it is available in the other setup. (Hence the same source).
The Connect Object would NOT work with the Glass, so the Volume is used.
Let me know if you have any questions about it. I’m happy to look into it.
P.S.: for a better performance, if the glass-pane is flat, try the Volume Mesher > Adaptive set to 0.1%