Restoring the widgets annot still shows "Sign here" for stamps but not for freehand

I’m any not be setting the annot of the signature properly to the widget.
Currently I’m doing
WidgetAnnot.annot = annotation.

You may not be able to set the .annot property directly.
It also looks as if the annotation is attached to the widget by position if there is not annotation set.