Hi there!
To do this you need to create a custom SelectionModel, and then assign it to the annotation.
There is a sample repo here which contains some examples, specifically this one.
This guide also has some information on creating custom selection models.
Thanks,
Logan