Masks affect the visibility of objects within their shape. If an object is affected by a mask, any part of it outside the mask's restricted area will be invisible. The restriction area of a mask can either be determined by an image with the Mask component or a Rect Transform with the Rect Mask 2D component.
With UI, masking is primarily used for scrolling menus, so items that exist outside of the menu's area will not be visible. However, it is also used to cut out images. For example, the following image shows a cat's image being cut out by a circular mask:
You may note that the edges on the cat with mask don't look great. To avoid this, ensure that you use sprites with appropriate image resolutions and try different Filtering modes on the Sprite's Import Settings.
