A notice box is a block with a title line and basic text. Unlike the "Section with Title" object, the notice box cannot have other objects added into it. The title and body text can both be customized by clicking directly on the text after clicking the "Edit" button in the header of the website.
The notice box appears on the right in this image, while a "Section with Title" with a text block added to it is on the left.
The notice box can be found under the "Generic Items" tab when editing your website.