In the old Note layout you could delete an Attachment from the Note thread itself, with the new Note view you need to first go to Note Info to be able to remove the attachments. We've heard from clients that they would like to delete the attachment from the Note thread itself.
Please add your support to this feature if you'd like to see this re-implemented.