Make sure we cleanup the manifest files at the right spot of the CI process. When not building a PR, behavior is different. Signed-off-by: Anas Nashif <anas.nashif@intel.com>