Due to limitations in their API, we have to poll and check for creation and completion events. (imported from commit be65e507fac16a7f8ad3dc57b2af9c4b98aacf39)