It might be Gutenberg generating a preview? The shortcode's output also came above the page's content, not in the space where the shortcode sat. Like it had altered the main query.