Do I have to reload my data every time?

No, you don’t! When you select a project from the Recent Projects(/Boards/Filters) subheading, the data associated with that project is not reloaded from your Jira instance but is instead retrieved from your browser cache. This is done to reduce server load and decrease the wait time. So if you've made structural changes to the board or think your data is getting out of sync, it's best to do a fresh reload by choosing your project from the "All Projects" section of the dialog box. For these same reasons, Analytics does not look for new projects, boards, and filters every time you open this dialog box, so if you’ve added a new project, board, or filter, you will need to hit the “Reload” button at the top right to re-retrieve your data from Jira.