I had similar problem with an ATSAME70 which was an M7 core. Turns out that the ethernet driver from Microchip did not invalidate cache for the ethernet MAC driver. Hence it would work some of the time and not others, like when debugging was on. Disabling the data cache on the processor would allow code to run with no errors.
Microchip told me that none of their drivers were designed to work with cache on and to run ANY sample code or drivers required instruction and data cache to be turned off. Needless to say I rewrote their drivers.
Also note the need for barrier instructions (__DSB(), __ISB()) these are often needed and vendor's drivers are missing them. For example barriers are often needed at end of interrupt handlers. Note even then some peripherals have caches in the peripherals that can not be flushed with barrier instructions, for example Flash memory.
I also ran into this as that right after writing to flash memory I would read back the wrong values due to flash peripheral caching. I had to create my own cache flush by reading large block of flash no near where I wrote, then returning to where I wrote and reading again. The peripheral driver had a cache flush that did not work, so the reading different region was the only way to ensure the flash cache was flushed.
The function below disables the cache in the core, part of core_cm7.h
/**
\brief Disable D-Cache
\details Turns off D-Cache
*/
__STATIC_INLINE void SCB_DisableDCache (void)
{
#if defined (__DCACHE_PRESENT) && (__DCACHE_PRESENT == 1U)
register uint32_t ccsidr;
register uint32_t sets;
register uint32_t ways;
SCB->CSSELR = 0U; /*(0U << 1U) | 0U;*/ /* Level 1 data cache */
__DSB();
SCB->CCR &= ~(uint32_t)SCB_CCR_DC_Msk; /* disable D-Cache */
__DSB();
ccsidr = SCB->CCSIDR;
/* clean & invalidate D-Cache */
sets = (uint32_t)(CCSIDR_SETS(ccsidr));
do {
ways = (uint32_t)(CCSIDR_WAYS(ccsidr));
do {
SCB->DCCISW = (((sets << SCB_DCCISW_SET_Pos) & SCB_DCCISW_SET_Msk) |
((ways << SCB_DCCISW_WAY_Pos) & SCB_DCCISW_WAY_Msk) );
#if defined ( __CC_ARM )
__schedule_barrier();
#endif
} while (ways-- != 0U);
} while(sets-- != 0U);
__DSB();
__ISB();
#endif
}