[Vincent Wu]

Dear Sirs,

I am currently testing a comstomized SOC with ARM1176JZF-S core, 32KB-cache on Linux-2.6.31.2, and I encountered a cache-DMA data coherence problem in SD host driver. The phenonmenon is random data error while file copying. The error occurs once or twice during 100 iterations of 1GB-file copying test.

The implementation of the DMA operation has followed the suggestion of the kernel document “DMA-API.txt”, that is, the DMA_TO_DEVICE synchronisation is done after the last modification of the memory region that will be read by the DMA device, and the DMA_FROM_DEVICE synchronisation is done before the driver accesses data that was changed by the DMA device. However, the data coherence still cannot be guaranteed.

I referred to an ARM technical support knowledge article at http://infocenter.ar...qs/ka14041.html . For the ARMv6\v7 speculative execution, the DSB and DMB instructions are necessary to ensure data access correctness. The Linux ARMv6 cache operation has utlized DSB instruction, “mcr p15, 0, r0, c7, c10, 4” in the kernel source file, “linux-2.6.31.2/arch/arm/mm/cache-v6.S” to handle the speculative execution issue, but that cannot fix the stale data problem in my case.

Finally, I added a DMB instruction, “mcr p15, 0, r0, c7, c10, 5” after each cache invalidation and clean operation, and then the data error problem does not occur anymore. But I am still not sure if my modification is reasonable. I have some questions about the DSB and DMB instructions as listed below:

1. The Linux kernel utilizes DSB instruction only. Is the DSB instruction sufficient for ensuring the cache coherence?

2. If the DSB instruction is sufficient, why is the stale data still written into the SD card until I added a DMB instruction after the cache invalidation and clean operation?

3. Although the ARM technical article has described the functions of DSB and DMB instructions, I still cannot tell the critical difference between the two instructions. Could anyone explain?

Thanks for helping me!!

This post has been edited by Vincent Wu: 30 September 2011 - 06:13 AM

[isogen74]

http://forums.arm.co...807-dmb-vs-dsb/

[Vincent Wu]

引用框（isogen74 ＠ 30 September 2011 - 12:53 PM）

http://forums.arm.co...807-dmb-vs-dsb/

Dear iosgen74,
Thank you for your reply.

Actually, I had read the article which you posted before I posted this issue. In my understanding, DSB is to synchronize both data modification and instruction stream, and DMB is to synchronize data modification only, so a DSB instruction after cache invalidation and write-back is sufficient.

However, in my case, I need to add an additional DMB instruction to ensure the data coherence, and it conflicts my previous understanding. Does DMB execute something that DSB does not perform?

This post has been edited by Vincent Wu: 03 October 2011 - 02:59 AM

[isogen74]

引用

Does DMB execute something that DSB does not perform?

No, the DMB is a sub-set of the DSB behavior. However if you are performing cache maintenance operations, then a DSB is what you want to use, not a DMB. You want to ensure that the instruction's behavior has completed executing (i.e. cache maintenance complete) before executing following instructions. The DMB instruction is only really concerned with explicit memory accesses (loads and stores), which is why it doesn't help for cache maintenance.

The following guide provides more information on barriers, and gives more examples:
http://infocenter.ar...ookbook_A08.pdf
Cheers,
Iso

This post has been edited by isogen74: 03 October 2011 - 08:04 AM

[Vincent Wu]

isogen74, on 03 October 2011 - 08:02 AM, said:

No, the DMB is a sub-set of the DSB behavior. However if you are performing cache maintenance operations, then a DSB is what you want to use, not a DMB. You want to ensure that the instruction's behavior has completed executing (i.e. cache maintenance complete) before executing following instructions. The DMB instruction is only really concerned with explicit memory accesses (loads and stores), which is why it doesn't help for cache maintenance.

Sounds like I had gone a wrong way. Thanks for your explanation!

For the Linux version earlier than 2.6.34, I found a patch to solve DMA-cache coherence problem due to speculative execution.
http://nv-tegra.nvid...8e09f1ee#patch2
This patch seems able to work fine in my case.

[isogen74]

No problem; glad of be of some help.

Thanks for posting the link to the patch; I wasn't aware of that one =)

Iso