Crumbling behind wall

Could be some sort of structure movement due to moisture and drying if wood been used as a frame for wall or cracking of brick/block work.

I assume the wall are plasterboard, Does it sound hollow or any new/additional movement to the face of the wall when pressure applied. if the board had been attached using DD method, the dabs may have dried up and not adhered properly. these can break away and cause sound like crumbling bits dropping down etc. plasterer would be able to advise.

To be perfectly honest get a builder in to take a first-hand observation guessing won't get it fixed

Best way to find out is to get a plasterer to cut out a small access at the bottom of the stud wall, and there you'll see the aftermath of the "noise". Best if you point out where on the wall you may have heard the noise. They could also use a hole saw (the bigger the better view you'll get)., but you may not necessarily see beyond that (i.e. how is the plasterboard fixed to the wall itself) This way, the plasterer can then place back the cut out board and skim and make good. This is not cheap and you will have to pay at least a day rate plus any materials used, but this will depend on the tradesmen (could be 1/2 day rate). Hope you get it sorted and put your mind to rest