What I would do is:
a) use an external event line from your FPGA to the CPU. So you can trigger the DMA controller from the FPGA. No polling needed. No waste of time.
b) use ONE FIFO inside the FPGA. Each FIFO entry consists of 2 informations:
1) the data byte
2) the channel number to which the data byte is belonging.
The DMA will dump these data into memory, and inside your software, you can read the channel numbers and sort the data to the right place.
regards
Wolfgang