U-boot only sets up one ethernet port, so you should not have to have the second port hooked up to work.
Do you have an the am335x evm-sk, does it behave the same way using the TI-SDK prebuilt images with the network setup you have?
another thing to try is to use wireshark on the linux PC and see which MAC address is showing up as part of the ARP process at the beginning of the ping sequence.