You want to leave J11 open, otherwise the RZ board will back feed 5 volts to the host PC on the VBUS pin. Remember, gadgets are not supposed to be driving the VCC line in a USB connector.
After you plug it in, after you do a "ifconfig -a", what do you see? Do you also have the kernel networking TCP/IP components enabled in the kernel?