There are multiple points here.
1. There is a force exerted by the rubber ball on the table during the collision. If the ball hits the table in vertical direction, then that force is balanced by the reaction force from the ground. Hence table stays in equilibrium. As the forces are balanced, it continues to be in its state of inertia of rest. Thus there is very little loss of energy in the collision (due to small in-elasticity).
If the ball hits the table from the side, then the friction force between the legs of the table and the ground act against the force exerted by the ball on the table in the horizontal table.
If the momentum transferred from the ball to the table is high and the force exerted by the ball is higher than static friction on the four legs of the table, then the table will move a little.
2. Also, the mass of the table is much higher than that of the ball. So significant momentum transfer will occur ,only if the velocity of the ball is very high in the horizontal direction.