Work done by a force is defined as the product of the component of the force in the direction of the displacement and the magnitude of this displacement ($W = Fd \cos \theta$) . If there is no displacement ($d = 0$), there is no work done even if the applied force is large. Thus, when a man pushes hard against a rigid wall and fails to displace it, the work done by him on the wall is zero .