Protons and neutrons - collectively called nucleons - belong to the family of hadrons. They are built of quarks and bound by the strong force that is mediated via gluons. The force is acting between two quarks and has an unusual behavior: It is very small when the quarks are at close distance and increases as the distance grows and then remains constant even if the quarks are removed further and further from each other.
If one attempts to separate a quark-antiquark pair, the energy of the gluon field becomes larger and larger, until a new quark-antiquark pair can be created. As a result, one does not end up with two isolated quarks but with new quark-antiquark pairs instead. This absolute imprisonment of quarks is called confinement. One of the greatest intellectual challenges of modern physics is to understand confinement not just as a phenomenon but to comprehend it quantitatively from the theory of the strong force.
Another puzzle of hadron physics addresses the origin of the hadron masses, i.e. of the particles composed of quarks. In the nucleon, less than 2% of the mass can be accounted for by the three valence quarks. Obviously, the bulk of the nucleon mass results from the kinetic energy and the interaction energy of the quarks confined in the nucleon. Physicists believe that new experiments exploiting high-energy antiproton and ion beams will also elucidate the generation of hadronic masses.
To get a more detailled overview about a particular topic please follow one of the links:
Search for exotic particles and
Study in-medium effects of hadronic particles
Generalized parton distribution, Drell-Yan
Measurement of nuclear properties