Histone variant H3.3 differs from the canonical histone H3.1 by only five amino acids, yet its chaperone death domain-associated protein (DAXX) can specifically recognize H3.3 over H3.1, despite having a large DAXX-interacting surface on the H3.3-H4 heterodimer common to that on the H3.1-H4 complex. This observation gives rise to the question of, from the binding energy point view, how high binding specificity may be achieved with small differences of the overall binding energy for protein-protein interactions in general. Here we investigate the mechanism of coupling of binding specificity and affinity in protein-protein interactions using the DAXX-H3.3-H4 complex as a model. Using a multi-scale method, we found that the hydrophobic interactions between DAXX and the H3.3-specific region contributed to their initial binding process. And the structural flexibility of the interacting partners contributed to the binding affinity after their encounter. By quantifying the free energy landscape, we revealed that the interaction between the specific residues of H3.3 and DAXX decreased the encounter barrier height while the folding of H3.3-H4 and DAXX increased the depth of the free energy basin of the final binding state. The encounter barrier height, which is not coupled to the thermodynamic stability of the final binding state, had a marked effect on the initial binding rate of flexible histones and chaperones. Based on the energy landscape theory, we found that the intrinsic binding energy funnel of this uncoupled recognition process was affected by the structural flexibility and the flexibility modulated the degree of coupling between binding specificity and affinity. Our work offers a biophysical explanation of the specific recognition between the histones and their chaperones, and also extends the use of energy landscape theory for understanding molecular recognitions in general.