Template based protein structure prediction (commonly referred to as homology or comparative modeling) uses knowledge of solved structures to model a protein sequence's native or true fold. First, a parent structure is found and then a template structure is built by mapping the target sequence onto the parent structure. This putative structure is refined using a combination of backbone moves, side-chain packing, and loop modeling. Template based protein structure prediction has always held great promise to produce atomically accurate models close to the native conformation based on two major assumptions. First, similar sequences exhibit similar protein folds. Second, soluble proteins populate a discrete fold space with many representatives already solved in our Protein Data Bank (PDB). Ironically, beginning so close to the native structure is also the primary source of problems confronting this method and is the reason for the lack of progress in this category of structure prediction. In this review, the general concepts and procedures for template based structure prediction are outlined based on the following topics: sequence alignment, parent structure selection, template structure building, refinement, evaluation, and final structure selection. Then, a description of established software and algorithms is provided where the advantages and limitations of the different methods will be pointed out. This is followed by a discussion of the developments in template based structure prediction up to the 7th Critical Assessment of Structure Prediction meeting. Lastly, we will address the increased difficulty in improving templates that start so close to the native structure, and discuss the improvements needed in this field.