It is well recognized today that mathematical and statistical models in general provide approximations rather than exact representations of real-world phenomena. From this perspective, factor analysis models can be viewed as providing approximations of the phenomena that give rise to observed correlations among measured variables. In my talk I will examine two aspects of this issue. First, I will first review some of the early factor analysis literature in an effort to describe the perspective of that era regarding the correspondence between the model and the real world, and I will discuss how that perspective may have contributed to some of the controversies in the field. Second, I will discuss implications of the approximate nature of the common factor model for the problem of estimation of model parameters. This discussion will focus on the correspondence (or lack thereof) between the nature of error in empirical data on the one hand, and assumptions of estimation methods about the nature of error on the other hand. It will be shown that this correspondence can have dramatic effects on quality of results obtained by different estimation methods.