Research Biography of Paul Smolensky

Paul Smolensky has pursued a unified theory of the mind/brain inspired by a fundamental analogy with modern physics: just as quantum and classical theory do for the physical world, Smolensky holds that for cognition, connectionist and symbolic theory provide valid formal characterizations at micro- and macro-levels, respectively.

A research program in which connectionist and symbolic theory collaborate to form a multi-level analysis of the mind/brain was laid out in Smolensky’s influential article, .On the proper treatment of connectionism. [4]. At a time when connectionism and symbolic theory were overwhelmingly seen only as competitors, this image of an integrative research program defined a unique alternative vision, and established the ground on which Smolensky and a small number of like-minded cognitive scientists have worked ever since. The Integrated Connectionist/Symbolic (ICS) Cognitive Architecture, constructed by Smolensky and collaborators, is developed in depth and broadly applied in the comprehensive collection, The Harmonic Mind [10].

Defending the importance and viability of the connectionist substrate in this architecture led Smolensky into a lengthy debate with Jerry Fodor and his collaborators [13]. In the ICS Architecture, connectionism does not “merely implement” a classical symbolic theory — rather it furnishes ineliminable subsymbolic accounts of processing and novel explanations of central aspects of higher cognition, such as unbounded productivity; further, it forms the basis of a new theory of grammar. Through his extended debate with Fodor, Smolensky has brought to the attention of philosophers the important foundational implications of crucial technical aspects of connectionist theory [10: §§22.23].

In the ICS Architecture, the abstract high-level computational properties of the connectionist micro-level theory are formally described by a symbolic macro-level theory. In three major contributions to the seminal 1986 Parallel Distributed Processing (PDP) volumes, Smolensky first showed how mathematical analysis of high-level properties of neural computation could make substantial connection with symbolic theory: using vector calculus, neural activation patterns can be identified with conceptual-level symbolic description [2]; spreading activation can be analyzed as optimization of well-formedness or Harmony, a principled form of statistical inference [1], and (with David Rumelhart, James McClelland, and Geoffrey Hinton) a particularly flexible kind of schema-based reasoning [3]. Many neural network theorists have exploited optimization analysis techniques such as these and others introduced independently around this time by S. Grossberg, J. J. Hopfield, Hinton & T. Sejnowski, and others. The work building on Smolensky’s emphasized optimization as a key link between neural and symbolic computation.

Substantially extending the vector analysis of distributed representations in 1988, Smolensky introduced tensor analysis into connectionist theory, establishing a formal isomorphism between high-level properties of certain distributed connectionist networks and symbolic computation [10: §§5, 7, 8]]. To promote further integrative cognitive research based on other types of formal high-level analysis of neural computation, in 1996 Smolensky wrote extensive pedagogical and integrative material for Mathematical Perspectives on Neural Networks [7], which he edited with Michael Mozer and Rumelhart.

A particularly crucial test area for a unified connectionist and symbolic theory is language, especially aspects related to grammar. This has been the focus of Smolensky’s work since 1990. Collaborative work with syntactician Géraldine Legendre showed that tensorial distributed representations combined with optimization entails Harmonic Grammar, a new framework in which symbolic linguistic representations are assigned numerical well-formedness values (the Harmony of the connectionist representations that realize them). The grammar is realized by the connection weights of a network, the outputs of which are optimal — maximal-Harmony — representations [10: §§11, 20].

Smolensky’s most influential work arose from what was intended to be a confrontation in 1988 with Alan Prince, a preeminent phonologist also known as a critic of connectionist research on language. Smolensky and Prince found a strong basis for collaboration in their shared respect for formal analysis and explanation in cognitive science. Addressing phonology, and taking Harmonic Grammar as a starting point, they built Optimality Theory (OT), which adds strong principles of restricted, universal grammatical explanation [5, 8]. OT provides the first formal theory of cross-linguistic typology, postulating that all grammars are built of literally the same set of well-formedness constraints — but, crucially, these constraints, like those of connectionist networks, are conflicting and violated in well-formed structures. A possible grammar is precisely a hierarchical constraint ranking, in which each constraint has absolute priority over all lower-ranked constraints combined. This crisp theory of constraint interaction enables a singular precision of grammatical analysis and explanation, as demonstrated in a series of penetrating papers by Prince and collaborators [15].

After the circulation of Prince & Smolensky’s OT book manuscript in 1993, OT rapidly became a dominant theory of phonology [14;], the first major challenger to Chomsky and Halle’s serial symbol-manipulation framework which had provided the field’s foundation since the 1960s. Smolensky’s own contributions to linguistic theory since 1993 have addressed a range of formal issues in phonology, syntax, and semantics, such as the grammaticization of scales, the supra-linear interaction of local constraint violations, underspecification, and the structure of features in phonological representations [10: §14]; the initial state of the learner [10: §12]; and the grammatical role of competition among interpretations in comprehension [6, 11].

In collaboration with a number of leading linguists, Smolensky has played a major role in the expansion of OT outside phonology. Work in 1993 with Legendre on the typology of grammatical voice and case marking systems was the first published OT work outside phonology [10: §15]; their joint research on the typology of wh-questions [10: §16] also played a ground-breaking role in creating the field of OT syntax [12]. Bruce Tesar’s work with Smolensky in 1993 on learnability of OT grammars remains the cornerstone of that active area [9]. Experimental research with Peter Jusczyk established a new paradigm for probing young infants’ knowledge of phonological grammar [10: §17]. Collaborations with Lisa Davidson [10: §17] and with Suzanne Stevenson [10: §19] pushed OT towards a theory of performance in phonological production and syntactic comprehension, respectively.

As a result, OT is not simply a theory of phonological competence; it is the grammatical component of an emerging unified theory of linguistic cognition, integral to not just phonological, syntactic and semantic knowledge, but also to the theory of performance, both production and comprehension, and to learning. It is a high-level description of connectionist processes using spreading activation to perform sub-symbolic maximization of the well-formedness of distributed linguistic representations [10: §21].

Outside his own research, Smolensky has also worked to promote a formal, principle-based, aggressively interdisciplinary vision of cognitive science, strongly influenced by his training with Rumelhart and McClelland. This vision has driven his efforts as two-time President of the Cognitive Science Society, as President of the Society for Psychology and Philosophy, as lecturer at the Linguistic Society of America Summer Institute and Annual Conference, and as Chair of the Cognitive Science Department at Johns Hopkins University, where he has built a strong PhD program that is training a new generation of innovative, multidisciplinary cognitive scientists. Smolensky’s students are playing a leading role in extending linguistics to embrace the full cognitive science of language; they include M. Goldrick and J. Hale as well as the speakers featured at the Cognitive Science Society’s Rumelhart Prize Symposium, L. Davidson, A. Gafos, B. Tesar, and C. Wilson.

[1] Smolensky, Paul. 1986. Information processing in dynamical systems: Foundations of harmony theory. In Parallel distributed processing: Explorations in the microstructure of cognition. Vol. 1, Foundations, David E. Rumelhart, James L. McClelland and the PDP Research Group, 194-281. MIT Press.
[2] Smolensky, Paul. 1986. Neural and conceptual interpretations of parallel distributed processing models. In Parallel distributed processing: Explorations in the microstructure of cognition. Vol. 2, Psychological and biological models, David E. Rumelhart, James L. McClelland and the PDP Research Group, 390-431. MIT Press.
[3] Rumelhart, David E., Paul Smolensky, James L. McClelland, and Geoffrey E. Hinton. 1986. Schemata and sequential thought processes in parallel distributed processing. In Parallel distributed processing: Explorations in the microstructure of cognition. Vol. 2, Psychological and biological models, David E. Rumelhart, James L. McClelland and the PDP Research Group, 7-57. MIT Press.
[4] Smolensky, Paul. 1988. On the proper treatment of connectionism. The Behavioral and Brain Sciences 11, 1-74.
[5] Prince, Alan, and Paul Smolensky. 1993/2004. Optimality Theory: Constraint interaction in generative grammar. Technical Report, Rutgers University and University of Colorado at Boulder, 1993. Revised version published by Blackwell, 2004. Rutgers Optimality Archive 537.
[6] Smolensky, Paul. 1996. On the comprehension/production dilemma in child language. Linguistic Inquiry 27, 720-31. Rutgers Optimality Archive 118.
[7] Smolensky, Paul, Michael C. Mozer, and David E. Rumelhart, eds. 1996. Mathematical perspectives on neural networks. Erlbaum.
[8] Prince, Alan, and Paul Smolensky. 1997. Optimality: From neural networks to universal grammar. Science 275, 1604-10.
[9] Tesar, Bruce B., and Paul Smolensky. 2000. Learnability in Optimality Theory. MIT Press.
[10] Smolensky, Paul, and Géraldine Legendre. 2005. The harmonic mind: From neural computation to Optimality-Theoretic grammar. Vol 1: Cognitive architecture. Vol 2: Linguistic and philosophical implications. MIT Press.
Additional Citations
[11] Blutner, Reinhard, and Henk Zeevat, eds. 2003. Pragmatics in Optimality Theory. Palgrave Macmillan.
[12] Legendre, Géraldine, Sten Vikner, and Jane Grimshaw, eds. 2001. Optimality-Theoretic syntax. MIT Press.
[13] Macdonald, Cynthia, and Graham Macdonald. 1995. Connectionism: Debates on psychological explanation. vol. 2 Blackwell.
[14] McCarthy, John J., ed. 2004. Optimality Theory in phonology: A reader. Blackwell.
[15] Prince, Alan S. 2006. The structure of Optimality Theory.