Statistics and the independence of syntax and semantics
I know we're moving on to animal communication, but I had a comment on last week's materials that I felt should be made. It seemed to me that the "take-away" lesson we converged on during the course of our discussion of the LSLT excerpts was that Markovian processes, whether hidden (over POS tags) , high order (lots of history), or both, were dismantled by Chomsky as implausible accounts of natural language phenomena. This is undoubtedly true, but not even remotely controversial for most folks who are interested in statistical models of language. What I thought was also quite evident from reading LSLT but wasn't really addressed was the fact that Chomsky also argues for independence of meaning and grammaticality (I know this sounds so obvious as to almost be another example of the inanity of people who "work on statistics"). However, the implications of this claim are actually extremely precise for any statistical model. Specifically, we know that the independence of two events A & B has the following properties:
P(A,B) = P(A)P(B)
P(A|B) = P(A)
P(B|A) = P(B)
Therefore, if we are interested in "grammaticality" and are working with a model which conflates meaning and grammaticality (ie, P(A,B)), we must immediately doubt whether we can really even address the later without factoring out the former. Conversely, if we have a model which predicts both, we would expect their relationship (if Chomsky is right) to conform approximately to the relationship expressed in (1).