You are correct about the GENERAL rule -- when you use an expression "a group [OR groups] of people" it IS the word group(s) that determines whether the verb used. That's why you say, "A large group of people WAS standing outside the door."
But "a lot of" (and "lots of") is not the same as "group".
There is a set of expressions called "deteminers" that do no function as independent nouns and so do NOT decide the number of the verb. The expressions of quantity, "a lot of' and "lots of" are examples of these expressions called "determiners."
It helps to note that this pair of expressions are informal substitutes with "much" and "many." When substituting for "much" --before nouns that are NOT counted (e.g., 'furniture', 'money') they take the singular verb form(just as "much" does). When used in place of "many" (before plural nouns), they take the plural verb form. Thus:
"There ARE a lot of books [plural] on the table." (=> response: "How MANY *are* there?")
"There ARE a lot of people [plural]." (same response; note that we treat "people" as a plural noun)
"There IS a lot of money in this wallet!" (=> "How MUCH *is* there?")
http://www.udel.edu/eli/grammar2.html
see also http://www.orangutanenglish.com/English%20Grammar%20Guide%202.htm#determiners
http://itre.cis.upenn.edu/~myl/languagelog/archives/002990.html