I doubt anyone would consider Harry Potter a literary classic. But it greatly depends on who defines the word 'classic' and who defines which books fulfill the criteria. I might even take it is as far as to saying: "who cares?". A 'classic' is wholly an arbitrary thing. Some will certainly perceive Harry Potter a classic; others will not. How do you objectively determine who is right? That group which has the most members?
I belong to the latter group. The Harry Potter series is a great series because it reaches the imaginations of both adult and child, but in it's quest to do so, it takes the safest possible path. Harry Potter is mainstream, and not in any way provocative (ignored the few ultra right-wing Christians, who think it promotes witchcraft) or groundbreaking. Therefore it deserves the popularity it has received, but not a spot on the shelf alongside Ulysses, War and Peace, 1984 - or even the Lord of Rings for that matter.