Wammy's House was founded because of L, so that they could find his successor. The orphanage he grew up at was a normal one. The first child said to be taken to Wammy's was A, and the second, Beyond Birthday. It was said that A killed himself under the pressure of succeeding L, and Beyond ran away for the same reason. If A was the first, and Beyond was the second, and they were both going to try and succeed L, that should make it obvious that he didn't actually live it Wammy's. However, since Wammy's was founded BECAUSE of L, he can still be included in this club.

I actually didn't know that.

I though it was Canon that Wammy's House existed before L with their goal changing to also finding a successor for L but having originally training facility for especially gifted children to become experts in different fields before L ever came into the pictire with them still doing that even while also looking for a successor to L.