I have listened to this song in the context of the other songs that make up the story of "The Wall". The meaning of the song, of which the posted lyrics here make up only a part, is not what many think it to be; one of shrugging off the limitations imposed by authorities and going your own way. The "wall" of which the authority figures are just another brick of is the wall that the protagonist has built up around himself as a shield from a world that hurts him. His cruel teachers, his lost father (lost in a war when he was very young), the taunts of other children. This was compounded by his mother, who channeled all of her own fears and desires into him, helping him build his wall. The wall has grown so tall that not only can nobody else get in, he can't get out. Each brick in the wall further isolates him (and by extension, each individual, building up walls between us all).