Supplementary material for article "A.N. Chernodub, D.V. Nowicki, Norm-preserving Orthogonal Permutation Linear Unit Activation Functions", submitted to ICANN'2016.