I have scrapped this answer -- please consider my other answer instead. I'm leaving this here for reference.
You should simply use Mathematica's built-in random number generator. For example:
n = 50;f[] := (n /Plus @@ #) # & /@ & /@ Table[Random[], {i, 1, n}, {j, 1, n}];This function (with no argument) f[] will return such a matrix. For n=50, we can test the timing.
Timing[For[i = 1, i <= 10^4, i++, f[]]][[1]]/10^4(* Output: 0.0013197685 *)The average instance of $f[]$ for me takes about 0.0013s. So at scale, if you had $10^6$ of these and $n=50$, it's somewhat feasible (about 21 minutes total to generate them all).
It might be useful to know what your n (or N) value is. If n=5, this is a much different question than n=1000, in terms of how practically we can produce $10^6$ of these $n\times n$ arrays.
I think both of the answers here have still misconstrued your use of l however. I will be updating this answer if I can incorporate that constraint into this.
(Edited due to misreading the question.)
