mjhjmtu

This question is motivated by one of the problem set from this year's Putnam Examination. That is,

Problem. Let $S_1, S_2, dots, S_2^n-1$ be the nonempty subsets of $1,2,dots,n$ in some order, and let
$M$ be the $(2^n-1) times (2^n-1)$ matrix whose $(i,j)$ entry is
$$M_ij = begincases 0 & mboxif S_i cap S_j = emptyset; \
1 & mboxotherwise.
endcases$$
Calculate the determinant of $M$. Answer: If $n=1$ then $det M=1$; else $det(M)=-1$.

I like to consider the following variant which got me puzzled.

Question. Preserve the notation from above, let
$A$ be the matrix whose $(i,j)$ entry is
$$A_ij = begincases 1 & # (S_i cap S_j) =1; \
0 & mboxotherwise.
endcases$$
If $n>1$, is this true?
$$det(A)=-prod_k=1^nk^binomnk.$$

Remark. Amusingly, the same number counts "product of sizes of all the nonempty subsets of $[n]$" according to OEIS.

POSTSCRIPT.

If $B$ is the matrix whose $(i,j)$ entry is
$B_ij = q^#(S_icap S_j)$
then does this hold?
$$det(B)=q^n(q-1)^n(2^n-1-1).$$

Here is a proof that follows my argument at https://artofproblemsolving.com/community/u432h1747709p11384734 as closely as possible. Sorry for its length, much of it due to exposition of matrix folklore. (If you know this, scroll all the way down to Theorem 1, or maybe to the two paragraphs preceding it.)

We are grown-ups and don't need to index the rows and the columns of a matrix by numbers. We can pick any set $P$, and define a $P times P$-matrix (say, with rational entries) to be a family $left(a_p, qright)_left(p, qright) in P times P in mathbbQ^P times P$ of rational numbers indexed by pairs $left(p, qright) in P times P$. When the set $P$ is finite, such a $P times P$-matrix always has a well-defined determinant, which can be defined by
beginequation
det left( left(a_p, qright)_left(p, qright) in P times P right)
= sum_sigma in S_P left(-1right)^sigma prod_p in P a_p, sigmaleft(pright) .
endequation
Here, $S_P$ denotes the set of permutations of $P$, and the notation $left(-1right)^sigma$ stands for the sign of a permutation $sigma$. If you think that this is problematic because the sign of a permutation is usually defined using a total order on $P$, then take a break and convince yourself that it does not actually depend on this total order. (This is proven in maximum detail in the solution of Exercise 5.12 of my Notes on the combinatorial fundamentals of algebra, 7th of November 2018.) Determinants of $P times P$-matrices behave as nicely as determinants of "usual" $n times n$-matrices, and arguably even better, since it is easier to define minors and cofactors when the rows and columns are indexed by an arbitrary finite set. Do keep in mind that the rows and the columns must be indexed by the same set; we cannot define the determinant of a $P times Q$-matrix for different $P$ and $Q$, even when $left|Pright| = left|Qright|$.

Of course, an $n times n$-matrix (for a nonnegative integer $n$) is the same as an $N times N$-matrix for $N = left1,2,ldots,nright$. Thus, our notion of $P times P$-matrices generalizes the classical notion of $n times n$-matrices. This generalization "respects" standard concepts like determinants -- e.g., the determinant of an $n times n$-matrix does not change if we instead consider it as an $N times N$-matrix. Does this generalization exhibit genuinely new behavior or does it just add convenience? Arguably it's the latter, because conversely, you can turn any $P times P$-matrix (for any finite set $P$) into an $n times n$-matrix. Namely, if you have a finite set $P$ and a $P times P$-matrix $A = left(a_p, qright)_left(p, qright) in P times P in mathbbQ^P times P$, then you can set $n = left|Pright|$ and fix a bijection $phi : left1,2,ldots,nright to P$, and form the $n times n$-matrix $A_phi := left(a_phileft(iright), phileft(jright)right)_left(i, jright) in left1,2,ldots,nright times left1,2,ldots,nright in mathbbQ^n times n$, and observe that $A_phi$ is "basically the same as" $A$ except that the rows and the columns have been reindexed. We shall refer to this operation (going from $A$ to $A_phi$) as numerical reindexing. So $P times P$-matrices differ from $n times n$-matrices only in how we index their rows and columns. Nevertheless they are useful, because often there is no canonical bijection $phi : left1,2,ldots,nright to P$ (exhibit A: this Putnam problem), and we are algebraists and want to make as few non-canonical choices as possible.

Whenever $P$ is a finite set, we can multiply two $P times P$-matrices: The product $AB$ of two $P times P$-matrices $A = left(a_p, qright)_left(p, qright) in P times P in mathbbQ^P times P$ and $B = left(b_p, qright)_left(p, qright) in P times P in mathbbQ^P times P$ is defined to be the $P times P$-matrix $left(sum_r in P a_p, r b_r, q right)_left(p, qright) in P times P in mathbbQ^P times P$. This multiplication behaves just as nicely as usual multiplication of $n times n$-matrices. In particular, $detleft(ABright) = det A cdot det B$ whenever $A$ and $B$ are two $P times P$-matrices. And of course, you can prove this by numerical reindexing, because numerical reindexing preserves both products and determinants (assuming, of course, that you pick one bijection $phi : left1,2,ldots,nright to P$ and stick with it).

If you remember the definition of triangular $n times n$-matrices, then its generalization to $P times P$-matrices is straightforward: When $P$ is a totally ordered finite set, we say that a $P times P$-matrix $A = left(a_p, qright)_left(p, qright) in P times P$ is lower-triangular if
beginequation
a_p, q = 0 qquad textfor all left(p, qright) in P times P text satisfying p < q ;
endequation
and we say that a $P times P$-matrix $A = left(a_p, qright)_left(p, qright) in P times P$ is upper-triangular if
beginequation
a_p, q = 0 qquad textfor all left(p, qright) in P times P text satisfying p > q .
endequation
Again, lower-triangular $P times P$-matrices become lower-triangular $n times n$-matrices upon numerical reindexing, as long as we make sure to pick our bijection $phi : left1,2,ldots,nright to P$ to be an order isomorphism (which, by the way, is unique). So we can derive properties of lower-triangular $P times P$-matrices from analogous properties of lower-triangular $n times n$-matrices by numerical reindexing. In particular, we can thus show that the determinant of a lower-triangular $P times P$-matrix equals the product of its diagonal entries; i.e., if $A = left(a_p, qright)_left(p, qright) in P times P$ is a lower-triangular $P times P$-matrix, then $det A = prodlimits_p in P a_p, p$. The same holds for upper-triangular $P times P$-matrices.

Now, let's not forget about the question.

Fix a positive integer $n$. Let $N$ be the set $left1,2,ldots,nright$. Let $P$ be the set of all the $2^n - 1$ nonempty subsets of $N$. (We shall later totally order $P$, but for now $P$ is just a finite set.)

We shall also use the Iverson bracket notation, which will save us some ampersands and braces. This allows us to rewrite the definition of $A_ij$ in your question as $A_ij = left[ left|S_i cap S_jright| = 1 right]$. But as we said, we drop the numerical indexing, and instead define a matrix whose rows and columns are indexed by $P$ (that is, by all the nonempty subsets of $N$). Thus, the claim of the exercise rewrites as follows:

Theorem 1. Let $A$ be the $P times P$-matrix $left( left[ left|S cap Tright| = 1 right] right)_left(S, Tright) in P$. Then,
beginequation
det A = left(-1right)^left[n neq 1right] prod_k=1^n k^dbinomnk .
endequation

The key to proving this is the following simple fact:

Lemma 2. Let $S in P$ and $T in P$. Then,
beginequation
left[ left| S cap T right| = 1 right] = sumlimits_U in P left(-1right)^left left|Uright| cdot left[ U subseteq S right] left[ U subseteq T right] .
endequation

This, in turn, shall be derived from the following binomial identity:

Lemma 3. Let $k$ be a nonnegative integer. Then,
beginequation
sum_i=1^k left(-1right)^i-1 i cdot dbinomki = left[k = 1right].
endequation

Proof of Lemma 3 (sketched). If $k = 0$, then this equality holds for obvious reasons (indeed, the left hand side is an empty sum and thus vanishes, while the right hand side vanishes since $k = 0 neq 1$). Hence, for the rest of this proof, we WLOG assume that $k neq 0$. Hence, $k$ is a positive integer. Thus, $k-1$ is a nonnegative integer.

It is well-known that $sumlimits_i=0^n left(-1right)^i dbinomni = left[n = 0right]$ for every nonnegative integer $n$. Applying this to $n = k-1$, we obtain $sumlimits_i=0^k-1 left(-1right)^i dbinomk-1i = left[k-1 = 0right] = left[k = 1right]$. Multiplying both sides of this equality by $k$, we obtain $k sumlimits_i=0^k-1 left(-1right)^i dbinomk-1i = k left[k = 1right] = left[k = 1right]$ (where the last equality sign is easy to check directly). Hence,
beginalign
left[k = 1right]
&= k sumlimits_i=0^k-1 left(-1right)^i dbinomk-1i
= sumlimits_i=0^k-1 left(-1right)^i cdot underbracek dbinomk-1i_substack= left(i+1right) dbinomki+1 \ text(by a simple computation) \
&= sumlimits_i=0^k-1 left(-1right)^i cdot left(i+1right) dbinomki+1
= sumlimits_i=1^k left(-1right)^i-1 cdot i dbinomki
endalign
(here, we have substituted $i-1$ for $i$ in the sum). This proves Lemma 3. $blacksquare$

Proof of Lemma 2 (sketched). If a subset $U in P$ does not satisfy $U subseteq S$, then the Iverson bracket $left[ U subseteq S right]$ is zero and thus renders the whole product $left(-1right)^left left|Uright| cdot left[ U subseteq S right] left[ U subseteq T right]$ zero. Similarly, this product also is rendered zero when $U in P$ does not satisfy $U subseteq T$. Thus, the product $left(-1right)^left left|Uright| cdot left[ U subseteq S right] left[ U subseteq T right]$ is zero unless $U in P$ satisfies both $U subseteq S$ and $U subseteq T$. Hence, the sum $sumlimits_U in P left(-1right)^left left|Uright| cdot left[ U subseteq S right] left[ U subseteq T right]$ does not change its value when we restrict it to those $U$ that satisfy both $U subseteq S$ and $U subseteq T$ (because all the addends that we lose are zero anyway). In other words,
beginalign*
sumlimits_U in P left(-1right)^left left|Uright| cdot left[ U subseteq S right] left[ U subseteq T right]
&= sumlimits_substackU in P; \ U subseteq S text and U subseteq T left(-1right)^left left|Uright| cdot underbraceleft[ U subseteq S right]_=1 underbraceleft[ U subseteq T right]_=1 \
&= sumlimits_substackU in P; \ U subseteq S text and U subseteq T
left(-1right)^left left|Uright|.
endalign*
Now, let $K$ be the intersection $S cap T$, and let $k = left|Kright|$. Then, the sets $U in P$ that satisfy both $U subseteq S$ and $U subseteq T$ are precisely the nonempty subsets of the $k$-element set $K$. Thus, we know how many such sets $U$ exist of each size: there are exactly $dbinomk1$ ones of size $1$, exactly $dbinomk2$ ones of size $2$, and so on, all the way up to $dbinomkk$ ones of size $k$. Hence,
beginalign*
sumlimits_substackU in P; \ U subseteq S text and U subseteq T
left(-1right)^left left|Uright|
&= dbinomk1 left(-1right)^1-1 cdot 1 + dbinomk2 left(-1right)^2-1 cdot 2 + cdots + dbinomkk left(-1right)^k-1 cdot k \
&= sum_i=1^k dbinomki left(-1right)^i-1 cdot i
= sum_i=1^k left(-1right)^i-1 i cdot dbinomki \
&= left[k = 1right] qquad left(textby Lemma 3right) \
&= left[left|Scap Tright| = 1right]
endalign*
(since $k = left|Kright| = left|Scap Tright|$).
Combining what we have, we obtain
beginequation
sumlimits_U in P left(-1right)^left left|Uright| cdot left[ U subseteq S right] left[ U subseteq T right]
= sumlimits_substackU in P; \ U subseteq S text and U subseteq T
left(-1right)^left left|Uright|
= left[left|Scap Tright| = 1right] .
endequation
This proves Lemma 2. $blacksquare$

Lemma 4. (a) We have $sumlimits_S in P left|Sright| = n 2^n-1$.

(b) We have $sumlimits_S in P left( left|Sright| - 1 right) = n 2^n-1 - 2^n + 1$.

(c) We have $prodlimits_S in P left(-1right)^left = left(-1right)^left[n neq 1right]$.

(d) We have $prodlimits_S in P left|Sright| = prodlimits_k=1^n k^dbinomnk$.

Proof of Lemma 4. (a) Fix $i in left1,2,ldots,nright$. Then, exactly $2^n-1$ subsets of $left1,2,ldots,nright$ contain $i$ (since we can freely choose which of the other $n-1$ elements such a subset should contain). Of course, all of these $2^n-1$ subsets are nonempty, and thus belong to $P$. Hence, exactly $2^n-1$ subsets $S in P$ contain $i$. Hence, the sum $sumlimits_S in P left[i in Sright]$ has exactly $2^n-1$ addends equal to $1$, while all its other addends are $0$. Therefore, this sum equals $2^n-1$. In other words, we have
beginequation
sumlimits_S in P left[i in Sright] = 2^n-1 .
labeldarij1.pf.l4.1
tag1
endequation

Now, forget that we fixed $i$. We thus have proven eqrefdarij1.pf.l4.1 for each $i in left1,2,ldots,nright$.

But we have $left|Sright| = sumlimits_i=1^n left[i in Sright]$ for each subset $S$ of $left1,2,ldots,nright$ (because the sum $sumlimits_i=1^n left[i in Sright]$ contains exactly $left|Sright|$ many addends equal to $1$, whereas all its other addends are $0$). Summing up this equation over all $S in P$, we obtain
beginalign
sumlimits_S in P left|Sright|
&= sumlimits_S in P sumlimits_i=1^n left[i in Sright]
= sumlimits_i=1^n underbracesumlimits_S in P left[i in Sright]_substack= 2^n-1 \ text(by eqrefdarij1.pf.l4.1) \
&= sumlimits_i=1^n 2^n-1 = n 2^n-1 .
endalign
This proves Lemma 4 (a).

(b) We have $left|Pright| = 2^n - 1$ (since the set $P$ consists of all the $2^n$ subsets of $left1,2,ldots,nright$ apart from the empty set). Now,
beginalign
sumlimits_S in P left( left|Sright| - 1 right)
= underbraceSright_substack= n 2^n-1 \ text(by Lemma 4 (a))
- underbracesumlimits_S in P 1_= left
= n 2^n-1 - left(2^n - 1 right) = n 2^n-1 - 2^n + 1 .
endalign
This proves Lemma 4 (b).

(c) If $n neq 1$, then $n geq 2$ (since $n$ is a positive integer), and therefore both $2^n-1$ and $2^n$ are even integers. Hence, if $n neq 1$, then $n 2^n-1 - 2^n + 1 equiv 1 mod 2$. On the other hand, if $n = 1$, then $n 2^n-1 - 2^n + 1 = 1 cdot 2^1-1 - 2^1 + 1 = 0 equiv 0 mod 2$. Combining the previous two sentences, we conclude that $n 2^n-1 - 2^n + 1 equiv left[n neq 1right] mod 2$. Hence, $left(-1right)^n 2^n-1 - 2^n + 1 = left(-1right)^left[n neq 1right]$. Now,
beginalign
prodlimits_S in P left(-1right)^left
&= left(-1right)^Sright
= left(-1right)^n 2^n-1 - 2^n + 1
qquad left(textby Lemma 4 (b)right) \
&= left(-1right)^left[n neq 1right] .
endalign
This proves Lemma 4 (c).

(d) Recall that $P$ is the set of all nonempty subsets of $left1,2,ldots,nright$. Hence, among the elements of $P$ are exactly $dbinomn1$ subsets of size $1$, exactly $dbinomn2$ subsets of size $2$, and so on, and finally exactly $dbinomnn$ subsets of size $n$ (and no other subsets). Therefore, the product $prodlimits_S in P left|Sright|$ has exactly $dbinomn1$ factors equal to $1$, exactly $dbinomn2$ factors equal to $2$, and so on, and finally exactly $dbinomnn$ factors equal to $n$ (and no other factors). Therefore, this product equals $prodlimits_k=1^n k^dbinomnk$. This proves Lemma 4 (d). $blacksquare$

Proof of Theorem 1 (sketched). Define two $P times P$-matrices
beginalign*
B &= left( left(-1right)^ - 1 left|Tright| cdot left[ T subseteq S right] right)_left(S, Tright) in P times P qquad textand \
C &= left( left[ S subseteq T right] right)_left(S, Tright) in P times P .
endalign*

Lemma 2 says that $A = BC$. (More precisely, Lemma 2 says that the $left(S, Tright)$-th entry of $A$ equals the corresponding entry of $BC$ for all $S in P$ and $T in P$; but this yields $A = BC$.) Thus, $det A = detleft(BCright) = det B cdot det C$. Now, it remains to find $det B$ and $det C$.

Here we shall finally endow our set $P$ with a total ordering. Namely, we pick any total order on $P$ with the property that every pair of two subsets $S in P$ and $T in P$ satisfying $S subseteq T$ must satisfy $S leq T$. There are many total orders that do the trick here; for example, we can order them by increasing size (resolving ties arbitrarily). Now, it is easy to see that the $P times P$-matrix $B$ is lower-triangular. Since the determinant of a lower-triangular $P times P$-matrix equals the product of its diagonal entries, we thus conclude that
beginalign
det B
&= prod_S in P left( left(-1right)^left left|Sright| underbraceleft[ S subseteq S right]_=1 right)
= prod_S in P left( left(-1right)^left left|Sright| right) \
&= underbraceleft(prod_S in P left(-1right)^leftright)_substack= left(-1right)^left[n neq 1right] \ text(by Lemma 4 (c))
left( prod_S in P left|Sright| right)
= left(-1right)^left[n neq 1right] left( prod_S in P left|Sright| right) .
endalign

Likewise, the $P times P$-matrix $C$ is upper-triangular. Since the determinant of an upper-triangular $P times P$-matrix equals the product of its diagonal entries, we thus conclude that
beginequation
det C = prod_S in P underbraceleft[ S subseteq S right]_=1
= 1 .
endequation
Now,
beginalign
det A
&= det B cdot underbracedet C_=1 = det B = left(-1right)^left[n neq 1right] underbrace right)_substack= prodlimits_k=1^n k^dbinomnk \ text(by Lemma 4 (d)) \
&= left(-1right)^left[n neq 1right] prod_k=1^n k^dbinomnk .
endalign
This proves Theorem 1. $blacksquare$

In hindsight, the above proof could have been restated without introducing $P times P$-matrices, since picking a total order on a finite set $P$ is equivalent to picking a bijection $phi : left1,2,ldots,nright to P$. But I prefer to give the exercise an "invariant" formulation, and $P times P$-matrices are worth exposing anyway.

A few more telegraphic remarks:

• The decomposition $A = BC$ constructed in the above proof of Theorem 1 is an LU-decomposition.




• The proof I found first was the same recursive argument appearing a few times in the AoPS thread linked above. But the structure of the recursion foreshadowed the existence of a slicker proof -- the block-row-reduction operations felt like multiplying by a lower-triangular (in a properly chosen order) matrix. So I dug deeper and found the above.




• This reminds me of Lemma 7.1 of Chris Godsil, An Introduction to the Moebius Function.

Darij already gave a great elementary detailed exposition. I wanted to remark that all such kinds of results (including the postscript) are special cases of a classical result that (at least in the generality of semilattices) goes back to Lindström in Determinants on semilattices. For this question we are in the special case of the Boolean lattice.